Criminals demand money from Albion Online
It was an interesting weekend for Albion Online; full of twists and turns, server restarts and ransom demands. To get a better feel for just how serious things got, the CEO and Founder himself took to the forum to explain what was happening: Bercilak → DDOS & Blackmailing saying that a ransom demand was given and is purely asking for money. Now while it is nicely spelled out on the forums, if you were in game during the DDoS attacks you would have been greeted with the above server wide message (which was immediately ridiculed in chat and many vowed to start a DDoS Criminals guild). While DDoS (Distributed Denial-of–Service) attacks have been plaguing MMO’s for years, the community responses are more unique in this particular case as some are doubting that there even is a DDoS happening (language warning: angry tweet) and what we are seeing is a False Flag attempt to shift blame off the servers and database issues that have been going on since launch. Is there something going on; would Albion Online try to make up a fake event to hide their internal issues? Let’s examine the evidence and see if we can come to a conclusion.
Server issues: Weak hardware or something else?
Albion Online launched July 17th and like many other founders I was ready and waiting to login with anticipation of gathering, crafting, making silver and buying some very cheap gold on the exchange (I was luckily able to get a few thousand gold at 2 silver each). As I pressed the login button, created my character and entered the world everything seemed ok; I saw a hundred characters or so all making a mad rush to the next zone to either get on a boat to go head to their desired starter islands or begin their journey by chopping some rough logs and pounding raw stones. I, myself, had to go to the boat and head to Forest Crossing as that is where my guild was starting and so I clicked on the Boat Captain, selected Forest Crossing and one loading screen later there I was; things were off to a good start for me as it was about 3 minutes into the launch and I made it to my starter island…and then it happened. I walked to the next zone line and went into perma-walk with no loading or zoning out.
It happens just like an epidemic
The funny thing about server issues like the one in question, is it does not affect everyone at the same time; unlike a server shutdown or restart where everyone is booted off immediately, this instead spreads to users as they do an action that triggers the problem which in this case was zoning to a new area. The chat responded just like people at the start of a zombie apocalypse:
- First they yell out some symptoms: “I can’t zone”, “My guy is stuck walking”
- Then the call to order: “Stay calm, let the server catch up”, “Just give it time”,
- As things get worse they come up with solutions: “Reboot the server”, “Everyone log off”, “Stop spamming chat it makes the servers worse”
- Then when hope is lost: “WIPE WIPE WIPE”, “WIPE HYPE”, “WIPE THE SERVERS”, “RELAUNCH”
As more user’s got stuck in loading transition screens causing everything to grind to a halt, the servers were restarted. About 35 minutes later, I was back in and the server seemed ok; I even got my founder bonus finally and the founder mail. This was not to last though, as it happened again and further throughout the day much to the irritation of all the players, myself included as this was not the time to fall behind.
The official story
So what happened? We all finally got a full breakdown of what was officially causing all the server troubles in a post from the CEO of Albion Online himself: Update on Server Issues and Compensation:
- Specific zones being overcrowded → I can attest to this, when I played during launch everything was ok the first few minutes until the starter zones filled up with hundreds of players and then it all went downhill. If you managed to zone into a low populated area everything more or less went fine. The second you zoned back into a main city like Lymehurst it was back to black loading screen for 10 to 15 minutes at a time.
- Solution: Create sub-zones in the main cities for banks and auction houses to spread players out; this is a solid proposal even if it means more loading/zoning.
- Lag in large scale fights → This seems to be more closely related to physical server limitations, as it was explained there is a hard limit as to how many players can interact in a zone before degradation happens. The more abilities being used means more updates from the server to all clients and this makes sense from a technical perspective to be a limiting factor.
- Solution: Try to spread players out by creating pvp objectives in multiple places throughout the game world. Another solid plan as it not only means reducing the issue but increases potential for smaller guilds not to get steam rolled by larger guilds; if there is only one castle available in a day the biggest guild will just send all their players to it and that is not very fun or rewarding; who wants to split loot with that many players anyway, even if we were all in the same guild.
- Server reboots → This is the heart of the “black loading screen forever” issue we saw on launch, the main database stops responding to queries which then queues up all commands and actions until you need a server reboot to clear it all out. The most troubling part of this issue is that they do not know what the root cause is which means it will be a recurrent and random issue, there is also fear (and we players have seen it) that when the database and game servers goes out of sync, a rollback might unintentionally happen since the save data is also queued up.
- Solution: Apache Cassandra has been directly contacted and is working with the dev team. This is as good as it gets in a situation like this and I hope for the best; which is a root cause analysis with a proper solution.
The above sums up all the issues we have seen so far regarding gameplay since launch but how does this stack up against the current DDoS attack symptoms? Users were experiencing the following this weekend:
- Users cannot login and are getting disconnected
That does not match any of the symptoms we saw during launch but does match the effects of classic DDoS attacks; before we declare that Bercilak is indeed telling the truth about the ransom though, we have one more point to examine: Compensation.
Many of you are rightfully upset and angry about the server issues post-launch. We would like to apologize for this and can assure you that everybody in our team is fully dedicated to fix them as soon as possible – our consumption of coffee certainly has increased by a factor of 10, and we have already signed up our lead programmers for the cast of the next season of The Walking Dead as they’d be able to play the zombie roles without needing any make-up. Joking aside: Everybody who has played between release and today will be compensated with an additional 7 days of premium time. In the unlikely case that we do not improve the server situation over the next 2 weeks, additional compensation will be awarded.”
If you read this you might see the extremely important part, if not I will point it out: “In the unlikely case that we do not improve the server situation over the next 2 weeks.”. The importance has to do with the time frame, this was posted July 31st and gives the Albion development team 2 weeks to execute measures to fix the issues before having to make good on their additional compensation promise. We are only on the first weekend after the announcement so why would Albion create a false flag this early into the schedule when they have another full week of time left? It wouldn’t make any sense, so I have to say that the most likely answer is “Bercilak is telling the truth“. They did get a ransom notice and the probable cause is the stance they took on Gold trading. We got our first hint that this was related to gold sellers from this post
Eltharyon → “This is most likely in response to our recent actions against gold sellers to coerce us to not interfere with their illegal business.”
Gold Sellers, Gold Sellers everywhere
World of Warcraft did more than just bring video games and MMO’s to the general public (ie. casuals); it also bought the Gold Seller businesses front and center with annoying gold seller spam in world chat, obnoxious whispers/tells and the most concerning of all: Account hijacking with a side of credit card fraud. Anyone playing Albion Online has seen the increase in gold seller spam in cities and whispers/tells in their chat so it would be no surprise that the recent changes done to stem gold sellers rustled more than a few jimmies. The details of the changes can be found here but are centered around making it harder to trade gold from person to person and instead make the only place for trading gold to be the Exchange Market. I commend Albion for making these changes as gold sellers give a negative gameplay experience for everyone in the game; whether it is bots farming resources, bots spamming chat channels and whispering direct messages or just the influx of private transactions that would have normally gone through the official Exchange messing up prices; it is all detrimental to the long term economy of the game and the sooner it can be stopped the better it will be for all real players.
Leave any questions or comments below.
Good Luck in all things,
- [Post of the Month] End of an Era: GameMaker 1.X Sunset coming at end of July 2018
- [Made In GameMaker] PurpleBit Surfing Cow by Heavy Sheep Games
- [Indie Interview] Heavy Sheep Games with Luiz Gustavo
- [Made in GameMaker] GameDevDan vs Life Launches on Steam
- [Made in GameMaker] Bounce Rescue by Bitecore Studios