When to start thinking about scalability? [closed]

Question

I'm having a funny but also terrible problem. I'm about to launch a new (iPhone) app. It's a turn-based multiplayer game running on my own custom backend. But I'm afraid to launch.

For some reason, I think it might become something big and that its popularity will kill my poor lonely single server + MySQL database.

On one hand I'm thinking that if it's growing, I'd better be prepared and have a scalable infrastructure already in place.

On the other hand I just feel like getting it out into the world and see what happens.

I often read stuff like "premature optimization is the root of all evil" or people saying that you should just build your killer game now, with the tools at hand, and worry about other stuff like scalability later.

I'd love to hear some opinions on this from experts or people with experience with this. Thanks!

Answer 1

It's actually quite an easy choice.

Right now, you have zero users, and scalability is not a problem.

Ideally, you want to reach the point where you have millions of users, and scalability becomes a problem.

Right now, you don't have a scalability problem; you have a number-of-users problem. If you work on the scalability problem, you will not fix the number-of-users problem, which means you will have solved a problem you don't have yet, and you will not have solved the problem you do have. The most likely result is that your product won't make it, and all your work will be for nothing.

If you work on the number-of-users problem, you will solve a problem you have right now, and then you can focus on the next problem, which might be scalability.

The nice thing about scalability problems is that, by definition, having them usually means business is pretty damn good, and this in turn means you can afford to spend money on optimizing for scalability. You don't go from zero users to ten million overnight, and if you keep an eye on the system's performance, you will have plenty of time to optimize when the time comes.

Of course it helps to keep scalability in mind while writing the code you need right now, but it doesn't make a lot of sense to spend dozens or even hundreds of man-hours on a feature of which you don't know if you'll ever need it, and the most likely scenario is that you won't. Right now, your main concern is to ship. What happens after that; well, you can worry about that later.

Answer 2

TL;DR You should think about scalability before the first line of code is written.

First things first. Scalabilty != Optimization

You should think about scalability before the first line of code is written. This doesn't mean that you build out some massive infrastructure on the off chance your game might be a hit. Thinking about scalability means:

• Making sure the code is written so it scales. I've seen plenty of projects where no thought was given to needing to scale. The results are a codebase that won't scale regardless of the hardware you throw at it, or is prohibitively expensive to scale.
• Figure out your scaling strategy. Have a plan on how to support all the users. You have a MySQL db, are you going to shard it or cluster, or something else. Strategies like sharding require some forethought because it places requirements on the architecture. Clustering, less so. Are you supporting sessions, and how will sessions react with multiple front end servers. Will you need sticky sessions, in your load balancer.
• Figure out implementation strategy. Are you going to use AWS for scaling. Can you leverage any products or services that give you dynamic scaling out of the box? This also involves understanding your costs.

BUT it sounds like you already have a codebase. The question now is when to start scaling. This is completely dependent on your code.

If your code lends itself to scaling then you have the hard part done. You can get an AWS account, spin up servers as needed and away you go.

If your code doesn't scale or has bottlenecks then you have work to do. You need to identify what your bottlenecks are and fix them. The "when" is really hard to know. Some services plateau, some rise steadily, and some explode. Deciding when to throw resources at something like scaling is usually a function of business and it's evaluation of the risks.

In your position, I might release as a "beta" and manage user expectations. That way I can get the product out, and see how it unfolds.

Answer 3

There's no reason to optimize until you know optimization is needed. How do you know optimization is needed? You measure.

Assuming that your server has some sort of web-based interface, you can simulate a lot of users by using tools such as Apache JMeter. Learn how to use the tool, then start stress-testing your back-end. You should be able to learn enough to know what the limits are to your system. You can then combine that information with the number of users you have and the average number that are running at one time, to decide when to scale up.

Answer 4

So, there are two times you should think about scalability.

First, it should be gently pondered before you write a single line of code. This is to ensure you don't write yourself into a scalability hole and to make sure your code is instrumented to give you the measurements you need for the second time.

The second time to consider scalability is in enough of an advance of things becoming unacceptably slow. That means you need to know what "too slow" means and how your thing reacts under load. If you have a service whose driver (probably qps) increases by N% per month, you have rather different times from "95% of machine resources consumed" if your resource usage is load-squared or linear in load.

With a turn-based game, you ought to have a decent margin of safety (you are probably not having a single game world, and if you do, there is probably an internal geometry, meaning you do not have "everyone interacts with everyone each turn" problems).

Without knowing specifics, I would take one or two days to think about where you have scaling issues and what possible strategies you have for resolving them. But, this is important, Think About. Not do, just think (and document). Unless you have scalability problems that start manifesting at a few hundred users, you should then have time to check load and spin up more back-end resources.

Answer 5

From your description it sounds like there are two possible outcomes:

• The game is a failure and then you don't care.
• The game is successful and then your backend won't be able to deal with the load and the outcome would be a failure.

Hmmm.

Here are some questions to ask yourself:

• How many users can your current backend handle with acceptable performance?
• Do you have some sort of plan to limit the impact to current users if you're seeing some sort of rapid growth (e.g. temporarily pull the game from the app store)
• How quickly can you come up with a better backend if you are successful?
• What are the business implications of waiting. Can you feed yourself? What are the risks?
• What are the business implications of releasing now given the answers to the question above.

The answer to your question should become evident once you consider these. No expert can tell you what to do without more information as every system is different and every business is different.

Answer 6

Your server will be used interactively by users. This means that latency is affecting user-experience in a very profound way. Bad latency always results in bad user-experience.

At least do some ad-hoc load-testing as described by Bryan.

---

A more serious approach

Do some simulation runs and find out what latency does to your user-experience (either using a network delay simulation or just sleep() inside of your application). Find out at what latency it's getting noticable, getting annoying and getting unusable.

Then comes the first step in the direction of optimizing. Decide on a SLA for your server : e.g. at worst 10% calls with annoying latency and 1% calls with unusable latency. With those limits you can use load-testing to find out how many users your server can support.

Pure throughput testing without measuring latency (or at least manually using the server during the load test) is not that useful because it doesn't tell you whether the measured throughput numbers result in bearable user-experience.

A very nice presentation about measuring latency by Gil Tene: http://www.infoq.com/presentations/latency-pitfalls

Answer 7

At the business requirements stage, which then is used to establish a common understanding for performance for all elements downstream such as architecture, ops, development, QA and monitoring in prod. If you don't establish a common understanding for what is required up front then you will have each of the parts of the organization making assumptions about performance (or not thinking about them at all) when engaging in particular tasks across the life cycle of the application. This is true whether you are engaged in waterfall, short waterfall, agile or whatever the development methodology of the moment is hot on the resume keyword list.

Performance and scalability is hard. Functionality is easy. Poor scaling code will grow to fill whatever resource pool you provide to it, so shifting the cost bubble by purchasing larger hardware only takes you so far before you have to either fix the inefficient code or purchase ever more hardware. Leaving this to last in priority is also very costly. There are architecture and design decisions which are made early in the life cycle of the application which may have to be completely reversed in order to hit a late arriving requirement related to performance - Think of a high performance sports car manufacturere having to switch from aluminum to carbon fiber late in the design cycle to hit a power/weight ratio related to performance and how this impacts tooling, training, the construction of the car, etc...

Ask the architects, developers and ops people in your organization what are the performance requirements for the application. If these are not captured from the business then don't be surprised if you receive different answers (or no answers) from different individuals even within the same group. Those "assumptions" always come back to smack the organization in deployment.