Skip to main content

Do you Scale?

Great post by Dare, Scalability: I Don't Think That Word Means What You Think It Does
following up on a post from Scott Loganbill at Google about their choice of "scalable" services.

Scalability is a funny question. Every application is designed to handle a certain type of architecture - it's usually one of those fundamental sales analysis questions "and how many orders do you typically process a day? How many people do you have working for you".

While scalability problems can be due to both technology and design, it's typically design that is the major culprit. (ok, yes, the guy that said he can run a 15,000 transaction a day system in Access - he likely chose the WRONG technology).

But everytime I think of scalability, I always think of 37 Signal's approach to scalability - in short, "Do you really need 12 servers now if you can run on two for a year?" - note read the comments from David Hansson "... Not worrying too much about scaling doesn’t mean building Basecamp on an Access database with a spaghetti soup of PHP making 250 database calls per request. It simply means having a profitable price per request, chilling the f*ck out, and dealing with problems as they arise."

Actually, their better quote is here. "If you've got a huge number of people overloading your system then huzzah! That's one swell problem to have."

Many of these posts and statements are generally made for large systems designed for mass usage. Not necessarily the same issues facing the systems that small businesses or even medium-sized businesses need. But still, just thinking about scalability when designing your databases can help in a big way.

I see silly design decisions that limit scalability all the time and they are generally made for no reason at all. A limit of 2 characters for Sequence numbers - what?!? you're never going to get over 99 items on a single order? Regardless of how unrealistic it may seem, chances are someone will hit it.

And then there are those that become requirements: A 5 character code for a customer ID - then the customer comes up and says I need to use 6. Is that really a scalability issue? Not really - until you've reached 26^5 customer codes, but the customer may perceive it as such. But the whole 26^5 argument does show there is a scalability design flaw in that definition. Will it ever show up? Hard to say.

Two Real Factors
So consider two different factors when it comes to scalability: hw uptime/access (as in can your hardware infrastructure handle a large number of users/transactions) and design (can your application handle a large number of users/transactions). Both go hand in hand yet demand different solutions.

The infrastructure factor can play off the design - if your application is designed for small groups but has lots of small groups, you can always put one group onto another server, if need be.

The Design can detrimentally affect the infrastructure. You can have as many servers as you want but if you constantly pull down lots of records when you only need one, you're asking for headaches.

And One more...
There is one more scalability factor to consider...and it's one that I think many people disregard because it's a Design issue but not so much an architecture design one - it's a user experience scalability design. Many applications were/are built for single transactions (the first web banking site for my bank only allowed one bill to be paid at a time). Just as much as uptime/downtime can play into it, if your application doesn't make it EASY to handle the increase in use, you can just forget about your other scalability problems.

Purists will say "Every software should scale " (and scale infinitely). Pragmatists will say "there will always be SOME scalability flaw in any design".

Think of this another way...even scales have scalability problems - in the design. Many only go to 300lbs. With the obesity epidemic these days, I'm not sure that's a limitation that users can overlook.

Comments

Anonymous said…
Hi Andrew,

Your post on scalability even describes my wrong use of Excel ;-). I enjoy the categorization you make when it comes down to scalability problems and I couldn't agree more that the 37signals quote is often most applicable.

Eiso

Popular posts from this blog

Blogs and RSS come to Microsoft.com

MS has just introduced their portal and it's pretty comprehensive. Nothing quite like learning that some people use AIM instead of MSN messenger, or that there really may be a need for supporting 4 monitors ( Cyrus Complains ) However, it's really a great sign that MS is serious about supporting the blogging community which seems to have um, exploded in size in the past year. Blogs and RSS come to Microsoft.com

Elevating Project Specifications with Three Insightful ChatGPT Prompts

For developers and testers, ChatGPT, the freely accessible tool from OpenAI, is game-changing. If you want to learn a new programming language, ask for samples or have it convert your existing code. This can be done in Visual Studio Code (using GitHub CoPilot) or directly in the ChatGPT app or web site.  If you’re a tester, ChatGPT can write a test spec or actual test code (if you use Jest or Cypress) based on existing code, copied and pasted into the input area. But ChatGPT can be of huge value for analysts (whether system or business) who need to validate their needs. There’s often a disconnect between developers and analysts. Analysts complain that developers don’t build what they asked for or ask too many questions. Developers complain that analysts haven’t thought of obvious things. In these situations, ChatGPT can be a great intermediary. At its worst, it forces you to think about and then discount obvious issues. At best, it clarifies the needs into documented requirements. ...

Programmers vs. Developers vs. Architects

I received an email this morning from Brandon Savage 's newsletter. Brandon's a PHP guru (works at Mozilla) but his newsletter and books have some great overall perspectives for developers of all languages. However, this last one (What's the difference between developers and architects?) kind of rubs me the wrong way. Either that, or I've just missed the natural inflation of job descriptions. (maybe, it's like the change in terminology between Garbage man and Waste Engineer or Secretary and Office Administrator) So maybe it's just me - but I think there's still a big difference between Programmer, Developer and then of course, architect. The key thing here is that every role has a different perspective and every one of those perspectives has value. The original MSF create roles like Product Manager, Program Manager, Developer, Tester, etc - so every concept may pigeon hole people into different roles. But the statements Brandon makes are often distinction...