Inside a Starbucks a business plots world domination
Aspiring "Next Facebook Founder" Non-Programmer: "You need to write the code so that it can scale to at least 3 million users in 6 months"
Contract Programmer Hired Off Craigslist: "Pardon me?"
ANFFNP: "If you look at the business plan, just after it defines your equity share in our little venture, you'll see that in week 20 we will have just over 1 million users. But by week 30 we will have 3 million users. After that, growth is exponential."
CPHOC: "Umm, I think we can worry about that later. I'm using Rails and Twitter was built on it. Sure they had some problems at first, but they managed to clear them up."
ANFFNP: "Well, I went to a startup pub night last week and this guy that wrote a URL shortener told me that Rails doesn't scale. I don't think we should use Rails."
CPHOC: "Okay... I guess I can use ASP.NET or PHP."
ANFFNP: "I heard at a FireFox 3.0 launch party last year not to use a Windows server because of security problems and that PHP powered Web 2.0. But our site's going to be Web 3.0, so PHP doesn't give us a competitive advantage."
CPHOC: "Well I've always wanted to learn Erlang..."
ANFFNP: "What's that?"
CPHOC: "It's this language that a phone company made to run its infrastructure. It apparently scales really well and was benchmarked at handling over 20 million processes once."
ANFFNP: "Excellent. I knew you were the right man for the job. I'll re-adjust our scalability numbers upward then. 20 million is about 6 times my original projections, so if I double them we should be good."
This isn't far from the truth with one of the companies I was involved in!! The non-techs had these totally unrealistic projections for millions of users and the tech wanted to write in their own PHP framework instead of Drupal so that it could scale.
If you're worried about being able to handle a volume of 10,000 on the first day, then the answer is probably "NO".
OTOH, if you're concerned about designing something industrial strength from the inside out, then the answer is probably "YES".
YES, you might as well build reuseable components now if you know you'll need them. If you know what you're doing, that shouldn't take any longer that writing the same code multiple times.
YES, you should write processor intensive code at a low level if you know you'll only have to rewrite it later. Again, doing it right in the first place shouldn't take any longer.
YES, you should build a flexible enough architecture if your specs and design aren't yet frozen in stone (whose are?)
YES, you should probably be doing lots of things right now to avoid doing them twice. You must use your professional judgement about which ones.
A blanket NO bypasses that judgement. That's not being fair to yourself.
I agree that the answer is not exactly "No", but I disagree with most of your points.
Building reusable components, coding at a low level and building a flexible architecture all take considerably more effort than their alternatives, IMHO.
That being said, it makes sense to have the big picture in mind for the first time around and to take some small considerations on the way.
Avoid doing inherently bad things like schlepping around tons of unneeded data and not doing some simple caching.
Like the recent post about embracing technological debt, much of the early work may be discarded before it see the light of day so no point in over investing.
Rather that doing lots of things right the first time, I spend more effort not tightly coupling the pieces so that when you have to write something twice (and you will), it's a clean separation. I'd get the processor intensive stuff done with as little effort as possible (and likely barely fast enough), but in a way that I could rip that part out and plug in some native code when the time came.
My favorite examplei is naked SQL vs an ORM. The ORM will never be able to write queries that perform as well as if you write them all by hand. But, you'll be able to crank out functionality much faster.
Eventually, if you're lucky, you'll have enough traffic to warrant going back and rewriting some of the ORM calls using a SQL query optimized by hand.
Lastly, 99% of the time, the query you end up optimizing will not be the one you expected to when you started writing the app. That's because between when you started and now, the direction has changed significantly and you finally know what it is that you're really building.
No, arguing about semantics does not mean what you think it does! It means arguing about what words mean instead of arguing substantively about the meanings themselves.
Quick answer: just don't prematurely screw it up. Some features and business ideas justdon'tscale.
The quickest way to check is to imagine you have everything in a big dumb (semi-normalized, but not fantastic) database. Eyeball the queries you're going to use. Then figure out how much work the database has to do to execute those queries. Also consider read/write rates and contention.
One of the first concerns I have about a site is performance. Not scaling , but making sure the site itself operates very quickly. I have found that the path to getting a fast load time often leads to a reasonable level of scaling.
Doesn't it also depend on who 'I' am? If I am part of a tiny startup that is going to launch a twitter clone, then no, I guess not. But if I am at Facebook or Twitter or Google and I am adding a new feature to the app, then hell, yes. You will have millions of users on Day 1.
That's very true. Plenty of 'start-ups' never make it past their 10,000th signup.
My approach is to throw stuff out there as soon as possible and see what gets traction and what does not. Plenty of time to concentrate on the few that do pan out that way.
Good ideas are two a penny, in the last 10 years out of all the websites we've built for ourselves and for customers the number of duds is staggering. A good idea in and off itself is not a guarantee that something will take off. Some of the most well thought out things I've built have totally flopped and some of the midnight hacks still give me bread on the table a decade later.
So my approach now is to keep the first implementation as thin as I think I can get away with without people noticing how solid something is.
Knowledge about building scalable systems is invaluable though when you do hit the jackpot.
Very, very true. I've worked for more than a few companies that built multiply-redundant, highly scalable everything, and then ended up with ten small paying customers.
The first guy to set up one of these domains was pretty clever. Anyone else who had the idea before they saw one gets a free pass. But the rest of them are just lame. Kinda like milliondollarhomepage2.com et al.
Aspiring "Next Facebook Founder" Non-Programmer: "You need to write the code so that it can scale to at least 3 million users in 6 months"
Contract Programmer Hired Off Craigslist: "Pardon me?"
ANFFNP: "If you look at the business plan, just after it defines your equity share in our little venture, you'll see that in week 20 we will have just over 1 million users. But by week 30 we will have 3 million users. After that, growth is exponential."
CPHOC: "Umm, I think we can worry about that later. I'm using Rails and Twitter was built on it. Sure they had some problems at first, but they managed to clear them up."
ANFFNP: "Well, I went to a startup pub night last week and this guy that wrote a URL shortener told me that Rails doesn't scale. I don't think we should use Rails."
CPHOC: "Okay... I guess I can use ASP.NET or PHP."
ANFFNP: "I heard at a FireFox 3.0 launch party last year not to use a Windows server because of security problems and that PHP powered Web 2.0. But our site's going to be Web 3.0, so PHP doesn't give us a competitive advantage."
CPHOC: "Well I've always wanted to learn Erlang..."
ANFFNP: "What's that?"
CPHOC: "It's this language that a phone company made to run its infrastructure. It apparently scales really well and was benchmarked at handling over 20 million processes once."
ANFFNP: "Excellent. I knew you were the right man for the job. I'll re-adjust our scalability numbers upward then. 20 million is about 6 times my original projections, so if I double them we should be good."
CPHOC: sigh...
Sorry, I'm in one of those moods today :-)