Twitter on its architecture: We can rebuild it. We have the technology.

Venture Beat — Thu, 22 May 2008 12:50:59 -0700

Amid all the downtime, quite a few fans and haters of the micro messaging service Twitter have taken the opportunity to wax poetic about what Twitter should be doing to solve its issues. Now Twitter developer Alex Payne has responded on the Twitter Developer Blog.

The basic gist: Twitter was built the wrong way, now we must re-build it one piece at a time. This will take awhile. Come work for us!

According to Payne, Twitter was originally architected not as a messaging service, but rather as a kind of content management system (CMS). Because of this, Twitter has had major scaling issues over the past year and change — issues we’ve all experienced in both the system being down and general slowness.

While Payne states that the system as it exists today is much better than it was a year ago, there is still much work to be done. He hopes that one day the idea of simply “throwing more machines at it” will work (more or less) for scaling. Yet he warns, Twitter is not at that point yet and it will be slow-going to get there as Twitter is not about to shut the entire service down for a re-build from scratch.

Says Payne:

Our direction going forward is to replace our existing system, component-by-component, with parts that are designed from the ground up to meet the requirements that have emerged as Twitter has grown. First and foremost amongst those requirements is stability. We’re planning for a gradual transition; our existing system will be maintained while new parts are built, and old parts swapped out for new as they’re completed. The alternative - scrapping everything for “the big rewrite” - is untenable, particularly given our small (but growing!) engineering and operations team.

Interestingly enough, Payne doesn’t once mention Ruby on Rails, the framework on which Twitter is built that a lot of people (commenters on this blog and others at least) seem to have a major problem with. There were even rumors that Twitter was going to abandon Ruby on Rails. The service denied those rumors.

Payne also takes the time to mention that Twitter is hiring — no doubt thanks in part to that new round of funding that is rumored to have closed.

If you want to make Twitter better, there’s no more direct way than getting involved in our engineering efforts. We love kicking around ideas, but code speaks louder than words.

Indeed.