As the maker of one of the largest applications using Ruby on Rails on the Web, Twitter knows a thing or two about scaling applications built with the popular development framework.
Britt Selvitelle, a senior engineer at Twitter, offered a few tips and tricks for scaling Ruby on Rails and expressed particular appreciation for the Rails framework itself and the language is it based on, Ruby.
“For us, for a large part of our system, Ruby has been the tool that fit,” Selvitelle said. He said Twitter has primarily used Ruby and Rails for its applications with a sprinkling of Erlang. “We’re constantly experimenting,” he said.
Selvitelle said the Twitter development team prefers the Ruby environment, although, “if we need to drop down to a lower-level language we are prepared to, but we haven’t had to.” In the future, he said, “We might have to optimize certain parts of the system with C++ or something, but as of yet, we haven’t had to.”
As part of Twitter’s strategy for scaling Rails, the company has taken advantage of more hardware capacity via its Web hosting provider-first Joyent and later NTT America-as a first resort to accommodate increased volume.
Cache early and often
In addition to hardware capacity, Selvitelle said Twitter addresses the issue of scaling Rails by caching early and often. In a talk on scaling Rails applications, Selvitelle advised developers to abstract long-running processes to daemons and “index everything you will query on, avoid complex joins, use joint indices when you must join tables, and avoid scanning large sets of data,” among a series of other tips.
Selvitelle said Twitter has gone from being a small side project at Obvious, the San Francisco company that launched Twitter, to becoming one of the most popular sites on the Web.
Click here to read about some startups that are focusing on scalability for Ruby on Rails.
“There are several layers; it’s like peeling an onion,” he said of the levels of complexity involved in maintaining the Twitter site. And some of the time things just don’t work, as in some of the outages the company has had over recent months. The company related a recent outage to “caching changes” and others were said to be related to issues with Twitter’s Web hosting provider.
“The fact is we have more users and higher complexity,” Selvitelle said. “We have to balance how to keep the site performant and add new features. There’s a fine line between making your system robust and supporting users.”
Meanwhile, Ruby as the Twitter team’s language of choice “is a fantastic language, and here in our own South Park neighborhood in San Francisco we’re lucky to be around some of the best Ruby developers in the industry,” Selvitelle said, speaking of Engine Yard, a hosting service for Ruby and Rails applications.
“Ruby is a beautiful language and it’s fun to write software in; it gives us a lot of creativity,” Selvitelle said. Ruby also has enabled the Twitter development team to use agile development techniques, he said.
Moreover, the development team has been able to work more efficiently because it eats its own dog food, Selvitelle said. “We not only write the software, we use the software,” he said, speaking of the Twitter messaging Web 2.0 technology. “We use it all the time. Our lives revolve around the technology.”
Meanwhile, despite Twitter’s professed satisfaction with Ruby on Rails, there have been recent reports that Twitter might be looking to abandon Rails because of scalability issues. However, Twitter officials debunked the claim. First, Obvious founder Evan Williams refuted the claim in a Twitter post, saying: “FWIW: Twitter currently has no plans to abandon RoR. Lots of our code is not in RoR, already, though. Maybe that’s why people are confused.”
Selvitelle told eWEEK that reports of Twitter abandoning Rails are “Not true in any sense. We use Ruby as our primary language. We have plenty of back-end architecture in other languages. Especially prototypes. We still use Rails and have no plans to discontinue this in the future.”