The Little Machine that

By Jason Brooks  |  Posted 2006-01-29 Print this article Print

Could"> "I demonstrated this on Halloween 2001, and it was pretty simple," said Thompson. "These Sun 4500s were costing about a quarter of a million dollars apiece, and we could run 300 to 400 users on them. And here's this little box that cost us about $4,000. It was running faster, but you could only run 180 people on it.

"Josh Levine, the CIO at the time, said, Well, then we just need to get two, for $8,000. Do it-put a box in production. So, it was a fairly short meeting."

With this mandate, ETrade's architecture department got back to work, putting the new stack through a series of tests. In December 2001, they put the Linux box into production, alongside the rows of Sun 4500's that sat behind ETrade's load balancers.

"Heres this little $4,000 box, playing, literally, with the big boys of computing and doing very, very well," said Thompson.

"Everybody was pretty floored by this-a lot of the systems engineers who do our production operation were all going over to the box and couldn't believe how well this little machine was doing."

Click here to read about the benefits of open-source ERP.

When Thompson and his team began porting the application to Linux, they first copied the whole code base into a separate Linux branch.

It was in this branch that the team made the adjustments necessary for the code to run properly on Linux. For example, the move from Solaris to Linux also involved moving from the Sun Studio C compiler to GCC, or GNU Compiler Collection, the dominant Linux and open-source compiler, and the team had to sort through syntax errors turned up by the compiler change.

During the next couple of months, the team ported the rest of the application. In the spring of 2002, the team brought in 50 IBM x330s servers and, in stages, shifted traffic through their load balancers from the rows of Sun machines to the rack and a half or so of x86 Linux boxes.

The Sun servers were reallocated, retired or sold to other companies.

While cost savings was the primary motivation for exploring an open-source x86 migration, ETrade found that the move also yielded significant performance increases.

The open-source migration also afforded ETrade a higher level of flexibility. This was especially important because ETrade was making many acquisitions at this time, according to Framke.

"Linux fit our model perfectly for ... scaling horizontally rather than vertically," he said. "It gave us the flexibility to bring a lot of products online that were not just brokerage in nature."

Flush with its initial open-source success, ETrade's IT organization was enthusiastic about expanding the model, setting its sights in 2004 on the BEA Tuxedo transaction management server.

"We're happy to pay for software where it really helps us out, but, on the other hand, if we find an open-source alternative where we're not paying for licensing, that's the way we'll go," said Framke.

ETrade is about halfway through the development of the Tuxedo replacement, which is expected to save the company $2 million a year.

Coming off the open-source migration experience, Thompson was interested in examining the processes that produced the software hed just finished deploying so successfully.

In a test environment, Thompson installed Gentoo Linux, a distribution that exposes its users to the breakneck rate of change that open-source software undergoes.

Despite these massive changes, however, the system on which Thompson was running Gentoo was relatively stable. Knowing that managing change better and faster than the competition is a key to success, Thompson and Framke sought to find out how open sources methodology-its "secret sauce"-could be applied in ETrade's own development team.

The secret, ETrade learned, is that open-source projects are organized modularly, with limited commit rights. ETrade is now charting a course for reorganizing the company's internal development processes to match the way that open-source projects and Linux distributions are organized.

"As our code base grows, as the business gets more complex, it'll become harder for us to achieve the same productivity and rate of change that we've enjoyed in the last few years," said Framke.

"We want to break up into separate packages, to allow different packages to evolve at a different rate of change. This is not how we do software today at all-basically, the whole application moves forward."

HP appoints new VP for open source, Linux. Click here to read more.

ETrade also is interested in open-sourcing some of its internally developed components. Framke and Thompson believe they have something to offer, stemming from ETrade's early bets on SOA (service-oriented architecture).

"We've benefited in so many ways from what community processes have taught us," said Thompson, "and I think its just natural that there might be something that we do that the open-source community may be interested in."

Click here to read reader response to this article.

Senior Analyst Jason Brooks is at

As Editor in Chief of eWEEK Labs, Jason Brooks manages the Labs team and is responsible for eWEEK's print edition. Brooks joined eWEEK in 1999, and has covered wireless networking, office productivity suites, mobile devices, Windows, virtualization, and desktops and notebooks. JasonÔÇÖs coverage is currently focused on Linux and Unix operating systems, open-source software and licensing, cloud computing and Software as a Service. Follow Jason on Twitter at jasonbrooks, or reach him by email at

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel