Goldman Sachs Trades Up and Scales Out

Case Study: Goldman Sachs bucks tradition by opting for a drop-in, commercial clustering tool to handle its exponentially growing data.

When Goldman Sachs faced an important build-versus-buy decision on a project to add clustering to one of its Java-based trading solutions, the renowned investment house decided to buck tradition and actually look for a commercial solution that would suit their needs.

After a bake-off of competing offerings, Goldman decided on San Francisco-based Terracottas clustering software as the foundation for its system upgrade.

Terracotta simplifies the scale-out of Java applications by adding needed services to the Java run-time, said Ari Zilka, president and chief executive of Terracotta. By delivering these API-free clustering and caching services, Terracotta dramatically improves the availability and scalability of production-level Java deployments, Zilka said. The Terracotta product suite provides powerful tools to help enterprise organizations design and develop clustering and caching, he added.

/zimages/4/28571.gifClick here to find out what Terracottas CEO had to say about open-source Java.

"We do scale-out, as opposed to scale-up," Zilka said. "Scale-out is commodity hardware," he said, adding, "Were not a hardware vendor so were not talking about hardware, just a deployment model."

David Campbell, vice president for principal investments at Goldman, said the company had experienced a flood of new demands for its ever-increasing pool of data and decided to upgrade its systems to accommodate the need.

So around the middle of last year, "The team was tasked with taking the existing system and scaling it by an order of magnitude and delivering the services," Campbell said. "They were looking at their applications and considering ways to tune those. The data began to grow ... and they were partitioning the data by hand.

"On top of this, the problems continued to grow exponentially with the growth of the data," Campbell said. "So they said they wanted the data in a single pool across the application that would allow them to build clustered services that could scale and move things independently based on which way the business was going."

/zimages/4/28571.gifClick here to read more about Microsofts foray into high-end clustering.

After a trial run of potential candidates, "The feeling was if we could get the benefits of a commercial product and leverage their development it would help us along," Campbell said. "About three months into the effort, Terracotta was introduced to the group, and they brought it in, and, within 3 hours, they had it up and running."

"Its a little embarrassing that it took a little more than 3 hours," Zilka said. "Today its more like an hour and a half. We call it less than a week. ... The out-of-the-box experience—TOOBE—is what we call it. We learned that from the guys who founded WebLogic years ago."

Meanwhile, "being able to get the results our guys wanted without having to do any work essentially was just really very, very impressive," Campbell said.

At the time, Goldman was the first Terracotta customer interested in doing this massive data scale-out, Zilka said. "And we found some things that Terracotta hadnt thought about at that scale, and over the next two months, Terracotta gave a series of releases that addressed our concerns," added Campbell.

Zilka said Terracotta is a drop-in solution that requires no intervention to the users business logic or existing code.

"Things like Spring and Hibernate are meant to simplify your business logic," Zilka said. "So you have to go to frameworks like Spring, which are not clusterable, and then to Hibernate. And all of these tools get complicated in the world of scale-out. Clustered app servers are expensive, and they dont scale. Our layer is a physical server as a set of JARs [Java Archives] that fit on your server."

/zimages/4/28571.gifThe Terracotta Clustered Spring Run-time provides an enhancement to the Java run-time that makes applications built with Spring scalable across clustered servers. Click here to read more.

Zilka said Terracotta preserves the use of frameworks and acts as a run-time discovery and cluster manager.

Next Page: Changing the Java game.