Amazon Web Services Awards EC2 Spot Instances Users
Princeton Consultants built OptiSpotter, which uses custom algorithms to optimally bid on and consume AWS Spot Instances. Spot Instances have prices and availability that fluctuate constantly, and running Spot Instance jobs can be cancelled by Amazon at any time without notice. In exchange, Spot Instance hourly prices are often at 10 percent or less of On-Demand prices. Elliott further explained the AWS Spot Instances offering: "Spot Instances are excess EC2 instances whose price is based on real-time supply and demand. When you request an EC2 Instance via a Spot request, you specify a bid price; as long as your bid exceeds the Spot price, you are provisioned a Spot Instance. When the Spot price exceeds your bid, your Spot Instance is interrupted." OptiSpotter won the Amazon award for its innovative approach that transforms research computing from massive, multi-hour On-Demand jobs into hundreds of smaller pieces that can effectively use Spot Instances. The hedge fund's research turnaround time was sped up five to ten times—a critical competitive advantage in today's fast-moving market conditions—and its costs were reduced 90 percent. "Princeton Consultants' OptiSpotter is an exciting, innovative application taking advantage of AWS Spot Instances that dramatically improves the speed and cost-effectiveness of big data quantitative research," Matt Wood, chief data scientist at AWS, said in a statement. "The judges of our first Spotathon coding challenge were impressed with their creative approach."Sashihara said he sees potential use of OptiSpotter by all sizes of organizations in which researchers seek significantly faster throughput and lower processing costs in the Amazon cloud. Princeton Consultants customizes OptiSpotter to the client's needs. "OptiSpotter maps massive, multi-hour jobs into thousands of small sub-jobs, queues them based on memory and I/O requirements, then it monitors the Spot price history and queues of outstanding jobs to determine the most efficient way to deploy Spot Instances," Elliott said. Meanwhile, AWS gave honorable mention to a couple of other organizations for their use of Spot Instances. The first is Numerate's drug discovery application built on its Numatix platform. Numatix accelerates drug discovery while reducing EC2 compute costs by more than 80 percent, Elliott said. "Numerate's proprietary machine-learning algorithms predict the properties of small (drug-like) molecules and run Numatix on EC2 Spot Instances to scale to 10,000 cores to search large sets of molecules (>100 million) and identify those likely to lead to new drugs," Elliott said in his post. "All this for $100 per hour. Numerate's use of Spot Instances enables them to search enormous chemistry spaces in hours, and flexibly decide how fast they require results and how deep to conduct their analyses. Numerate plans to open Numatix up for broader use beyond drug discovery and is another exemplary case of a powerful cloud solution that reduces computational costs and time-to-results so that scientists can rapidly iterate on their discoveries." The second honorable mention went to Lawrence Berkeley National Laboratories' (LBL) Turbine Science Gateway (TSG), which supports the Department of Energy's Carbon Capture Simulation Initiative (CCSI) by providing a Web application and execution environment for running and managing scientific applications and storing and archiving results, Elliott said. "Utilizing TSG, simulation runs that would take months on a single machine can be done overnight on EC2, running tens of thousands of simulations on hundreds of Spot instances and saving over 70 percent on EC2 compute costs."
"We see a new world in research and data processing," said Steve Sashihara, CEO of Princeton Consultants, in a statement. "For time-sensitive big data research, renting processing capacity from AWS is an attractive alternative to buying and maintaining supercomputers. Our approach and OptiSpotter application make access to AWS through the use of Spot Instances even more compelling.