Amazon Web Services recently awarded prizes for innovative use of its Spot Instances, which are excess instances of the company's industry-leading Amazon Elastic Compute Cloud computing capacity.
PiCloud and Princeton Consultants won the grand prize and runner-up awards for their use of AWS Spot Instances in the first annual Spotathon, said Stephen Elliott, senior product manager on the Amazon EC2 team, in a recent blog post. Amazon Web Services (AWS) recognized the winners at its re:Invent conference.
Spotathon is a contest to find the most innovative applications that leverage Amazon Web Services Spot Instances and help small companies achieve massive scale and save time and money. Spot Instances is a real-time market in which users bid for spare Amazon Elastic Cloud Compute (EC2) computing capacity.
PiCloud's platform as a service for high-performance computing (HPC), batch processing and scientific computing took the grand prize of $2,500 in AWS credit in the Spotathon. PiCloud provides high-level APIs that scientists and engineers can use to submit units of computational work—like finding nucleotide sequences in a genome, conducting oil and gas geophysics simulations, or doing financial risk analytics—rather than provisioning, administering and tearing down instances themselves, Elliott said.
"By running 85% on Spot Instances, PiCloud provisions 50% more servers at the same cost, improves its customers' experience by delivering results 33% faster, and saves 65% over the On-Demand price," Elliott said in his post. "PiCloud has served thousands of researchers who have collectively processed over 100 million jobs, and is exemplary in how it uses AWS and Spot Instances to reduce researchers' 'time to science.'"
"In the investing world, as in many others, speed to result is a crucial competitive advantage. Princeton Consultants' realized that the computational scale and cost-competitiveness that can be achieved on Spot Instances would allow startup hedge funds to master the sheer quantity of financial data (hundreds of terabytes) and compete against the dominant firms by enabling them to rapidly and inexpensively test and tune new investment theses," Elliott said. "With OptiSpotter, researchers consume tens of thousands of instance hours on Spot and save up to 90% on their compute bill. More importantly, they can get feedback on their investment theses in hours or less, meaning they can iterate and tune an idea several times a day, rather than having to wait until the next morning (or for days) to back-test a new algorithm."
For helping a startup hedge fund greatly accelerate its research and reduce costs, and compete successfully against some of the world's largest funds, Princeton Consultants, an IT and management consulting firm stood out, Elliott said.
Princeton Consultants was retained by a high-frequency hedge fund to improve its ability to tune and devise new trading algorithms that trade hundreds of millions of dollars per day. A typical quantitative research project could require reading hundreds of terabytes of market data and tens of thousands of hours of processing. Some early quantitative research organizations are already using Amazon Web Services (AWS) for their research, but in general, they are using AWS On-Demand Instances, which allows for hourly rental of computers and disk.