Dynamic Python

 
 
By Darryl K. Taft  |  Posted 2007-03-05
 
 
 

Python Slithers into Systems


Last summers box office hit "snakes on a plane" was the unlikely story of a batch of snakes getting loose in flight on a 747. In what some might view as just as unlikely, ITA Software is using the Python language to empower its airline reservation system.

Many computer language purists say that languages such as Java, C++ or C should be used for enterprise applications. However, ITA, a Cambridge, Mass., provider of airline IT software and services, is proving that dynamic languages such as Python can be rock-solid for enterprise work.

Indeed, ITA in September 2006 announced an agreement with Montreal-based Air Canada, a subsidiary of ACE Aviation Holdings, to develop a new reservation management system for the airline. Air Canadas next-generation reservation system will include reservations, inventory control and seat availability, along with airport check-in and airport operations modules.

Sean Menke, Air Canadas executive vice president and chief commercial officer, said the new system "will be a key instrument in Air Canadas strategy to maintain market leadership through product and customer differentiation. Moreover, it will provide unprecedented flexibility and significant cost savings."

Leaving Legacy Behind

Dan Kelley, director of application integration at ITA, said ITA will deliver the system to Air Canada next year, and it will be deployed across the entire Air Canada network, including reservation call centers and airport locations throughout the world.

The reservation system will be delivered as an application hosted by ITA Software.

ITA Software was founded in 1996 by computer scientists from the Massachusetts Institute of Technology. The company provides airlines and travel distribution companies with airfare pricing, shopping and availability management systems and other technology.

ITA Softwares customers include Air Canada, Alaska Airlines, Alitalia, Continental Airlines, US Airways, Star Alliance, Galileo International, Kayak, Orbitz and others.

Like many of ITAs customers, Kayak.com, a comparison-shopping Web site for airfares, uses ITA Softwares QPX airfare pricing and shopping system. Unlike the traditional mainframe-based systems empowering most airline software systems, QPX uses large numbers of commodity Linux-based PCs and employs a component-based architecture that has no single point of failure and that scales linearly.

"What ITA did historically is we did search," Kelley said. "Think of us as Google for airfares. So if somebody like Orbitz, who is a customer of ours, used us, theyd write a bunch of Java code that displays their Web sites and does their front end. And they write a bunch of code that actually does reservations on one of the big mainframes that maintains the information about who bought what ticket. And the only thing we were providing was search."

However, ITAs contract with Air Canada will have the company pulling Air Canadas systems off the mainframe and onto a farm of Linux PCs.

"With Air Canada right now, were building an airline reservation system that is going to replace their mainframe-based system, which does 95 percent of what they do as a business," Kelley said.

Yahoo delivers resource for Python developers. Click here to read more.

"When you go and you check in on a kiosk at an airport, or you check a bag, or buy a ticket through a travel agent or Web site, youre talking to this one mainframe that right now sits in Winnipeg, [Manitoba]."

Instead of one monolithic system, ITA is writing a series of components that are all going to be Linux processes running on different machines. Kelleys unit is taking a group of 30 to 50 components and combining them into a cohesive operating environment. Some of the key components are written in Python. The Air Canada system has 110 interfaces, or ways that people can interact with the system.

"And its hooked up to like 500 different computers here and elsewhere," Kelley said.

Yet, "in addition to being a huge technical challenge, nobody in the history of airline computing has ever swapped out a mainframe-based reservation system for something else," he said.

Next Page: Dynamic Python.

Dynamic Python


"Thats the scary thing for us. Sometime in the next year Air Canada is going to turn off for a few hours, and then were going to turn back on [using] the new system. That type of thing has never been done—going from a legacy system to a new system."

Dynamic Python

Much of the code ITA employs is written in Python, despite skepticism by some that dynamic languages are not ready for prime time. However, people such as Guido van Rossum, the creator of Python, point to the successful use of the language at places such as Google and YouTube, which endure enterprise-scale traffic on a daily basis.

Meanwhile, ITA has about 200,000 lines of Python code in use in its production software.

ITA has implemented a reliable multicast transport with congestion control in order to deliver multiple streams of data, Kelley said. The server is a mixture of C++ and Python, running on top of the Twisted Python framework.

"In addition, some of the data feeds we receive from our customers and partners are handled by network servers running on Twisted," Kelley said.

ITA has worked closely with its airline partners to build high-performance servers to compute seat availability, Kelley said. Their servers aggregate data from public and private data sources, including both streaming and file-based data.

Each server requires a unique set of data feeds, and all the data feeds must be monitored to ensure the data arrives when expected and gets loaded as required, all while ensuring that system performance is not impacted by these data management tasks. ITA uses Python to automate all these tasks to make its systems reliable, easy to maintain and easy to update as its customers needs change, Kelley said.

Moreover, ITA uses a Python application to monitor its heterogeneous production environment that contains hundreds of servers.

"In order to do this efficiently, we developed our own monitoring framework that enables the requirements to be expressed in configuration rather than in code," Kelley said.

And ITA uses agile development methods, "which require that our software is always buildable," he said.

Microsoft ships Python on .Net. Click here to read more.

"To do this, we need to be able to perform automated regression tests with every build, but our systems require dynamically changing data for meaningful tests. ITA has built regression frameworks in Python to test both availability and reservations systems. This framework enables us to load test data while we test and has been built around a configuration language so that it is straightforward to add new test cases as needed."

The first piece of core software ITA built using Python was a software load balancer, Kelley said.

"We knew we needed a software load balancer to meet a customer commitment to have their site up and running," he said.

"We had this hacked-together version that somebody had written in Perl in a week, but it fell over continuously. And we needed something that we could put together that would replace it but that also would be maintainable over the long run." Python answered the call.

"Since then, weve changed how we use Python a ton internally," Kelley said. "We have lots more production software written in Python. Weve basically reimplemented all our production service monitoring in Python and also our production software management infrastructure for a significant amount of what we run.

"A big component to that has been our use of Twisted Python. Were pretty reliant on the Twisted framework, and we use it for our base-line management software that we use to run the great majority of production services that we have, our monitoring infrastructure and the next-generation thing that we have coming, which is a suite of programs that will automate the upgrade process for us," Kelley said.

A Mix of Components

The Air Canada system will be a mix of components written in different languages, Kelley said.

Next Page: But why Python?

But Why Python


?">

"There are components written in Java, LISP, C++ and Python. For each component area, we got a functional spec from the customer saying it has to do X, and we had to figure out who the right people were to work on that project, and those people decide what implementation language to use," he said.

There are two critical components that are written in Python, without which the system couldnt exist, Kelley said. The first is the inventory controller, which keeps track of what seats are available on what planes.

However, "we think of the most core [component] as the piece that takes reservation requests and records them in a database," Kelley said. "Thats written in LISP. But the next most critical thing is the thing thats maintaining inventory levels on airplanes, and thats written in Python."

The other core component is written in Python and is all "messaging well do with travel agents and other reservation systems. That messaging daemon is written in Python," Kelley said.

In addition, a significant percentage of Air Canadas ticket volume was done through travel agents last year, he said. In the new system, every one of those transactions will go through a Python message route.

But why Python?

"Historically, its not known for being something that somebody would go out and code enterprise software in," Kelley said. However, "its definitely an enterprise-caliber language in terms of stability, scalability [and] the ability to have a large number of people work together on a project.

"But its just coming into its own where you could defend it to nontechnical people as a language on which you could develop enterprise software. One of the things we have going for us is, because were founded by computer scientists, we dont have to defend our use of that programming language because its not Java," Kelley said.

"We have a wonderful ability here to choose the right tool for the job. We have components that are written in Java, in C++, in Python, and Ruby and Perl. [Python is] definitely viewed internally here by some of the best computer scientists in the world, people from MITs AI [artificial intelligence] and CS [computer science] labs, as enterprise worthy," he said.

Indeed, ITA has to defend its ability to meet its SLA (service-level agreements) against the Python-based technology.

"So we think, absolutely, that its ready for prime time," Kelley said. "What were doing is saying to a billion-dollar business, Yes, we can write components in this particular programming language, and they will keep your airline running."

Adrian Holovaty, a developer at Washingtonpost.com and the creator of Django, a Python Web development framework, bristles at the criticism of Python as possibly not being ready for the enterprise.

"The word enterprise in this context is mostly meaningless to me," said Holovaty in Chicago. "Its really just a marketing word that has no basis in logic."

Holovaty pointed to Google and YouTube, saying that it "doesnt get much higher traffic than that. Personally, I have direct experience using Python as my primary development language daily at my day job at Washingtonpost.com. Its a fantastic language that I couldnt live without," he said.

Moreover, when ITA hires new people, the company likes to hire those with Python experience "because weve had a lot of luck with Python people having a lot of core problem-solving and system-building ability," Kelley said.

He said it is pretty easy to find Java or C programmers who are good at line coding but not generally good at problem solving. "Its much more unusual for us to find people who can analyze a problem domain and then implement a solution where they cross a bunch of problem domains," Kelley said.

Python developers typically can, he said.

Meanwhile, getting off the mainframe will afford Air Canada all kinds of productivity and cost benefits, including lower cost of maintenance, better ability to integrate with modern architectures, and the ability to add components and enhancements more easily than with the mainframe, Kelley said.

Check out eWEEK.coms for the latest news, reviews and analysis in programming environments and developer tools.

Rocket Fuel