Dynamic Python

By Darryl K. Taft  |  Posted 2007-03-05 Print this article Print

"Thats the scary thing for us. Sometime in the next year Air Canada is going to turn off for a few hours, and then were going to turn back on [using] the new system. That type of thing has never been done—going from a legacy system to a new system." Dynamic Python
Much of the code ITA employs is written in Python, despite skepticism by some that dynamic languages are not ready for prime time. However, people such as Guido van Rossum, the creator of Python, point to the successful use of the language at places such as Google and YouTube, which endure enterprise-scale traffic on a daily basis.
Meanwhile, ITA has about 200,000 lines of Python code in use in its production software. ITA has implemented a reliable multicast transport with congestion control in order to deliver multiple streams of data, Kelley said. The server is a mixture of C++ and Python, running on top of the Twisted Python framework. "In addition, some of the data feeds we receive from our customers and partners are handled by network servers running on Twisted," Kelley said. ITA has worked closely with its airline partners to build high-performance servers to compute seat availability, Kelley said. Their servers aggregate data from public and private data sources, including both streaming and file-based data. Each server requires a unique set of data feeds, and all the data feeds must be monitored to ensure the data arrives when expected and gets loaded as required, all while ensuring that system performance is not impacted by these data management tasks. ITA uses Python to automate all these tasks to make its systems reliable, easy to maintain and easy to update as its customers needs change, Kelley said. Moreover, ITA uses a Python application to monitor its heterogeneous production environment that contains hundreds of servers. "In order to do this efficiently, we developed our own monitoring framework that enables the requirements to be expressed in configuration rather than in code," Kelley said. And ITA uses agile development methods, "which require that our software is always buildable," he said. Microsoft ships Python on .Net. Click here to read more. "To do this, we need to be able to perform automated regression tests with every build, but our systems require dynamically changing data for meaningful tests. ITA has built regression frameworks in Python to test both availability and reservations systems. This framework enables us to load test data while we test and has been built around a configuration language so that it is straightforward to add new test cases as needed." The first piece of core software ITA built using Python was a software load balancer, Kelley said. "We knew we needed a software load balancer to meet a customer commitment to have their site up and running," he said. "We had this hacked-together version that somebody had written in Perl in a week, but it fell over continuously. And we needed something that we could put together that would replace it but that also would be maintainable over the long run." Python answered the call. "Since then, weve changed how we use Python a ton internally," Kelley said. "We have lots more production software written in Python. Weve basically reimplemented all our production service monitoring in Python and also our production software management infrastructure for a significant amount of what we run. "A big component to that has been our use of Twisted Python. Were pretty reliant on the Twisted framework, and we use it for our base-line management software that we use to run the great majority of production services that we have, our monitoring infrastructure and the next-generation thing that we have coming, which is a suite of programs that will automate the upgrade process for us," Kelley said. A Mix of Components The Air Canada system will be a mix of components written in different languages, Kelley said. Next Page: But why Python?

Darryl K. Taft covers the development tools and developer-related issues beat from his office in Baltimore. He has more than 10 years of experience in the business and is always looking for the next scoop. Taft is a member of the Association for Computing Machinery (ACM) and was named 'one of the most active middleware reporters in the world' by The Middleware Co. He also has his own card in the 'Who's Who in Enterprise Java' deck.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel