PostgreSQL 8.4 Comes with New Features, Development Strategy

The PostgreSQL 8.4 database has hit the streets, bringing with it a host of new features. But perhaps just as important is what is going on behind the scenes.

As the open-source database has trucked along, the amount of work required to get the database production-ready has grown. To make sure everything gets done, contributors to the project have reorganized their efforts, changing their development process and adding a new team of people to help the project's committers do basic review of patches.

"Basically if you go back say three releases, the way that we developed Postgres was really sort of classic waterfall ... So people send in patches to our patches list and they got incorporated by a small group of committers who worked on their own schedule," explained core team member Josh Berkus. "All that sort of built up until we declared a feature freeze, at which point we stopped accepting the patches. Then the committers would work like crazy to merge everything that we had received up until then."

That system worked fine back in 2003, when around 50 or 60 patches were received per year. Today, there are roughly 400 patch submissions each year and the old system no longer scales, Berkus said. As a result, the PostgreSQL community in 2008 began what it calls Commit Fests, in which the focus shifts from development to review. During the Commit Fest, all patches in the queue are either committed to the CVS repository, rejected or returned to the author for additional work.

"Starting with this development cycle, every time we do our patch integration—that is our Commit Fest—at the end of that process we will do what we will call an alpha release," Berkus said. "You'll [see] these sort[s] of alpha releases coming from the Postgres project every two months. The hope behind that is … to get people playing with major features that are added early in the annual development cycle earlier."

PostgreSQL advocates press on. Click here to read more.

The new strategy was implemented with Version 8.4, which includes 293 new features. Some of the more prominent enhancements are parallel database restore, new query monitoring tools and the ability to set per-column permissions. There are also in-place upgrades from 8.3 to 8.4 through the pg_migrator beta.

"In the past, when you were recovering a database from a backup or migration file, you could only use one processor to load the database, no matter how many processors you had," Berkus said. "With 8.4, you can use all of the cores on your machine. This parallelism speeds up database restore by two [times] to eight [times] depending on the number of cores you have and the structure of your database."

As far as query monitoring, users can now automatically log select EXPLAIN plans to the PostgreSQL log for analysis and make use of a new system that tracks how many times each of their functions are accessed. In addition, 43 new Dtrace/SystemTap probes have been added to aid users in tracing their servers in real time.

With Version 8.4 now out, contributors can focus on Version 8.5. Among the features slated for that release are hot standby, SQL/MED external data connections, column-level collations and index-only access. Users can expect that sometime in mid- to late 2010.

"We've used PostgreSQL for seven years now, and we're really looking forward to many of the features in 8.4, particularly column permissions, per-database locale, partial matches for GIN indexes and user-defined exceptions," Jeffrey Webster, CTO of ZooLoo.com, said in a statement. "PostgreSQL has allowed us to grow without sacrificing data integrity."