The agency turns to NetBeans, other open-source tools to project the water supply.At its heart, predicting the supply of water is a math problem—an extremely complicated math problem.
Algorithms are created to gauge the effects of melting snow, precipitation, daily temperatures and the flow of rivers and streams. The payoff for all these dry formulas is a set of predictive models that can help municipalities ensure that folks will have enough of the
wet stuff to drink, bathe, wash their cars and water their lawns.
Lording over these complicated calculations is the responsibility of the Department of Agricultures Natural Resources Conservation Services. NRCS, in Portland, Ore., issues monthly water-supply forecasts for more than 750 places in the western United States. The compute-intensive job requires some 250 developers located across the country, according to Frank Geter, national modeling specialist at NRCS. It also, Geter points out, requires "a robust, collaborative software development system."
Research Scientist Olaf David, from Colorado State University, has been working with the USDAs Agricultural Resource Services in Fort Collins, Colo., for the past two years to develop such a system to be used by NRCS and a host of other USDA agencies. Early in the process, it became clear that the entire USDA research community needed decentralized, network-dispersed, team-oriented code development and project management.
As it stood, ARS had more than 100 models for a variety of purposes that had been developed on a case-by-case basis, using whatever technology was available at the time. That disjointed system was proving difficult and expensive to maintain, and efforts to edit and improve the models were complicated. David insisted on a new object-modeling system that supported Web-based team communication, issue tracking, user access permissions, collaborative code and document management, while still supporting USDAs existing version control systems. And it had to be easy to install and maintain. David and his team chose a platform consisting of open-source development tools from Sun Microsystems, Intland Software and Collab_Net.
Read more here about Sun beefing up NetBeans.
"We standardized on [Suns] NetBeans-based modeling platform," David said. "We chose NetBeans, [Intlands] CodeBeamer and Subversion [from Collab_net] because its the only integrated collaboration solution that could handle development and deployment of simulation models in order to support our modeling projects."
In 2004, in cooperation with the Environmental Protection Agency and the U.S. Geological Survey, USDA began prototyping the new system, which went into production in 2005. According to David, the resulting platform enables the kind of collaboration critical at ARS, which often has up to 300 simultaneous software development projects in the works. Through the integrated infrastructure, NetBeans developers are able to collaborate, share knowledge and work effectively as a team from different research locations.
"Its important to note that its not just developers working on code," David said. "We also have data analysts, others doing geospatial processing, fixing the parameters of the models based on data sets such as elevation and vegetation layers. And some are pure scientists that have no clue about coding. They just want to get the process right. The system has to support them all."
Davids team supports all those efforts from USDAs development and data center hub known as the Collaborative Software Development Laboratory, or CoLab, in Fort Collins. "Through access to CoLab, people can now develop, and we now collaborate with them providing documentation, testing source code and delivering the source code in a way that lets them access and download from wherever they are."
CoLab currently supports more than 700 registered users from 15 different institutions accessing a 20GB code repository and some 4GB of documents. Within CoLab, users can store documentation, browse and search source code, and set up automatic builds to compile, test and run models. Since CoLab went live, users have seen significant decreases in project delivery times, better project management and an overall improvement in the quality of source code, David said.
The modularity and reusability of code makes the NetBeans platform particularly well-suited to the development work being done by USDA and its agencies, said Jeet Kaul, vice president of developer products and programs at Sun, in Santa Clara, Calif.
"The platform provides the services common to desktop applications, such as window and menu management, storage, and so forth," Kaul said. "These tedious aspects of writing an application now come for free. The result is that developers get to concentrate on the business logic that implements what the application is actually supposed to do, rather than spend the time rewriting what almost every application needs."
With the Java and NetBeans development environment established at CoLab, David said he can focus his attention to specific projects, such as the water-supply forecasting models at the NRCS resource center. Researchers and analysts at NRCS are working to port a valuable, if dated, Fortran-based precipitation runoff model to aid the agencys short-term water analysis and forecasting. NetBeans is proving valuable in building the foundation for an updated modeling platform for NRCS, according to David.
"The older model was written in the 1980s sometime," David said of the compute-intensive forecasting tool. "The challenge was to bring it into Java, then run it in a modeling framework that allows you to be flexible with different approaches. The framework we settled on is based on Java and NetBeans. The whole objective for us was to take this model and have it available in Java and run it in batch mode across a number of geographic areas."
Part of the projects charter was to make the developers work future-proof as well. "Because the NetBeans platform is based on standards and reusable components, pure Java applications based on it will work on any platform that supports Java 2 Standard Edition, including Windows, Linux, Mac OS X, Solaris, HP-UX, OpenVMS, OS/2 and others," said Suns Kaul. "The world of software is constantly changing. One way to be sure that what you write today will stand the test of time is to use standards and write cross-platform applications.
"That is what the NetBeans platform is for. Applications based on the NetBeans platform do not require proprietary binary libraries or components—just a Java 2 run-time environment on the target platform," Kaul said.
Check out eWEEK.coms for the latest open-source news, reviews and analysis.
Linux & Open Source 
