To Marine Corps Sgt. Marco Garcia, the Navy Marine Corps Intranet sounds like a great project, in theory. “They sold me on it,” he says, when he attended training in December on the new network. “The transition itself, though, seems to be a problem.”
Turns out, that assessment is an understatement. The project being deployed under the management of Electronic Data Systems (EDS) has been a headache for all involved. Navy planners originally thought they would hire an outsourcer in 2000 and have an upgraded and secure network in 2001. Now the conversion of nearly 350,000 computer “seats” has slipped to at least 2005.
Meanwhile, EDS is suffering because its billing depends on meeting required service levels and on the number of computer desktop and laptop workstations, or seats, deployed. Despite winning whats said to be the largest federal information- technology contract ever, worth as much as $8.8 billion, EDS has recorded a net loss of $1 billion on the venture so far.
Sgt. Garcia, responsible for supporting information systems for the logistics unit at his base in New Orleans, is frustrated that delivery of new Navy Marine Corps Intranet (NMCI) computer equipment keeps being delayed, and he cant order replacements in the meantime.
Despite the “intranet” name, NMCI involves more than setting up private Web sites. While deployment of personal-computer seats is a measure of the project, it also includes servers, data centers and help desks. Overall, NMCI aims to unify and standardize Navy networks, most of which were established years ago by base commanders who procured their own computers and hired their own contractors.
With NMCI, the Navy established a master contract. EDS is also responsible for overseeing other major participants, including Microsoft, Dell, Cisco, MCI, Wam!Net and Raytheon, and smaller subcontractors.
Check out eWEEK.coms Server and Networking Center at http://servers.eweek.com for the latest news, views and analysis on servers, switches and networking protocols for the enterprise and small businesses.
For the most part, the NMCI contract excludes the tactical networks used aboard ships or by Marines deployed to combat zones. In other words, NMCI is for what the Navy calls “shoreside” networks, primarily at bases in the continental United States.
EDS seems to have badly underestimated the complexity of the project in its eagerness to be the low bidder, committing to accomplishing the project at a fixed price without an adequate understanding of the obstacles ahead. “I think they should have known better,” says Lorrie Scardino, a Gartner Inc. analyst who follows the outsourcing market.
Mike Koehler, a former Delta Air Lines technology vice president who joined EDS in January and is assigned to help straighten out the project, agrees its “shameful” that the company let itself and its client believe this was just another project. “This is by far the largest distributed environment Ive ever seen and EDS has ever seen. Things that in a normal environment we would have been able to deal with become magnified at this scale and size,” says Koehler, the enterprise client executive for NMCI at EDS. “Grains of sand become mountains.”
Among the pitfalls:
- A minefield of “legacy” applications. The project uncovered more old software than expected, which must either be approved for use on NMCI, discarded or maintained separately.
- Cultural resistance and distrust. Independent-minded base commanders and local information-systems specialists werent necessarily quick to cooperate with the transition to a centralized system.
- The few, the proud, the Marines. Though organized as a branch of the Navy, the Marine Corps wanted to opt out and continue managing its own networks.
- Congressional meddling. Congress initially capped NMCI deployment at 60,000 seats and kept the Marines from being included. The last of those restrictions was lifted in 2003, after EDS demonstrated it had met the minimum requirements for 28 key service-level goals, ranging from user satisfaction to network availability.
- Coordination. Until the Navy established a central project office in 2002, EDS was forced to work out deployment plans separately with different organizations within the Navy, such as NAVAIR (Naval Air Systems Command) and NAVSUP (supply management).
Military planners had been thinking about a centralized, standardized network since the early 1990s. Outsourcing promised a way of getting a vendor to make the capital investment and contribute the skilled workforce, while turning ongoing upgrades and system management into a predictable expense.
The contract EDS won in 2000 was valued at up to $6.9 billion and in 2002 was expanded to an estimated $8.8 billion over 10 years (seven years, with a three-year option). The contract set maximum-pricing and minimum-performance requirements, while leaving room for schedule changes. By the beginning of this year, EDS had assumed network-management responsibilities for 303,000 seats, but as of January only about half of those—154,000—had been upgraded and certified as meeting the new network standards. In February, the Navy projected the conversion of all 346,133 seats would be done this year, but EDS has already said it will need until 2005.
EDS says the push to deploy lots of seats quickly at many locations inflated expenses and hurt cash flow because of the outlay for new computers. Now EDS wants to finish the job one base at a time.
One ex-sailor, who is currently working on NMCI for one of EDSs subcontractors, says the new strategy should work better, even if it takes longer. EDSs insistence on “trying to push as many seats as possible” created its own problems, he says, meaning technicians would often have to return to correct work that wasnt done right the first time.
The Navy has been largely insulated from EDSs cost overruns by the fixed-price contract. But EDS has signaled it will get more aggressive about billing extra for work it considers outside the bounds of the contract.
EDS stands to earn more by winning performance-incentive bonuses written into the contract for customer satisfaction. Those bonuses start when the user-satisfaction rate hits 85% and, if EDS could get the satisfaction rating above 95%, could be worth as much as $146 million annually. But so far that rating hasnt even broken 70%.
The users who express the lowest satisfaction tend to be those who had the best computer equipment and support prior to the introduction of NMCI. For example, only a little over 50% of naval aviators give NMCI a positive rating, and they have more complaints about the new restrictions on computer use. On the other hand, more than 75% of reservists, who were relatively poorly equipped in the past, rate NMCI favorably.
Complaints notwithstanding, Capt. Chris Christopher maintains the project is a success. A reservist voluntarily recalled to service to help with management of the Navys Year 2000 remediation, Christopher now serves as staff director of the NMCI project office.
“Without NMCI, we never would have got our arms around our application portfolio, which was totally unmanaged,” Christopher says. For decades, anyone with procurement authority could buy computers and software or hire developers to create custom applications. Security policies were widely ignored.
The Navy recognized that one of the challenges of NMCI would be replacing, consolidating or eliminating those legacy applications, since only officially sanctioned software would be installed on the new computers. Based on an application inventory performed for the Navys project to eliminate Year 2000 bugs, they thought they might have to deal with perhaps 9,000 of them. But NMCI turned up about 100,000.
So far, 3,000 applications have been approved for use with NMCI. Another 24,000 applications have been “quarantined,” meaning they cant be run on NMCI computers or connected to the new network, but will continue to be used until they can be replaced or retired. For example, NMCI wont allow applications that rely on NetBIOS, a protocol Windows networks use for file sharing, because it provides the potential for access to the desktop computers hard drive.
Barry Cason, a civilian employee for the Marine Corps who works at a supply warehouse in Albany, Ga., says he ran into trouble when NMCI technicians refused to load an essential piece of software onto his new computer. “So that wasnt so good,” he says.
Although he has suffered some system crashes using his new Windows 2000 computer to run the program, which was written for the older Microsoft Disk Operating System, he needs it to run the freight carousel at the warehouse. Since getting approval to continue using the software, Cason says he has been basically satisfied with the new network.
Network-security experts tend to be the biggest fans of the Navy intranet. Richard C. McElroy became convinced in 2002, while working as a security consultant on the project. His assignment included network-penetration testing and security analysis of products deployed on NMCI.
Another problem: ranking officers sometimes ordered security restrictions loosened for their own convenience—to get access to streaming video that otherwise would have been blocked by a firewall, for example. “Now its clear that these are the standards, and everyone has to adhere,” McElroy says.
Despite its emphasis on security, in August NMCI suffered a days worth of disruption from Welchia, one of the network-clogging computer worms making the rounds last summer. But in January, NMCI officials say they met the MyDoom.A worm with the sort of organizationwide response that would have been impossible previously. On Jan. 26, the day antivirus software vendor Symantec notified them of the threat, they reconfigured e-mail servers to block file attachments associated with the attack, and then distributed an antivirus update as soon as it was available.
NMCI holds out the promise of better training for Navy technicians. For example, Ernest Bell, a Navy petty officer first class and information-technology specialist, is getting Microsoft Certified Systems Engineer training while serving a shore rotation on the network operations centers help desk at the Navy base in Norfolk, Va. He says that experience will prove valuable when he gets his next shipboard assignment.
When fielding calls from frustrated users, hes careful to keep his cool. “You dont want to have the same attitude that they have,” he says. “Because theyre not mad at me, right?” Many complaints involve simple oversights, such as users not being given the access to shared network directories, Bell says.
The NMCI project is “both good and bad,” says Will Harper, a former naval officer who teaches Microsoft Certified Systems Engineer courses for the Navy. He agrees NMCI is improving security. Users who are complaining are mostly mad about being denied network privileges they shouldnt have had in the first place, he says.
On the other hand, Harper faults the Navy for hiring the lowest bidder and EDS for low-balling its bid. As a result, Harper says, EDS is “scrambling to meet the requirements as best they can, given their cost overruns and the fact that theyre on a fixed-price contract.”
Full speed ahead seems to have turned out not to be the right strategy for the project. Now, Navy officials say they will work with EDS to steer a better course and make sure the network reaches its destination.
.S. Navy Base Case”>
U.S. Navy Base Case
Navy Marine Corps Intranet (NMCI) Office: 2231 Crystal Drive, Suite 400, Arlington, VA 22202
Phone: (703) 685-5510
Business: To provide the computers and networks for the “shoreside” operations of the U.S. Navy and Marine Corps.
Leaders: NMCI director Rear Admiral Charles Munns; Department of the Navy chief information officer Dave Wennergren.
Project Spending: $758.95 million in NMCI expenses booked in fiscal year 2003, more than $1.5 billion expected this year.
Challenge: Replace fragmented networks and inconsistent computer architecture with a single, secure enterprise network.
- Originally, to complete conversion by 2001; now aiming for sometime in 2005.
- Reduce the number of applications on the network from about 100,000 to fewer than 30,000.
- Make incentive payments to outsourcer when more than 85% of users report a favorable experience.
- Reduce total cost of ownership for each desktop or laptop computer, from $3,851 to $3,741.