eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Theres a dirty little secret in networking that many users have encountered but few talk about: NIC drivers can be a drag on performance and can even cause system crashes. But one small testing vendor is working to expose the problems and hold vendors accountable for the performance of their hardware and drivers.
To date, network-interface-card issues have been mostly an inconvenience. But the movement toward faster technologies such as Gigabit Ethernet, coupled with the increasing use of more latency-sensitive applications such as voice, video and storage over IP, is raising the profile of the problems caused by the drivers.
A number of factors have kept NIC performance issues on the back burner. At the desktop, the performance hit has not generally been felt.
“Five or six megabits per second is more than fast enough on the desktop. Your internal hard drive can only go so fast when youre copying from a network drive to your PC,” said Ronald Knol, IT and VFX technical supervisor for postproduction and visual effects company Rainmaker Limited Partnership, in Vancouver, British Columbia.
In addition, performance-testing tools have not been able to isolate bottlenecks down to the driver level. “Most people cant see this, so they probably dont hear about it a lot,” said Kelly Daniels, chief technology officer at Apparent Networks Inc., also in Vancouver.
Daniels said Apparent has developed a performance-testing tool called AppareNet and is working to bring the NIC driver issue to light. The company is testing a host of popular hardware and software combinations and reporting its findings to the manufacturers for resolution. Daniels said some companies, such as Intel Corp., have been good about making fixes, while others have been slower to react.
In the meantime, experts agree network administrators should consider NIC conflicts when troubleshooting performance issues and should test drivers before deployment.
AppareNet can uncover such driver-based issues as full-/half-duplex mismatches, Message Transmission Unit conflicts, reordered packets and more. The tool goes in below the TCP protocol stack to isolate NIC and driver issues and measure the effect the NIC and its driver have on traffic flow.
“They bypass the … operating system and reside on the interface. There is no overhead from the OS, nothing from the PC itself thats going to prevent it from driving [full bandwidth],” said Dana Richard, an AppareNet user at storage-over-IP vendor Computer Network Technology Corp., in Minneapolis. “Only the Ethernet driver itself could do that.”
Richard said that when he tested various Fast Ethernet interface drivers, “you could see one give you 20M bps, another 50M bps, another 60M bps. In each case, when you hit the top end, packets would drop, get out of order or corrupt the driver—things you expect when overdriving an interface.”
Next page: Addressing the bottleneck.
Addressing the Bottleneck
The bottleneck is an issue many hardware makers are trying to address, with varying degrees of success. Generally speaking, the first generation of a new networking technology is not capable of “driving line rates,” said Carl Wilson, technical marketing engineer at Intel, in Hillsboro, Ore.
And when making the transition to a faster speed, it can take as long as a year for drivers to be updated to accommodate the faster line rates, said Apparent Networks Daniels.
“The common wisdom is that every time a manufacturer puts out a new driver, you should upgrade to it, although we never have any hard proof that it solves any problems,” said Kevin Baradet, an eWEEK Corporate Partner and CTO for the S.C. Johnson School of Management at Cornell University, in Ithaca, N.Y.
Most driver performance issues come up in the auto-negotiation phase of a connection, which determines the speed of the link and whether it will operate at full duplex or half duplex.
“The No. 1 issue I found was half-/full-duplex conflict,” said Fred Klassen, Apparent Networks co-founder and vice president of network technologies.
Intels Wilson agreed. “Thats primarily because the 10/100 auto-negotiation [specification] was not well-defined. There are a lot of areas open for interpretation,” he said.
Beyond the common issues, a host of more obscure problems can occur that are harder to isolate. For example, Klassen, who has specialized in troubleshooting hard-to-solve networking problems for 26 years, found that a particular driver level for Broadcom Corp.s NICs would drop packets or run slowly when linked to a Cisco Systems Inc. or Alcatel switch, but “if they are plugged into a 3Com [Corp.] switch or crossover cable, its the fastest card out there,” he said.
But with the increasing use of VOIP (voice over IP) and the “convergence of voice, video and data, that is where itll start getting dicey,” said CNTs Richard. “Also, with the advent of Gigabit Ethernet, it very well could be [the same problem over again]. If anybody thats going to deploy any one of these in a meaningful way is looking for performance, Id highly recommend that they test them thoroughly before they invest. If we hadnt, wed have wasted several thousand dollars.”