How We Tested

 
 
By Craig Ellison  |  Posted 2003-03-25 Print this article Print
 
 
 
 
 
 
 


How We Tested

In this roundup of early pre-standard products, we tested the Buffalo AirStation WBR-G54, the D-Link AirPlus Xtreme G DI-624, and the Linksys Wireless-G WRT54G routers, as well as each companys 802.11g network card. Note that each of the products has a "G" and perhaps a "54" in its name. So even though theres no 802.11g standard yet—and 54 Mbps is also the maximum signaling rate used in 802.11a products—the naming convention for these products may confuse consumers if the spec is updated.

Two manufacturers are currently supplying chipsets. The 54G chipset, made by Broadcom, is found in the Buffalo and Linksys products, and the Prism chipset, made by Intersil, is used in the D-Link products.

Because of the interoperability issues identified above, weve added a significant number of tests to our traditional wireless testing suite; these measure performance in various modes and interoperability among vendors and legacy 802.11b clients. All of our tests were performed using NetIQs Chariot, a respected network performance analysis tool (www.netiq.com).

For starters, we tested 802.11g network throughput at various distances. We tested with only one client and used 802.11g-only mode (not mixed mode).

The Buffalo and Linksys routers can be set to G-Turbo mode. In this mode, a router would never go into protected mode, and 802.11b clients would never "see" the router. This mode locks out 802.11b users (and hence isnt an option for businesses with an investment in 802.11b clients), but it yields the maximum speed the products are capable of producing.

As the chart on page 42 shows, throughput at close distances ranged from about 16 to 21 Mbps. As expected, it dropped off with distance. At 80 feet, for example, throughput was only slightly better than half what it was at close distances. But true to the grand scheme, 802.11g performance tracked closely with that from the most recent 802.11a equipment we tested ("Dual-Band Wireless Cards Are Not All the Same," First Looks, December 24, 2002).

Our second test was designed to evaluate the best possible performance of an 802.11g client on a network running in mixed mode (the most likely scenario).

We configured the router for mixed-mode operation and checked performance at a close distance using an 802.11g client. With no 802.11b client present, the client and router should have turned in the same performance as they did in "g"-only mode. We then booted up a notebook with an Orinoco 802.11b card and associated it to the router. The association of an 802.11b should force the wireless router into protected mode. For this test, we sent traffic only to the 802.11g client.

In this scenario, performance is reduced by the overhead of the protection mechanism that ensures that "b" and "g" clients can coexist. We dont chart the results here, since they generally tracked with the 802.11g-only results, but with a 10 to 20 percent reduction in throughput at any given distance.

Our third test (also not charted) was designed to measure performance in protected mode with traffic being sent to both 802.11g and 802.11b clients. For this test, we used one 802.11g client and one 802.11b client (the Orinoco) at close distances. Since both clients had equal access to the network, youd expect their individual throughput figures to be approximately the same, given that they were sending the same amount of data in each packet.

Indeed, thats the performance we saw with both the Buffalo and Linksys products. Each client was getting about 3 Mbps of throughput, yielding a total network performance that was slightly better than 6 Mbps. (Of course, as you increase the number of "g" clients on the network, the total network performance improves, since the higher throughput of each additional "g" client will pull up the average).

Unlike the Buffalo and Linksys products, the D-Link router/ NIC combination was show- ing about a 5:1 preference in performance for the "b" client. D-Link is aware of this issue, and a patch, not available by our deadline, should address the problem.

Our final tests (shown on the second chart) focused on vendor interoperability. For these tests, we created a devils matrix and tested each manufacturers card against the other manufacturers wireless routers. We tested for 802.11g performance at close distances. For routers that supported multiple configurable modes, we also tested several legacy cards in both mixed and Wi-Fi–only modes.

This test matrix yielded some interesting results. Most important, each manufacturers network card was able to connect successfully to every other manufacturers router and deliver acceptable throughput.

Initially, however, we did experience difficulty with an 802.11b legacy Cisco Aironet 350 card associating with either the Buffalo or the Linksys router configured for automatic mode. Each vendor provided a firmware patch that resolved the problem. After we updated the products, both legacy 802.11b clients turned in expected performance using all routers.



 
 
 
 
Craig Ellison is PC Magazine Labs' director of operations. The Labs staff, in consultation with PC industry experts, develops procedures and scripts for the independent and impartial testing underlying all PC Magazine reviews.
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Rocket Fuel