Testing 10/100M-BPS ethernet switches is not nearly as boring as it might sound. During eWeek Labs tests of the Extreme Summit 48si, we used Spirent Communications Inc.s SmartBits hardware and SmartFlow software to test the new Layer 3 features.
To gauge the performance of the Summit 48si—including the number of supported multicast sessions, quality-of- service and rate-limiting capabilities as well as such security features as access control lists and virtual LANs—we used a series of standard tests that incorporated some twists. For example, the default setup for SmartFlow, a test suite that generates repeatable, measurable network traffic flows, is to have one IP address assigned per flow. We used 20 IP addresses per flow on the Summit 48si to more realistically stress the device.
We also juiced up the tests by using random packet sizes ranging from 64 bytes to 1,518 bytes. We saved these configurations so we could run them on the Extreme switch and on other vendors devices to get roughly comparable results. Random packet sizes make switch hardware “think” more, which yields a better prediction of performance under real network loads.
Still, these tests are mere attempts at simulating the real world because it is just as unlikely that a switch will encounter completely random packet sizes in all traffic as it is that all the packets will be perfectly formed and of uniform length.
Further, different configuration parameters among the Cisco Systems Inc., Extreme Networks Inc. and Foundry Systems Inc. equipment made test results only approximations of relative performance. IT managers should know the characteristics of the traffic on their network—the amount, likely packet size, security rules and user restrictions—and use that information to make decisions about performance.
We tested the Summit 48si by connecting the ports to three SmartBits 6000 chassis that were cabled to act as one test-generation unit. We conducted subsequent tests with a single SmartBits 6000 chassis with 48 test ports. We ran our initial tests for 30 seconds to get preliminary results, but to prove out the numbers on the Summit 48si, we ran the tests for 5 minutes. We tested three traffic sizes to get a better idea of switch performance: 64 bytes to stress raw packet handling, random to get a flavor of real life and 1,518 bytes to benchmark large packet forwarding.
We used the Spirent gear to gauge throughput (the percentage of wire speed at which traffic passes with no packet loss) and forwarding rate (the number of packets forwarded through the switch when the input load is maintained at 100 percent). Both numbers show interesting aspects of performance.
Throughput is usually most interesting to IT managers who have a lot of connection-oriented TCP traffic. This is because as throughput drops, applications usually have to make adjustments that slow data transmission, and that means slower performance. The forwarding rate is a useful measure of what is likely to happen under intense loads. In SmartFlow, the throughput test is considered a “fail” even if only one packet is dropped.
If a switch has desirable features but has a low throughput rate as measured by SmartFlow, IT managers should take the numbers with a grain of salt. We spent a lot of time running test iterations that were measured in 1 percent differences of traffic load to find the numbers in our review of the Summit 48si. Put another way, devices may actually suit an enterprises needs despite a “failed” throughput test.
We ran all our tests in “full mesh” mode: Every port sends traffic to every other port, an extremely taxing test configuration but one that yields more interesting results for IT managers who are planning potentially complex networks. The opposite of full mesh is to have traffic flows go between set pairs of ports—a much less rigorous test but one that could be the basis for making a decision for a stable network environment.