Opinion: Testing enables informed choice, but it doesn't make choices easy.
Were barely past the halfway point of 2006, but the most-cited
article that eWEEK Labs will produce all year may already be possible
to predict. The Labs deep-digging comparison of multiple top-to-bottom IT stacks
will please the pragmatists and prickle the purists of both the open-source and proprietary-software persuasions.
Im tempted to eschew summarization in the hope that youll go read
the whole thing because much of the value of this work is in the
details. Even so, I think I can safely share one top-level conclusion:
Dont think that your choice is merely "Windows versus Linux," or even
a somewhat less simple ".Net versus Java versus LAMP." Open-source
midtier components on a Windows foundation did rather well, and the
relative strengths of various stack combinations will present different
pictures under different workloads.
This report should inspire you to think differently about the
choices available to you, not give you an excuse to stop thinking and
just adopt one monolith or another.
In this podcast, eWEEK Labs analysts discuss the results from their tests of IT stacks. Click here to listen.
Other points also occurred to me as I was looking over these
results. First, note the consideration that went into the testing philosophy
and the choice of associated tools. The conclusions from
the Labs come not from tests to destruction, but from observation of
behavior in the course of typical tasks and burdens. Its easier just
to load things up until they break, but which would you rather have: a
stack thats uniformly a nuisance to configure and maintain, at any
workload from negligible to near-infinite, or one thats convenient and
well-behaved over a range of workloads that handily covers all of your
Let the prospective buyer beware: If a product is promoted solely on
the basis of its capacity, perhaps it falls short in day-to-day
operability, and conversely.
I also recalled some of the issues that I raised in my report last
year on the broadening scope of the testing
that a development shop must be
willing and able to address. Its no longer enough to verify that an
application, or even a top-to-bottom stack, can do what its supposed
to do under the conditions that its builders had in mind. Envisioning
the unlikely and defending against the malicious are also part of
todays definition of due diligence.
I can imagine that choosing a single-source solution might reduce
the number of points of weakness, but I actually lean more in the
opposite direction: It seems likely to me that stack components from
multiple sources will have interfaces that are (i) more visible and
(ii) more likely to be known and understood by the time you get the
whole thing working. Well-integrated single-source stacks may offer a
simpler out-of-the-box experience to the buyer, but that simplicity may
reflect unnecessarily powerful and inadequately defended mechanisms
that are silently enabled by default.
Tell me about the stacks youre building at email@example.com
Check out eWEEK.coms for the latest news, reviews and analysis in programming environments and developer tools.