The peer review process

By Darryl K. Taft  |  Posted 2004-03-08 Print this article Print

There is no way to mechanically check the results of creative work. If you could do that, it wouldnt be creative work; it would be something you could do with a machine. So the only way to check complex, creative work for correctness is by the critical judgment of peer experts. This is why in science you can form a hypothesis about the sex life of hamsters or something … and you can come up with an incredibly elegant experimental design and a beautiful theory and you can test that theory—construct an experiment, test that theory, come up with results that either confirm or disconfirm your theory. But that in itself is not enough to get your experiment accepted as part of the core of scientific knowledge. Before that happens it has to undergo peer review. Other scientists, independent of your research group, need to check your results, confirm them and ferret out any hidden assumptions youve made that might invalidate the results youre presenting. And this is not just a process that goes on in theoretical science. If youre a civil engineer and youre constructing a suspension bridge or a skyscraper, you are not going to get to string wire or pour concrete before peer engineers who are not associated with your design group skeptically review that design for flaws that might kill people. Youre going to be required to do that. If the local government doesnt require it, your insurance company is going to require it. And the reason thats the case is weve discovered when youre doing large, complex engineering projects, peer review is a critically important way to mitigate risk. And what my friends and I in the open-source community are doing is saying, "Hey, wait a second, that applies to software, too." In engineering, whether its bridge building, aeronautical engineering or anything else, you get quality out of a peer review process thats transparent from top to bottom—where the assumptions, the working methods and the results are all subject to peer review at every stage of the process. Well, by gosh, the same thing works for software. We know in general, from our experience in building things like bridges and airplanes, we know that secrecy is the enemy of quality. Concealment is the enemy of good results. And if you look at it from that point of view, it becomes clear that open source is not an exceptional phenomenon. Its not some weird new idea that came out of nowhere. Its just good science. Its just engineering.
The aberrant thing, the weird thing, is closed source.
Next page: The Internet is an example of how open-souce development can scale.

Darryl K. Taft covers the development tools and developer-related issues beat from his office in Baltimore. He has more than 10 years of experience in the business and is always looking for the next scoop. Taft is a member of the Association for Computing Machinery (ACM) and was named 'one of the most active middleware reporters in the world' by The Middleware Co. He also has his own card in the 'Who's Who in Enterprise Java' deck.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel