Technology has evolved to the point that we now have collision avoidance systems for automobiles. So why don't we have mistake avoidance systems for data managers? Many of the issues that compromise data center performance stem from mistakes that could have been avoided. It's often the little things that take you down, but that "small" oversight can cause an outage that can bring a company to its knees.
There are hard lessons to be learned in the data center and most of us will endure a few of them during our careers. What's ultimately important is how we improve from these experiences. It is imperative that every data center manager follow what I call the golden rule of data center management: Whenever possible, learn from the mistakes made by data center managers everywhere. There are five steps you can take to do this.
Step No. 1: First, have the self-confidence to admit your mistakes.
Owning up to errors isn't an acknowledgement of failure. On the contrary, it's an indicator that you want to be proactive, learn more and operate the best possible facility.
Step No. 2: For an ongoing learning exercise, put your staff in "safe" situations where mistakes can be made without consequences.
Since it's difficult to conduct "what-if" scenarios in a live facility, I recommend using scripted role-playing and reenactments, followed by group discussion. This is a great way to develop team chemistry and foster conversations that can potentially resolve issues before they occur.
Step No. 3: Many data centers have found it helpful to create a "lessons learned" binder for their organization.
This binder contains descriptions and solutions to procedures, processes and equipment failures that may have happened at their facility or elsewhere. It serves as a reference guide for the team and provides guidance for new employees. I recommend reviewing new inserts to the binder during quarterly meetings.
Step No. 4: If you are building a new data center, I strongly recommend including a commissioning process where a third-party consultant, separate from the design and contractor teams, is involved at all stages.
This provides a set of fresh eyes that can identify potential problems and an objective source of insight to best practices. During construction, the commissioning agent can offer expertise on decisions regarding electrical conduit, proper equipment sizing and more. They can help you value engineer the project as it is moving along. When the data center is completed, the commissioning agent is a critical evaluator during the testing process. As part of the commissioning process, I suggest asking your commissioning agent to help you develop a learning exercise that can give your staff the chance to kick the tires of all systems without penalty.
Step No. 5: Finally, be courageous about making changes.
It might involve expense and pain, but asking for more money or time is far better than the potential consequences.