Why Companies Get DevOps and Continuous Delivery Wrong

We examine mistakes companies make in their DevOps practices and offer tips for testing, tool selection and information sharing to help fix some of these problems.

In DevOps, it is often the mentality that everything has to happen as quickly as possible. This can mean rushing things out the door with little or no testing. Legacy patterns of multi-day manual testing and multi-week performance test scripting are often cited as the reasons for choosing to “test in production” in place of proactive testing. The myth is that you can’t have both speed and quality, but the reality is that equipping dev teams to write their own tests with their own tools changes the game. Committing tests along with code makes near-instantaneous feedback for every build, every deployment candidate possible.

A common misconception about load testing is that it can only take place at the end of the lifecycle, as traditionally developers had to wait for performance tests to be developed and run by someone else. Waiting until the end of the process to test for performance adds unnecessary drama and surprises. Modern open-source tools and domain specific languages have made developer-driven performance testing possible. Developers who are already taking responsibility for functional tests are beginning to use low-volume performance tests to examine functionality and performance in a single pass, especially with API endpoints. These endpoint tests then become building blocks for automated integration, deployment and post-production tests.

Another common false idea about performance testing is that a test suite must be run sequentially. This goes back to the “nightly build” pattern in which several hours of tests were the norm. It is more efficient to run multiple smaller tests in parallel because the combined test will only take as long as the longest piece takes to run. For example, if you are running 12 tests of three minutes each and three that are four minutes long, running them sequentially will take at least 48 minutes, but running them in parallel will take just four minutes.

Whether developing a mobile or web app, open-source tools are becoming the norm for developers. Developers tend to have a favorite open-source tool that they use, and they are wary about using other tools that others impose on them. This doesn’t need to be a problem, as frameworks are emerging that “play nice” with a wide variety of tools. The key is to have a unified orchestration framework that individual tools can snap into. No matter what tool you use, the process of load testing can be part of a single unified whole without imposing restrictions on tool choice.

In today’s enterprise, the lines continue to blur between the roles and responsibilities on developer teams; this is especially true as DevOps becomes the norm. In some cases, the testing function can fall to a select few “engineer in test” developers, which limits the ability for tests to be run at all stages in the lifestyle. Ultimately, the ability to test for performance and quality should be a shared function, allowing for many developers to understand and share the same, achievable goal, break down silos that may exist in departments and focus on achieving performance throughout the process, not just at the end.

In large organizations, smaller, self-sufficient teams, rather than large departments are the new norm. When there are minimal interactions between teams, each one can implement DevOps and continuous delivery tools that meet their own needs. Instead of using one companywide set of identical and mandatory tools, teams should be empowered to implement tools and technologies tailored to their own needs. Understanding the process and goals prior to implementing load testing or DevOps tools will ensure project success and better align all teams toward a common goal. Once this understanding is in place, teams can execute in the way best for them, using the open-source tools they are comfortable with that help the team reach its broader goals.

For any technology implementation, but especially DevOps, it is crucial to understand that what will be introduced is a process using the tools—and not the other way around. This means the process and combined requirements of the organization must be understood before trying to fulfill the technical requirements. To do this effectively, all people across the organization, no matter what their role or department, need to be on the same page. To have a fully coordinated approach, one person should be appointed as a continuous delivery engineer, who will take time speaking with each department to understand their needs and coordinate a companywide approach. The continuous delivery engineer will boost visibility into individual needs while also helping develop an overall strategy.