Organizations worldwide have adopted Lean and Agile ways of working to cope with their current disruptive markets. Lean Startup principles are adopted by large multinational corporations, and Agile methodologies have outgrown the IT department, towards primary processes in lawyer firms, schools and construction agencies. This, however, does not say that these organizations actually bring new or adapted software to production with the required speed and frequency. Predominantly during this final step (often referred to as “the last mile”) the delivery hampers. The root cause? The organization has too many silos, which are not (enough) connected.
Who hasn’t seen them: IT departments where designers, developers, testers, support and operations live in splendid isolation from each other, with a minimal level of collaboration. The designers cherish their own requirements and methodologies, developers work on their code (possibly in Scrum teams), after which the results are pulled through the test factory, in order to be thrown over the operations wall at the end. Products, as delivered by the development teams (Scrum has named these “potentially shippable products”, or PSP), pile up in front of operations’ doorstep. By the way, using the PSP term consistently in Scrum implementation worldwide, has contributed greatly to the divergence of responsibility in the value chain. After all, from the (Scrum) developer point of view, their job was “done” once it was potentially shippable, hence on a pallet, waiting to be shipped. At that time, it still does not delivery any value at all! But the developer considered it done, as their work was done. No relation to value whatsoever.
And when these product increments are eventually implemented in a large release, it takes unacceptably long before errors can be related to their source. Integration problems don’t come to the surface before the tester is running the acceptance tests. And what about the customer satisfaction, if users are constantly faced with delays and unavailabilities? In short, DevOps addresses the need for higher user satisfaction, a dynamic balance between value and risks, shorter time to market, and more efficiency in the end-to-end chain through cross-functional collaboration.
The Three Ways
As beautifully illustrated in the DevOps bible “The Phoenix Project”, the value IT can deliver to an organization is completely dependent on its ability to make the organization collaborate as a whole. Although the name suggests only Development (Dev) and Operations (Ops) will more closely collaborate, the essence is much broader than just that. Bringing together Dev and Ops is referred to as “DevOps Lite” (after Patrick Debois), whereas true DevOps also entails the integration of crucial roles such as the business, testing/QA and security. This holistic thinking is the first principle (The First Way) of DevOps.
Besides that, it is considered fundamental to DevOps to not only have (mostly Agile) development teams deliver “potentially shippable products”, but to have the target deployment environments available as well (provisioning). Clearly this is where DevOps takes Agile implementations one step further, thereby providing the IT organization more valuable feedback (The Second Way) on the quality of the delivered products. Surely automation plays an essential role here. Without a high degree of automation, it is virtually impossible to provision and synchronize (DTAP) environments in a fast and standardized way.
Probably the most fundamental shift which is part of the DevOps way of working, is the way errors and risks are dealt with. Traditional organizations tend to have a cultural heritage where errors are being punished, hence covered up. DevOps organizations assume that errors and experiments are excellent, as they improve the organization’s resilience (The Third Way).It enhances the organizational capability to learn, moving these types of organizations towards a state of “antifragility” (Nicholas Taleb). They are known for their ability to absorb disturbances, even grow from them, and continuously adapt to changing circumstances.
The revolutionary aspect of DevOps is not about the individual components it touches. It is the contextual combination and application of these frameworks, methods and movements. The following essential relations are identified in relation to DevOps:
Agile: Many of the principles applied in organizations that have adopted DevOps, concur with the Agile principles. Think of short feedback loops, minimizing unit size and fast fl ow of planned work.
Lean: The Lean way of thinking is not only applicable to the factory floor. Lean elements such as Voice of the Customer, Flow, Pull and Kaizen are used more and more in IT organizations. Waste is reduced and errors are identified and solved at the source (“no defects downstream”).
Theory of Constraints: This methodology, related to Lean, is characterized by the elimination of bottlenecks. By consistently searching for essential limitations in your organization’s product and service fl ows, these constraints (or bottlenecks) can be taken away adequately.
ITIL: Without a doubt, ITIL also plays a significant role in DevOps organizations. If well applied, the introduction of Agile and Lean principles and instruments in the entire IT delivery chain (so including operations and support) account for faster and more fl exible service management processes. Take Configuration Management, which is crucial in DevOps in sharing information between several roles and domains.
Cloud: Many organizations have started their transformation to the cloud, either partly or full blown. Cloud technology enables fast provisioning , adjustment (scaling up/down) and synchronization of (DTAP) environments and in automating several build, integration, test or deployment tasks.
Typical patterns we encounter in DevOps environments include:
• Continuous Delivery
Delivery pipelines are automated, resulting in practices like continuous integration, continuous deployment, automated testing.
• Software Defined Anything
Servers, even entire networks are software defi ned nowadays. Physical, on-premise hardware is replaced by virtual machines and containers.
• Agile architecture
Huge monolythic applications are replaced by microservices, enabling fast feedback, low regression testing and maximizing the use of market standardization.
• Service flow
By using Lean processes, a value-driven approach challenging the end-to-end performance of the value stream, continuously optimizing batch size and queues.
• Functional vs non-functional requirements
A sound balance between functional and non-functional system behavior requires professional product ownership, but also built-in quality, security and monitoring.
• Learning culture
Failure is regarded as valuable learning points instead of opportunities for punishment, resulting in blameless postmortems and rewards for positive experimental behavior.