The Service Delivery Assessment
Over the years, a number of high performing software organizations have figured out many of the keys to delivering software with both speed and quality, what is now collectively known as DevOps. When the Accelerate State of DevOps Report was released in 2019, it used this knowledge to create a North Star as to the modern way of delivering software.
With the 2019 Report, and its companion book Accelerate (describing the science behind the report), we became able to assert that “highest performers were twice as likely to meet or exceed their organization’s performance goals”. Those goals are things like revenue, customer satisfaction, etc. Doubling the chance to meet or exceed company goals? What company doesn’t want that card in their pocket when competing in the marketplace?
What we found most appealing as long time veterans of the international DevOps movement was that the way to become a highest performer is to treat employees with respect, foster psychological safety, make them empowered, etc. Things typified as Westrum Generative culture. Both the company and the employees win? Incredible.
We’ve seen what some people are doing in the due diligence space, with measurements, but we feel that none of those assessments are forward looking. We want to tell a company how they are doing but also help them understand the conditions they would have to put in place to get better and why. Telling people to jump through hoops to check a box rarely alters a culture. We want to improve people’s lives in the process and as result set the company being evaluated up for success.
We also want it to be fast. Spending days logging into systems and evaluating one continuous integration system versus another goes directly against the grain of what we’ve learned as deep participants in the DevOps movement: architecture matters more than technology. We designed the assessment keeping architectural principles in mind and were able to accomplish our goal: companies going through our process are fully evaluated within two weeks from paperwork signed to completion.
The result is a custom report with impressions, observations, and answers to their questions from experts who have designed and built service delivery capabilities at ISPs, ecommerce startups, hedge funds, enterprise software companies, high transaction monitoring systems, and more. Companies also find out how they compare to others, and get a custom improvement playbook that tells them how to get better.
The report also contains more detailed breakdowns of the individual areas as outlined by Accelerate (Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Time to Recover). Each of these areas also has their own specific improvement playbook that identifies capabilities that must become strengths for companies to be high performers and double their chances to “meet or exceed their organization’s performance goals.”
The entire assessment is accomplished using three to six hours of remote interviews. We help companies determine which service delivery pipelines to evaluate, which improvements will give them the best ROI, with focuses on software delivery, infrastructure delivery, and incident response for those pipelines. There is an additional separate meeting with engineering leadership for non-technical questions about the conditions needed (e.g. Agile, on-call schedules, etc.) to become a high performer. Finally, there is the completed report review.
There are many assessments that will check a box if you have alerting, or security tests. Instead of this limited approach, for this assessment we judge how well you are doing those things based on decades of experience in high performing software delivery organizations. There is no binary check box. The thing that sets this assessment apart is the ability to tie those capabilities to business outcomes. For a financial services company, we were able to show them how the capabilities they were missing were essential for them to satisfy their investor’s thesis and transition into a SaaS company. For a logistics company, we were able to show them their problems were not in engineering, but perhaps in product or design. It is this custom ability to help both leadership and engineering understand their system in context that sets this assessment apart.
The Assessment is a valuable tool for engineering leaders who are new to an organization and want to quickly understand their service delivery capabilities. It’s also valuable to investors both pre and post deal to understand the changes that would be required to turn that five year investment thesis into a 3 year investment thesis and set their investment up to crush the competition.
We see clearly how the product will become more valuable to our customers over time. We have plans for making the information even more actionable. The more organizations we learn about, the better we are able to accomplish that goal.
There are certainly other offerings in the market to determine if a company is checking some boxes - surveys that are extremely vague about what is being reported, or internal investigations by those who don’t have as much experience in the space. We believe that the better option is to understand what a company does well, and what conditions must exist to give that company the best chance of success. Hitting some metric is worthless if everyone hates their jobs. Instead, teams could be looking forward to the future success of their company by doubling their chances to reach their goals in a way where everybody wins, using DevOps.