DevOps Blog

State of DevOps Report 2017

Stephen Watts
by Stephen Watts

Puppet has once again released their annual State of DevOps report. This is the 6th survey they’ve conducted and released, with more than 27,000 total responses. This blog post will deep-dive into findings from the 2017 State of DevOps Report and look at how it compares with previous findings. Let’s start by taking a quick look back at the five main findings of the 2016 report:

  1. Faster means better.

    Opposite to common sense, which tells us that moving fast means raising the error probability, the study has shown that organizations which were able to evolve and deliver/deploy software faster have been the ones producing more stable and resilient systems.

    This was a data proven finding; statistically validated over several types of organizations from start-ups to big established companies operating in several market verticals (some being heavily regulated like Healthcare or Finance).
  2. Faster means fewer surprises.

    Another statistical finding, out of the performed data analysis, clearly shows that high-performers spend less time doing rework and having to cope with unplanned work. High performers are able to leverage some 20 percent of their time in new work than the rest.
  3. IT performance is key.

    Over decades, the general assumption has been that IT is merely a tool and while being so, its specific performance is not a relevant contributor to the overall corporate core business performance. Well, the news is the study proves the assumption to be wrong.

    Again, statistical analysis out of a wide market sample (all verticals, shapes, and sizes) shown that, despite accounting for just about 15 percent of the market, companies with high-performing IT were twice as likely to exceed their productivity, profitability and market share goals.
  4. Psychological safety supports great teams.

    Companies that could create a team spirit where members were comfortable with taking risks in front of colleagues have proven to be the best performing. So, fostering an organizational culture where, upon something going wrong the attitude is to jointly find out why and correct it, instead of scapegoating the person who caused the problem leads to a competitive edge.
  5. Some surprises.

    Contrary to belief, the 2016 study has surprisingly shown that the following practices do not contribute improving IT performance:

    • Having a separate testing group that primarily creates and maintains acceptance tests.
    • Having your acceptance tests developed and maintained by a 3rd party (outsourced).
    • Having developers creating their own specific test environments.

    This leads to the potential confirmation of one of DevOps main core cultural approaches which is to have developers and testers working together from the early development stages onwards.

Now that we’ve reviewed the 2016 survey results, let’s consider the 2017 DevOps Report findings.

In the author’s own words the 2017 report’s focus was to demonstrate “how effective leaders influence the technical practices and process improvement changes that lead to higher IT and organizational performance.”

This year’s key findings:

  • Automation rocks.

    The analysis confirms that organization competitiveness is directly linked to their DevOps Automation degree.

    Besides looking at a dedicated time towards rework and unplanned work (like in the 2016 report), this year the analysis additionally tackled the amount of manual work that is still involved in Configuration Management, Change Management, Testing and Deployment activities. The findings are crystal clear.

  • Components impact the machine.

    It is confirmed yet again that the DevOps team structure as well as the way in which the produced code is structured directly impact the ability of the company to produce and deliver code.
  • Faster still means better.

    The report clearly states that IT performance is measured along two main dimensions: throughput of code and stability of systems.

    It was again confirmed that high-performers are characterized by faster delivery and deployment while the produced code is more stable.

    Nevertheless, the gap between high and low performers has narrowed, mainly due to the fact that low-performers had their DevOps maturity status improved allowing their delivery and deployment frequency to raise, while still not being able to improve recovery times or error rate.

    High performing team have embed the notion that by building upon quality they do not need to trade speed for stability, so they are able to deploy coherent code in an improving manner.

    The performance gap is colossal with high performers being able to deploy multiple times per day and implementing changes in less than one hour and medium performers taking between one week and a full month to both deploy new code as well as implement changes.
  • Leadership – five common characteristics.

    Leaders attitude is key for the team ability to become a high-performing one.

    In the author’s own words: “One of the exciting research focus areas this year is investigating the leadership characteristics that help drive high performance.”

    To thrive within a DevOps ecosystem, the team must be supported by:

    • High mutual trust culture
    • A set of processes over a toolkit and based on a reliable infrastructure that fosters productivity while reducing deployment timeframe.
    • A culture that stimulates experimentation and new ways of doing things so that processes may become lean and implementation cycles continuously shorter
    • A structure that has no organizational silos, therefore, empowering strategic alignment

    All the above needs to be steered by the “Transformational Leader” (as it has been labeled), who must possess the following five main characteristics that contribute to such progressive momentum:

    • Vision – Knowing the way and how to get there within a three- to five-year horizon.
    • Inspirational communication – A positive forward speech and attitude before challenges that convey inspirational feelings while motivating the team.
    • Intellectual stimulation – Instead of defining the way or merely showing it, the leader will challenge the team to think out of the box coming up with innovative ways of tackling issues.
    • Supportive leadership – Teams are also made of human beings and therefore the leader shows care and attention for the person’s needs, feelings, and aspirations.
    • Personal recognition – Humans require acknowledgment to remain enrolled and motivated to evolve and become better.
  • Leadership is not everything.

    Now, this may sound like a contradiction when considering what you just read in the previous paragraph; but we all know that the leader does not work alone, s/he needs a team and any team is only as strong as its weakest link.

    The study clearly points out that having analyzed the 10 teams which gave better ratings to their leader’s adherence to the “Transformational Leader” above-mentioned set of values, it was found that those were not the top 10 performing teams.

    So as per the report findings, “Transformational Leadership” is not enough by itself to drive high IT performance.
  • DevOps – one fits all.

    It has been clearly assessed that DevOps applies to all types of organizations.

    And since top performers exist in several market verticals it was possible to assess that all of them were likely to exceed the objectives set for the following areas:

    • Quantity of products or services
    • Operating efficiency
    • Customer satisfaction
    • Quality of products or services
    • Achieving organizational and mission goals
  • Segmentation drives speed.

    The best performing ranking companies were ordered according to the degree of achieved granularity, meaning performance raises in the direct proportion teams can be subdivided (while yet working together) to address specific services that can be developed and released independently.
  • Technical practices.

    The DevOps culture has evolved and comprehends a set of Operational Best Practices that leverage IT performance, as in means of acting and doing things (which then will have processes designed to enable them). Amongst these, one may find:

    • Continuous delivery wise – the ability to deploy on-demand combined with the prioritization of deployments based on client feedback constitute the main practices.
    • Architecture wise – the ability to perform tests without having integrated environments combined with the independence from other IT systems that mandate the target development.

    Some KPIs that show the maturity degree of a DevOps team towards continuous delivery are:

    • Percentage of successful large-scale changes to system design without permission from external stakeholders.
    • Percentage of successful large-scale changes to system design with no pre-requisite of other teams making changes to other systems, or implicating significant work for those teams.
    • Work completion percentage without having to specifically communicate and coordinate with 3rd parties.
    • Percentage of successful on-demand deployments independent from the other services the product depends on.
    • Percentage of on-demand testing done out of an integrated test environment.
    • Percentage of performed deployments during normal business hours with negligible downtime.
  • Lean contributes to customer satisfaction.

    Lean product management processes allow shorter response time towards features that customers want to have in place, therefore highly contributing to happy customers.
  • Growing community.

    Over the last three years, the number of experts that have moved into a DevOps team/ context (either in the same company or other) has significantly raised, clearly showing that the DevOps culture is expanding in the marketplace.

But what better way to understand the current context than with some graphical representation?

Understanding the report

The DevOps report is now in its sixth year and therefore all the involved work process and methodology have been evolving and are being fine-tuned.

The way in which teams and companies are classified according to the ratings of High, Medium or Low Performers is based not on pre-conceived definitions but rather on specific data analysis.

Contributors have been chosen amongst those who lead or work in IT and are familiar with DevOps via a sampling action though e-mailing and social media.

The report is based on a survey where the questions have been written after the definition of which were the hypothesis in need of being tested (given previous years findings and known facts and trends).

The way in which the statistical analysis was performed resorts to some standard and adapt methodologies like cluster analysis; measurement modeling; regression analysis; structured equation modeling and cross-sectional plus theory-based design.

Forrester Study: Accelerate software delivery

When developers build business automation workflows at the same time as other application components, software delivery speed and efficiency increase while errors drop. Read this study to learn what Jobs-as-Code can mean for your business.
Download Now ›

These postings are my own and do not necessarily represent BMC's position, strategies, or opinion.

About the author

Stephen Watts

Stephen Watts

Stephen is based in Birmingham, AL and began working at BMC Software in 2012. Stephen holds a degree in Philosophy from Auburn University and is currently enrolled in the MS in Information Systems - Enterprise Technology Management program at University of Colorado Denver.

Stephen contributes to a variety of publications including, Search Engine Journal, ITSM.Tools, IT Chronicles, DZone, and CompTIA.