Musing on whether the DevOps ideas you've been reading about could be the way to a happier, more productive working life — but not sure how to make the case, or how you'll demonstrate results? People using DevOps practices rely on several key performance indicators (KPIs) to judge the success of their DevOps efforts.
Of the 4,000-odd people in 90-plus countries who responded to our 2013 State of DevOps survey, more than 63 percent "doing DevOps." These respondents told us their most important measures of improvement are:
- Deployment frequency
- Speed of deployment
- Deployment success rate
- How quickly service can be restored after a failed deployment
- Culture, which actually can't be measured. See below.
Combining survey results with what some DevOps companies have said publicly about their experiences can help you decide which KPIs to measure if you're just starting to implement DevOps in your workplace—or thinking about starting.
1. Deployment frequency
Boosting deployment frequency has been a powerful motivator for shifts in development practices. The ability to make code changes quickly and easily is a key competitive advantage for any company that needs to deliver new features quickly to customers, and respond to their behavior.
People responding to our survey told us they were able to deploy much faster after implementing DevOps in their organizations. Teams that had been following DevOps practices the longest were shipping code up to 30 times faster — and completing deployments up to 8,000 times faster — than their lower-performing peers.
These numbers may sound exaggerated, until you hear real stories of companies that used to deploy once or twice a year, and now deploy multiple times per day. Some companies deploy changes as often as every few minutes.
One great example is AOL, which implemented a simple change that took deploy time from 6 hours to 45 minutes. (Disclosure: Because our survey was anonymous, we don't know the actual companies where respondents work. All anecdotes come from other reporting we've done.)
2. Deployment speed
Frequent code deployment depends in large part on being able to move quickly from committed code to that code running successfully in the production environment.
More than 25 percent of our respondents reported that their teams had were able to accelerate time to deployment to less than a day. Just under half of these said it took less than one hour. That puts these teams in good company: PayPal has said publicly that it has improved lead time to the point that new code makes it from a developer's desktop to production in an hour or less.
3. Failure rate
It's great to deploy more frequently and quickly, but if changes fail just as frequently, you've gained nothing. Failed deployments can take services down, resulting in lost revenue and frustrated customers.
DevOps practices can make a big difference in failure rate. The survey respondents whose organizations are performing well reported 50 percent fewer failures from code changes.
Some high-performing technology teams have taken service reliability to dramatic heights. Amazon Web Services says just 0.001 percent of its deployments cause outages. During President Barack Obama's most recent campaign, his technology team processed more than 180 terabytes of data over 18 months, and experienced just 30 minutes of downtime.
4. Time to recovery
When service does go out, the ability to recover quickly can make a huge difference to business results. It's not surprising, then, that large web companies like Google, Etsy, Netflix and Amazon push the envelope in their efforts to improve time to recovery, regularly breaking their applications and infrastructure to discover – and provision against – anything that can go wrong.
The best-performing organizations represented in our survey were able to restore service 12 times faster than their peers. Most of our respondents – almost 75 percent – reported being able to restore service in less than an hour. A much smaller group – about 28 percent – can restore services within a few minutes.
The fifth KPI: Happier lives for sysadmins and developers
The business benefits of DevOps practices are clear. Companies that can deploy changes quickly and reliably are able to introduce new features and improvements in response to the market, and ahead of competitors. Their revenue streams are more dependable, and they can plan better for the future.
All that is great — and quantifiable — but the human benefits of DevOps practices matter just as much to the people who adopt them. More than half our survey respondents said that cultural change within their organization — more collaboration and cooperation between developers and IT operations — was one of the top benefits of changing over to DevOps practices.
The stories we hear from customers and community members confirm that human benefit: the IT ops people who no longer get the 3:00 AM call to fix a broken deployment, and the developers who now see the IT team as allies and friends, instead of obstacles.
No comments:
Post a Comment