The service desk is a valuable ITSM function that ensures efficient and effective IT service delivery. A variety of metrics are available to help you better manage and achieve these goals. These metrics often identify business constraints and quantify the impact of IT incidents. Of course, the vast, complex nature of IT infrastructure and assets generate a deluge of information that describe system performance and issues at every network node. The challenge for service desk? Identifying the metrics that best describe the true system performance and guide toward optimal issue resolution.

We’ve talked before about service desk metrics, such as the cost per ticket. Another service desk metric is mean time to resolve, which quantifies the time needed for a system to regain normal operation performance after a failure occurrence. In this article, we’ll explore mean time to resolve (sometimes abbreviated MTTR), including defining and calculating mean time to resolve and showing how it supports a DevOps environment.

Check out a product that can help reduce mean time to respond by leveraging generative AI and observability >

What is mean time to resolve?

Beyond the service desk, it’s is a popular and easy-to-understand metric:

• DevOps professionals discuss mean time to resolve to understand potential impact of delivering a risky build iteration in production environment.
• Business executives and financial stakeholders question downtime in context of financial losses incurred due to an IT incident.
• Customers of online retail stores complain about unresponsive or poorly available websites.

In each case, the popular discussion topic is the time spent between failure and issue resolution. So, let’s define mean time to resolve.

‘Mean time to recovery’ is the average time duration to fix a failed component and return to an operational state. This metric includes the time spent during the alert and diagnostic processes, before repair activities are initiated. (The average time solely spent on the repair process is called ‘mean time to repair’, also shortened to MTTR.) It can be mathematically defined in terms of maintenance or the downtime duration:

In other words, MTTR describes both the reliability and availability of a system:

• Reliability refers to the probability that a service will remain operational over its lifecycle. It can be described as an exponentially decaying function with the maximum value in the beginning and gradually reducing toward the end of its life.
• Availability refers to the probability that the system will be operational at any specific instantaneous point in time.

The shorter the mean time to resolve, the higher the reliability and availability of the system. From a practical service desk perspective, this concept makes MTTR the metric valuable: users of IT services expect services to perform optimally for significant durations as well as at specific instances. For example, Amazon Prime customers expect the website to remain fast and responsive for the entire duration of their purchase cycle, especially during the holiday season. If the website is down several times per day but only for a millisecond, a regular user may not experience the impact.

Find out which predictive intelligence tool companies are using to continuously optimize their IT environments >

Mean time to resolve encourages DevOps

It’s a valuable metric for service desks on its own, but it also encourages DevOps culture and practices in a variety of ways:

• Low impact of incidents. The primary objective of mean time to resolve is to reduce the impact of IT incidents on end users. If an issue is resolved before a customer’s online activity is disrupted, the service will be accepted as efficient and effectively delivered.
• Resilient system design. The service desk goals associated with this metric are achieved by developing a resilient system or code. For example, a website feature could be developed as a separate code with web service called independently from other features. Any repairs or changes to a specific feature may not impact the performance of other website features. This would make the entire website resilient, with each feature easy to repair.
• Feedback loop. Any improvement to the software build requires a fast feedback mechanism that informs developers early during the SDLC pipeline. MTTR can be reduced when the bugs are small in scope, easy to fix, and identified during the early development stages.
• Reduced dependencies. Mean time to resolve increases when fixing a single issue requires the fix of multiple functions and systems tightly integrated and dependent to each other. By reducing such dependencies, the performance of this metric is improved. From a service desk perspective, the services are carefully evaluated to ensure low dependencies.
• Active monitoring. The resolution process can only begin after a fault is identified. Actively monitor the infrastructure logs to identify patterns of anomalous behavior and the underlying problem root cause. With this information, the service desk can perform appropriate problem management or incident response actions, thereby reducing downtime.
• Rapid iterations. Any fix applied to a system can have negative consequences. The focus on reducing meant time to resolve encourages small and fast fixes that can be easily deployed—and rolled back—in response to a negative outcome.
• Designing for failure. Reducing mean time to resolve encourages the service desk to design for, prepare for, and embrace failure. Downtime and service outages are inevitable, but the success of the service desk depends on how well it can respond and mitigate the impact of fast-fail product development and service delivery initiatives.
• Velocity, quality and performance. The DevOps goals of velocity, quality, and performance are achieved when build iterations are released rapidly at high quality, reducing waste processes. Incidentally, this is also the purpose of reducing issue resolution time for service desk activities and processes including incident and problem management.
• Continuous improvement. Repetitive issues downgrade overall system performance and any possibility of resolving issues in a timely manner. Finding the problem’s root cause and reducing repetitive issue resolution requests is a sign of continuous improvement.
• Automation and intelligence. To resolve issues quickly, service desk must gain end-to-end visibility and control into the IT network and assets. Advanced AI capabilities to eliminate alert noise and proactively identify problem root cause help reduce the issue resolution time, among other ITSM objectives.

By following the DevOps philosophy, service desk can achieve the wider ITSM objectives of efficiently and effectively delivering IT services. Mean time to resolve is one among many other service desk metrics that companies can use to evaluate for deeper insights into IT service management and operations activities. With any technology or metrics, however, remember that there is no ‘one size fits all’: you’ll want to determine which metrics are useful for your organization’s unique needs, and build your ITSM practice to achieve real-world business goals.

What’s next?

Dive into more about a closely related concept: Mean time to repair (MTTR) >

Most service providers understand the need for service level agreements (SLAs) with their partners and customers. But creating one might feel daunting because you don’t know where to start or what to include. In this article, we share some SLA examples and templates to help you create SLAs.

What is an SLA?

An SLA is a documented agreement between a service provider and a customer that defines: (i) the level of service a customer should expect, while laying out the metrics by which service is measured, as well as (ii) remedies or penalties should agreed-upon service levels not be achieved. It is a critical component of any technology vendor contract.

Before subscribing to an IT service, the SLA should be carefully evaluated and designed to realize maximum service value from an end-user and business perspective. Service providers should pay attention to the differences between internal outputs and customer-facing outcomes, as these can help define the service expectations.

Take IT Service Management to the next level with BMC Helix ITSM.›

Writing SLAs: An SLA template

Let’s examine a sample SLA that you can use as a template for creating your own SLAs. Remember that these documents are flexible and unique. Make changes as necessary, and ensure that you correctly identify and include the relevant parties. Also, consider additional topics that you may want to add to your agreement(s) to enhance them, such as:

• Review or monitoring period. How often the service provider and customer may review the terms of the SLA; perhaps, annually.
• Service credits. Something the service provider may offer in case your SLA is not achieved.
• A rider. Used when amendments to an SLA occur.
• End-of-contract or liquidation terms. This defines how and when customer or service provider can opt out of the SLA.

There are several ways to write an SLA. Below is a mock table of contents that you can leverage to start writing your own SLAs.

Now, I’ll break down each section with a few details and examples.

1.0  SLA

The first page of your document is simple, yet important. It should include:

• Version details
• Document change history, including last reviewed date and next scheduled review
• Document approvals

Last Review: MM/DD/YYYY

Next Scheduled Review: MM/DD/YYYY

2.0. Agreement overview

In the next section, the agreement overview should include four components:

1. The SLA introduction
2. Definitions, convention, acronyms, and abbreviations (a glossary)
3. Purpose
4. Contractual parameters

2.1. SLA introduction

Include a brief introduction of the agreement, relevant parties, service scope, and contract duration. For instance:

This is a Service Level Agreement (SLA) between [Customer] and [Service Provider]. This document identifies the services required and the expected level of services between MM/DD/YYYY to MM/DD/YYYY.

Subject to review and renewal scheduled by MM/DD/YYYY.

Signatories:

2.2. Definitions, conventions, acronyms, and abbreviations

Include a definition and brief description of terms used to represent services, roles, metrics, scope, parameters, and other contractual details that may be interpreted subjectively in different contexts. This information may also be distributed across appropriate sections of this document instead of collated into a single section.

2.3. Purpose

This section defines the goals of this agreement, such as:

The purpose of this SLA is to specify the requirements of the software-as-a-service (SaaS) solution as defined herein with regards to:

• Requirements for SaaS service that will be provisioned to [Customer]
• Agreed service targets
• Criteria for target fulfilment evaluation
• Roles and responsibilities of [Service Provider]
• Duration, scope, and renewal of this SLA contract
• Supporting processes, limitations, exclusions, and deviations.

2.4. Contractual parameters

In this section, you’ll want to define the policies and scope of this contract related to application, renewal, modification, exclusion, limitations, and termination of the agreement.

This section specifies the contractual parameters of this agreement:

1. Contract renewal must be requested by [Customer] at least 30 days prior to expiration date of this agreement.
2. Modifications, amendments, extension, and early termination of this SLA must be agreed by both signatory parties.
3. [Customer] requires a minimum of 60 days’ notice for early termination of this SLA.
4. …

3.0. Service agreement

This section can include a variety of components and subsections, including:

1. KPIs and metrics
2. Service levels, rankings, and priority
3. Service response
4. Exceptions and limitations
5. Responses and responsibilities
6. Service management

3.1. KPIs and metrics

Key performance indicators (KPIs) and other related metrics can and should support your SLA, but the achievement of these alone does not necessarily result in the desired outcome for the customer.

3.2. Service levels, rankings, and priority

3.3. Service response

3.4. Exceptions and limitations

Include any exceptions to the SLA conditions, scope, and application, such as:

This SLA is subject to the following exceptions and special conditions:

• [Service Provider] must ensure cloud service availability of 99.9999% during holiday season dated MM/DD/YYYY to MM/DD/YYYY.
• [Service Provider] may not be liable to credit reimbursement for service impact to data centers in Region A and Region B due to natural disasters.
• Response to requests of severity level 6 or below by [Customer] can be delayed up to 24 hours during the aforementioned holiday season.
• Requests for special arrangements by [Customer] may be expedited as per pricing structure specified in Appendix A.1.

3.5. Responses and responsibilities

Here, you’ll define the responsibilities of both the service provider and the customer.

[Customer] responsibilities:

• [Customer] should provide all necessary information and assistance related to service performance that allows the [Service Provider] to meet the performance standards as outlined in this document.
• [Customer] shall inform [Service Provider] regarding changing business requirements that may necessitate a review, modification, or amendment of the SLA.
• …

[Service Provider] responsibilities

• [Service Provider] will act as primary support provider of the services herein identified, except when third-party vendors are employed, who shall assume appropriate service support responsibilities accordingly.
• [Service Provider] will inform [Customer] regarding scheduled and unscheduled service outages due to maintenance, troubleshooting, or disruptions, or as otherwise necessary.
• …

3.6. Service management

Include service management and support details applicable to the service provider in this section.

3.6.1. Service availability

Service coverage by the [Service Provider] as outlined in this agreement follows the schedule specified below:

• On-site support: 9 AM to 6 PM, Monday to Friday, from January 5, 2023 to December 20, 2023.
• Phone support: 24 hours as per Section 3.2. of this agreement.
• Email support: 24 hours as per Section 3.2. of this agreement.
• …

Planning a cloud migration strategy? Start with the BMC Helix Platform. ›

References and glossary

Include reference agreements, policy documents, glossary, and relevant details in this section. This might include terms and conditions for both the service provider and the customer, and any additional reference material, such as third-party vendor contracts.

Appendix

The appendix is a good place to include relevant information that doesn’t seem to fit elsewhere, such as pricing models and charges. The following section is an example of information that you may want to append to your SLA.

A.1. Pricing models and charges

Include the pricing models for each service type with detailed specifications.

SLA best practices

Though your SLA is intended to be a legally binding agreement, it doesn’t need to be incredibly lengthy or overly complicated. It can further be a malleable document that is improved upon over time, with the consent of all relevant parties. Our advice: Begin building an SLA using the template above and the examples found herein and consult with your customers for any perceived gaps. As unforeseen circumstances are often inevitable, you can always revisit and tweak the SLA, if needed.

Additional resources

Additional SLA templates and examples are available here:

• Columbia University IT (PDF)
• Giva (PDF)
• PandaDoc
• SLA Template

While a new era of artificial intelligence (AI) may currently dominate tech headlines, cloud computing remains a hot and pervasive topic to this day. As you consider evolving your business more to the cloud, whether for application or infrastructure deployment, it is more important than ever to understand the differences and advantages of the various cloud services.

Although as-a-service types continue to grow, there are usually three core models of cloud service to consider and compare:

• Software as a service (SaaS)
• Platform as a service (PaaS)
• Infrastructure as a service (IaaS)

For each of these, we’ll look at the concept, benefits, and variances. We’ll also help you understand the key differences among SaaS, PaaS, and IaaS, so you can choose an approach that’s right for your organization.

(More interested in cloud setup? Learn more about public, private, and hybrid cloud differences.)

Key differences

Common examples of SaaS, PaaS, and IaaS

SaaS: Software as a service

Software as a service (SaaS), also known as cloud application services, represents the most commonly utilized option for businesses in the cloud market. SaaS leverages the internet to deliver applications, which are managed by a third-party vendor, to its users. A majority of SaaS applications run directly through your web browser, which means they do not require any downloads or installations on the client side.

SaaS delivery

Due to its web-delivery model, SaaS eliminates the need to have IT staff download and install applications on each individual computer. With SaaS, vendors manage all potential technical issues, such as data, middleware, servers, and storage, resulting in streamlined maintenance and support for the business customer.

SaaS advantages

SaaS provides numerous advantages to employees and companies by greatly reducing the time and money spent on tedious tasks, such as installing, managing, and upgrading software. This frees up time for technical staff to spend on bustiness-critical issues within the organization.

SaaS characteristics

There are a few ways to help you determine when SaaS is being utilized:

• Managed from a central location
• Hosted on a remote server
• Accessible over the internet
• Users not responsible for hardware or software updates

When to use SaaS

SaaS may be the most beneficial option in several situations, including:

• Startups or small companies that need to launch e-commerce quickly and don’t have time for server issues or complex on-premises software
• Short-term projects that require quick, easy, and affordable collaboration
• Applications that aren’t needed too often, such as tax software
• Applications that need both web and mobile access

SaaS limitations and concerns

• Interoperability. Integration with existing apps and services can be a major concern if the SaaS application is not designed to follow open standards for integration. In this case, organizations may need to design their own integration systems or reduce dependencies with SaaS services, which may not always be possible.
• Vendor lock-in. Vendors may make it easy to join a service and difficult to get out of it. For instance, the data may not be portable—technically or cost-effectively—across SaaS applications from other vendors without incurring significant cost or in-house engineering rework. Not every vendor follows standard APIs, protocols, and tools, yet the features could be necessary for certain business tasks.
• Lack of integration support. Many organizations require deep integrations with on-premises applications, data, and services. The SaaS vendor may offer limited support in this regard, forcing organizations to invest internal resources in designing and managing integrations. The complexity of integrations can further limit how the SaaS app or other dependent services can be used.
• Data security. Large volumes of data may have to be exchanged with the backend data centers of SaaS applications in order to perform the necessary software functionality. Transferring sensitive business information to public, cloud-based SaaS services may result in compromised security and compliance, in addition to incurring significant cost for migrating large data workloads.
• Customization. SaaS applications offer minimal customization capabilities. Since a one-size-fits-all solution does not exist, users may be limited to specific functionality, performance, and integrations as offered by the vendor. In contrast, on-premises solutions that come with several software development kits (SDKs) offer a high degree of customization options.
• Lack of control. SaaS solutions involve handing control over to the third-party service provider. These controls are not only limited to the software—in terms of the version, updates, or appearance—but, also to the data and governance. Customers may, therefore, need to redefine their data security and governance models to fit the features and functionality of the SaaS offering.
• Feature limitations. Since SaaS applications often come in a standardized form, the choice of features may be a compromising tradeoff against security, cost, performance, or other organizational policies. Furthermore, vendor lock-in, cost, or security concerns may mean it’s not viable to switch vendors or services to serve new feature requirements in the future.
• Performance and downtime. Because the vendor controls and manages the SaaS service, your customers now depend on vendors to maintain the service’s security and performance. Planned and unplanned maintenance, cyberattacks, or network issues may impact the performance of the SaaS application despite adequate service level agreement (SLA) protections in place.

Examples of SaaS

Popular examples of SaaS include:

• Google Workspace (formerly GSuite)
• Dropbox
• Salesforce
• Cisco WebEx
• SAP Concur
• GoToMeeting

Planning to migrate enterprise IT functions to the cloud? Check out the BMC Helix Platform. › 

PaaS: Platform as a service

Cloud platform services, also known as platform as a service (PaaS), provide cloud components to certain software while being used mainly for applications. PaaS delivers a framework that developers can build upon and use to create customized applications. All servers, storage, and networking can be managed by the enterprise or a third-party provider while the developers can maintain management of the applications.

PaaS delivery

The delivery model of PaaS is similar to SaaS, except instead of delivering the software over the internet, PaaS provides a platform for software creation. This platform is delivered via the web, giving developers the freedom to concentrate on building the software without having to worry about operating systems, software updates, storage, or infrastructure.

PaaS allows businesses to design and create applications and integrate special software components into the PaaS. These applications, sometimes called middleware, are scalable and highly available as they take on certain cloud characteristics.

PaaS advantages

No matter the size of your company, using PaaS offers numerous advantages, including:

• Simple, cost-effective development and deployment of applications
• Scalability
• Highly available
• Application customization without software maintenance
• Significant reduction in the amount of coding needed
• Automation of business policy
• Easy migration to the hybrid model

PaaS characteristics

PaaS has many characteristics that define it as a cloud service, including:

• Builds on virtualization technology, so resources can easily be scaled up or down as your business changes
• Provides a variety of services to assist with the development, testing, and deployment of apps
• Accessibility to many users via the same development application
• Integration with web services and databases

When to use PaaS

Using PaaS is beneficial, sometimes even necessary, in several situations. For example, PaaS can streamline workflows when multiple developers are working on the same development project. If other vendors must be included, PaaS can provide great speed and flexibility to the entire process. PaaS is particularly beneficial if you need to create customized applications. This cloud service also can greatly reduce costs and simplify some of the challenges that arise if you are rapidly developing or deploying an application.

PaaS limitations and concerns

• Data security. Organizations can run their own apps and services using PaaS solutions, but the data residing in third-party, vendor-controlled cloud servers poses security risks and concerns. Your security options could be limited as customers may be unable to deploy services with specific hosting policies.
• Integrations. The complexity of connecting the data stored within an onsite data center or off-premises cloud is increased, which may affect which applications and services can be adopted with the PaaS offering. Integration with existing services and infrastructure may be a challenge for legacy IT systems with components that were not built for the cloud.
• Vendor lock-in. Business and technical requirements that drive decisions for a specific PaaS solution may not apply in the future. If the vendor has not provisioned convenient migration policies, switching to alternative PaaS options may not be possible without affecting the business.
• Customization of legacy systems. PaaS may not be a plug-and-play solution for existing legacy applications and services. Instead, several customizations and configuration changes may be necessary for legacy systems to work with the PaaS service. The resulting customization can result in a complex IT system that may limit the value of the PaaS investment altogether.
• Runtime issues. In addition to limitations associated with specific applications and services, PaaS solutions may not be optimized for the language and frameworks of your choice. Specific framework versions may not be available or perform optimally with the PaaS service, and customers may not be able to develop custom dependencies with the platform.
• Operational limitation. Customized cloud operations with automated management workflows may not apply to PaaS solutions, as the platform tends to limit operational capabilities for end users. Although this is intended to reduce the operational burden on end users, the loss of operational control may affect how PaaS solutions are managed, provisioned, and operated.

Examples of PaaS

Popular examples of PaaS include:

• AWS Elastic Beanstalk
• Windows Azure
• Heroku
• Force.com
• Google App Engine
• OpenShift

Take the leap to the next level of IT Service Management with BMC Helix ITSM. › 

IaaS: Infrastructure as a service

Cloud infrastructure services, known as infrastructure as a service (IaaS), are made of highly scalable and automated compute resources. IaaS is fully self-service for accessing and monitoring computers, networking, storage, and other services. IaaS allows businesses to purchase resources on demand and as needed instead of having to buy hardware outright.

IaaS delivery

IaaS delivers cloud computing infrastructure, including servers, networks, operating systems, and storage, through virtualization technology. These cloud servers are typically provided to the organization through a dashboard or API, giving IaaS clients complete control over the entire infrastructure. IaaS provides the same technologies and capabilities as a traditional data center without having to physically maintain or manage all of it. IaaS clients can still access their servers and storage directly, but it is all outsourced through a “virtual data center” in the cloud.

As opposed to SaaS or PaaS, IaaS clients are responsible for managing aspects such as applications, runtime, operating systems, middleware, and data. However, IaaS providers manage the servers, hard drives, networking, virtualization, and storage. Some providers also offer more services beyond the virtualization layer, such as databases or message queuing.

IaaS advantages

IaaS offers many advantages, including:

• The most flexible cloud computing model
• Easy automated deployment of storage, networking, servers, and processing power
• Consumption-based hardware purchasing
• Complete client control of their infrastructure
• Resource purchasing as needed
• High scalability

IaaS characteristics

Characteristics that define IaaS include:

• Resources available as a service
• Variable, consumption-based costs
• Highly scalable services
• Multi-user hardware access
• Organizational control of the infrastructure
• Dynamic flexibility

When to use IaaS

Just as with SaaS and PaaS, there are specific situations when IaaS is most advantageous.

• Startups and small companies may prefer IaaS to avoid spending time and money on purchasing and creating hardware and software.
• Larger companies may prefer to retain complete control over their applications and infrastructure, but they want to purchase only what they actually consume or need.
• Companies experiencing rapid growth like the scalability of IaaS, and they can change out specific hardware and software easily as their needs evolve.

Anytime you are unsure of a new application’s demands, IaaS offers plenty of flexibility and scalability.

IaaS limitations and concerns

Limitations associated with SaaS and PaaS models—such as data security, cost overruns, vendor lock-in and customization issues—also apply to the IaaS model. Particular limitations of IaaS include:

• Security. While the customer is in control of the applications, data, middleware, and the operating system platform, security threats can still be sourced from the host or other virtual machines (VMs). Insider threat or system vulnerabilities may expose data communication between the host infrastructure and VMs to unauthorized entities.
• Legacy systems operating in the cloud. While customers can run legacy applications in the cloud, the infrastructure may not be designed to deliver specific controls to secure them. Minor enhancements to legacy applications may be required before migrating them to the cloud, possibly leading to new security issues unless adequately tested for security and performance in the IaaS systems.
• Internal resources and training. Additional resources and training may be required for the workforce to learn how to effectively manage the infrastructure. Customers will be responsible for data security, backup, and business continuity. Due to inadequate control of the infrastructure, however, monitoring and management of the resources may be difficult without availability of adequate in-house training and resources.
• Multi-tenant security. Since the hardware resources are dynamically allocated across users as needed, the vendor is required to ensure that other customers cannot access data left on storage assets by previous customers. Similarly, customers must rely on the vendor to ensure that VMs are adequately isolated within the multi-tenant cloud architecture.

Examples of IaaS

Popular examples of IaaS include:

• DigitalOcean
• Linode
• Rackspace
• AWS
• Cisco Metacloud
• Microsoft Azure
• GCE

SaaS vs. PaaS vs. IaaS

Each cloud model offers specific features and functionalities, and it is crucial for your organization to understand the differences. Whether you need cloud-based software for storage options, a smooth platform that allows you to create customized applications, or complete control over your entire infrastructure without having to physically maintain it, there is a cloud service for you.

No matter which option you choose, migrating to the cloud is the future of business and technology.

XaaS: Everything as a service

One term you’re likely seeing more frequently in the world is XaaS, short for everything as a service. XaaS refers to the highly-individualized, responsive, data-driven products and offerings that are fully controlled by customers—and the data they provide via everyday Internet of Things (IoT)-powered sources like cell phones and thermostats.

By using that data generated over the cloud, businesses can innovate faster, deepen their customer relationships, and sustain the sale beyond the initial product purchase. XaaS is a critical enabler of the Autonomous Digital Enterprise.

Related reading

• BMC Blogs: Cloud
• Cloud Infrastructure: A Brief Introduction
• Hybrid Cloud Security: Challenges and Best Practices
• Data Center Tiers: What Are They and Why Are They Important?

Other “as a service” offerings:

• AI as a service
• Database as a service
• Business Process Management as a service (BPMaaS)
• Functions as a service
• Testing as a service

Defending the Whole, IaaS, PaaS, and SaaS from Mark Nunnikhoven

Original reference images:
 

When examining what makes automation intelligent, it is important to first differentiate between traditional automation and data-driven intelligence. Automation refers to predefined rules and policies that are programmed to trigger in response to an event. For example, exceeding metrics thresholds would trigger specific control actions. From a ServiceOps perspective, intelligent automation is the ability to automatically kick off a fix that’s been identified through BMC Helix ITSM or BMC Helix Operations Management, or recommend manual fixes that can be executed quickly.

Intelligent automation simulates cognitive thinking for decision making and automation actions. A key differentiator for intelligent automation is that instead of using assumptions or previously known information, it relies on artificial intelligence (AI) models that learn and reason from data, effectively modeling a cognitive thinking engine that makes decisions based on trends and patterns observed in the data.

In this post, we will review some key differentiators between traditional automation and data-driven intelligent automation. First, let’s discuss the key characteristics of AI-based intelligent automation systems:

• Learning and reasoning: The automation technology is capable of learning from past data and using the contextual knowledge to make decisions. This is different from a traditional automation system that only makes decisions based on fixed knowledge pre-programmed beforehand with the given or assumed knowledge of how the system works.
• Adaptability: An AI-based intelligent automation system can account for changes and respond by adapting its reasoning capability in real time based on the availability of new information. System behavior is expected to change rapidly as it scales to a growing user base. These changes are not always predictable and can require thorough analysis, which is a time-consuming effort. AI systems can be tuned to detach from past learnings and embed new knowledge into the models based on real-time data streams. As a result, the model is up-to-date with the data available for training.
• Discovery: IT can only control what they can measure. Considering the complex IT architecture and service delivery models, IT operations and management teams struggle to find relationships and dependencies between application components and IT services. The lack of traceability and discovery of a dynamic service model makes it challenging to assign and find fix rules regarding dependencies. Intelligent automation provides an abstracted view of the relationships by modeling these complex and evolving dependencies with AI, which makes it easier for the system to autonomously trace and discover services.
• Data-driven AI models: The most prominent difference is the ability to learn from data. While traditional automation systems follow predefined rules, AI models simulate system behavior observed in large volumes of log metrics data generated across the IT network. Therefore, the learning of a data-driven AI model is both exhaustive and adaptable. Achieving the same results in a traditional automation system would require exact mapping of all system components and services, as well as true response to all event scenarios, which is virtually impossible to achieve manually considering the vast scale of IT infrastructure operations.
• Real time: AI models can also be trained to learn and respond in real time. Unlike traditional automation systems that trigger an automation action based only on known threshold values, an intelligent automation solution accounts for context, constraints, and patterns that should be evaluated exhaustively to form an intelligent decision. The result is a proactive response action with intelligent automation versus the reactive response of traditional automation tools that require reprogramming or configuration changes to account for real-time changes in system behavior.
• Future predictions: The utility of traditional automation tools is limited to predefined rules: it only triggers a control action when these thresholds are exceeded. AI models allow for intelligent behavior, observing trends and patterns within data and predicting the expected system behavior of a future state. This knowledge cannot be hard-coded but can be trained using information about past events and how they map to potential incidents and service outages.
• Complex modeling: The entire modeling process is a complex endeavor. While it may be virtually impossible to model every state and corresponding system behavior exactly, advanced machine learning (ML) algorithms provide several intelligent mechanisms to learn from data so that explicit modeling of the relationships and dependencies is not required by the AI solution. Instead, it learns to model the system behavior based on a set of inputs (new data at nodes) and outputs (system measurements). The ML model itself can grow to several hundreds of thousands of parameters depending on the model complexity, but that’s far less work than modeling the actual (unknown) state of the system itself.

Combining all these characteristics is a replication of human cognitive behavior. The capability of advanced human-like intelligence augments your existing workforce and can be scaled on demand, instead of employing and training new engineers on traditional automation tools.

Automation and data-driven intelligence are two important concepts that are often used interchangeably, but they are not the same thing. Automation is the process of using technology to perform repetitive tasks, while data-driven intelligence involves using data to make decisions and improve performance.

Incorporating data-driven intelligence can give organizations a competitive edge. If you are interested to see what BMC Helix Operations Management with AIOps can do for your organization, get 14 days free to explore the IT operations toolkit powered by AIOps.

Artificial-intelligence (AI)-enabled root cause isolation is an important component of the incident management strategy that allows organizations to proactively mitigate risk of service outages and downtime. Modern IT infrastructure environments consist of a complex and convoluted mix of hardware components, running software applications in a variety of service delivery architecture: on-premises, multicloud, software-defined, and containerized instances. The sheer volume of metrics, events, and log data containing insights on patterns of issues such as performance downtime and network infringements can easily overwhelm teams running traditional analytics and automation tools for root cause analysis.

AI has emerged as a promising application for incident management use cases. Unlike traditional automation, AI-enabled tools not only identify insights into past log metrics data but also predict future trends and then automate a proactive remediation action or provide guidance on best possible proactive risk management actions.

How it works

Root cause isolation capabilities should be an essential part of your AIOps tooling portfolio and is focused on predicting the most likely root cause situation underlying a service dependability issue. The technology that makes these predictions uses AI models—representing the IT infrastructure systems and their behavior under varying load patterns—that have trained and learned patterns of log metrics data over time. When the AI model determines a pattern of performance issues in the log metrics data, it replicates the future behavior of the infrastructure systems and predicts a likely future outcome based on recent historical events. In the case of root cause analysis, the AI models can be trained to analyze situational events from the infrastructure system and nodes, and predict the future impact on metrics such as mean time to identify (MTTI) or mean time to resolve (MTTR).

The AI-enabled root cause isolation works differently from traditional automation and analytics in the following ways:

•  The AI tool provides a list of most likely incidents as well as the most relevant root causes applicable to the events scenario.
• The AI model then determines the most likely set of nodes that map to the most probable root cause incidents.
• The model then finds a list of causes or situational events as well as automated triggers or change requests that help reduce the probability of service outages to specific root nodes.
• Additional useful information can include the health trajectory of individual nodes and services. Next, tools can leverage this information to create intuitive dashboards and reports that allow for decision making at various levels of the organization, including long-term strategic actions on technology investments, updates, and modifications.
• The AI system can be programmed to act autonomously on actions such as dynamical workload management and isolating nodes to contain damages.
• The key difference from traditional automation tools is that the rules for action do not have to be hardcoded or informed explicitly. Instead, the AI tool can be trained from historical events around the optimal system behavior and trigger actions to address any deviation when specific performance thresholds are exceeded.
• These triggers can also be replaced with insights, guidance, or change requests that can be manually reviewed and approved based on the organizational policies.

While the insights and subsequent control actions are not predefined in an AIOps solution, the tool uses a predefined knowledge graph and business service models for every underlying technology. The knowledge graph connects different nodes and identifies the relationships between these nodes and subsequently, the hardware components, IT services, and application components. Between the graph nodes to each so-called edge of the graph, the AI tool assigns weights or an importance value.

Based on the training of the model and the patterns of observed data, these weight values are updated autonomous, with different patterns of incidents ranked on the knowledge graph. Therefore, when a specific event and its corresponding series of events or traffic patterns are observed, the AI model is able to rank the nodes corresponding to the highest importance value or weights as a relevant criteria.

With this capability, AIOps teams can focus their efforts on innovation and service improvement instead of firefighting IT incidents that occur with little notice and can potentially cause a lasting impact. It is important to understand that the performance of AI models inherently depends on the data quality used to train them. If the data is sufficiently rich in terms of representing relationships between nodes and business service models and is available for processing in real time, the AI tool can help organizations identify and contain damages that might impact specific root cause nodes. It is also important to align cross-functional teams operating in silos and give them access to exhaustive log metrics data and proposed controlled action triggers.

In conclusion, AI-enabled root cause isolation is a powerful tool for incident management. Organizations can quickly identify and contain damage from IT outages or incidents, thus reducing risk and minimizing disruption to business operations.

Business organizations are rearchitecting their IT infrastructure and applications to overcome the challenges associated with older technologies. Instead of developing monolithic software tightly coupled with on-premises hardware that needs to be carefully managed to avoid unpredictable outages and performance downtime, organizations are turning to containerization and microservices that can run application components independent of the underlying hardware and external dependencies. The container acts as a bubble, where application components are packaged with all libraries, dependencies, and configuration files required to run a fully functional and portable computing environment.

This leads to a greater observability challenge for infrastructure and operations (I&O) teams: consumption far exceeds infrastructure budgets due to inadequate visibility into containerized systems. With a deluge of containers and infrastructure management tools spread across a large system, how do you keep track, process, and control the performance states of each application component and the wider infrastructure and consolidated system?

Combining Observability and Artificial Intelligence

Observability refers to the ability to infer the internal states of a system from its external outputs. In the context of distributed cloud computing, observability tools process log metrics data generated across the nodes of a networked system to trace an event to its origin. Observability is different from monitoring in that the latter uses an alert mechanism based on predefined and pre-configured rules. Unlike a monitoring scenario where metric thresholds can be directly attributed to potential events, observability takes a deeper perspective into gaining insights and understanding network behavior and application performance.

Modern observability tools are data-driven and rely on advanced artificial intelligence and machine learning (AI/ML) algorithms to classify events based on patterns hidden within network log big data. AI enhances observability capabilities to deliver predictable IT service outcomes in the following ways:

1. Modeling system behavior and dynamic services: Instead of manually creating relationships between configuration items across services and application components, an AI model can learn to model the system and its associated relationships. Once the model is trained to accurately emulate system behavior, the insights within new log metrics and changing system behavior can be mapped to system performance, identifying relationships, and discovering dependencies for observability use cases.
2. Adaptable learning and observability: As new containerized services are created, new configuration items may be dynamic and temporal—any dependencies could hold for a specified and unknown duration and cause significant impact to system performance. AI models can be trained dynamically, online, and on the fly as new metrics data is generated. This ensures observability while considering the changing system dynamics and therefore, accurate observability analysis.
3. Large-scale and complex analysis: Observability analysis involves the processing of log metrics from an ever-growing stream of information generated across the IT network. The parameters, relationships, and dependencies that affect each service and IT system grow exponentially, spreading across on-premises and cloud environments. Using fragmented infrastructure and application performance monitoring tools to keep track of all assets spread across the IT network is daunting at best. AI automates the process of collecting relevant metrics, discovering assets, and applying configuration changes automatically based on predefined organizational policies.
4. Cost optimization: With the growing number of container deployments, it gets challenging to keep track of container performance without an extensive and automated observability pipeline. AI technologies allow I&O teams to understand the true cost of distributed services and containerized infrastructure components with analysis of aggregated logs and traces that account for every component. AI models recognize where container deployments are over-provisioned and manage resources optimally as required. Therefore, infrastructure costs can be validated by consumption data and optimized based on the changing needs of Devs and QA teams.
5. Root cause analysis: The AI-enabled observability pipeline allows you to gain insights into the behavior of your IT system and ask “what-if” questions about how the system behaves with respect to changing dynamics, including introduction of new services, relationships, and configuration changes. This leads to faster debugging, root cause analysis, and proactive identification of potential impact before the incident spreads across the network.
6. Intelligent automation and integration: One of the most important tasks in generating accurate observability analysis is to collect data and integrate resource management across decoupled sources and tools. When I&O teams operate an observability pipeline that decouples the tools from the source of data, they can process metrics data separately, integrate the growing number of data sources, and use AI technologies to perform the necessary analysis. As a result, the task of problem identification and incident management can also be automated, and the integrated set of data assets can enable intelligent automation for application performance and infrastructure management tasks.
7. User experience improvements: AI models can be used to prioritize changes based on immediate customer feedback. By running observability data through the AI models, organizations can understand how specific system parameters, services, configuration changes, and performance metrics impact the end-user experience. The entire process can be automated for real-time analysis of system performance and to continuously make changes that generate improved value streams for the business and end-user.

With large-scale organizations increasingly investing in containerized technologies to improve the end-user experience, speed software development lifecycles, and improve the quality of software releases, I&O leaders are reevaluating the viability of traditional observability tools to effectively manage infrastructure operations. By combining advanced AI capabilities and observability, these organizations can gain insights into how complex infrastructure systems behave to help their IT teams optimize cost and infrastructure performance.

To aid with our understanding of Observability vs Monitoring let’s look at the evolution of the Enterprise IT world. Enterprise IT, application and business service development are increasingly complex. The interdependencies within the underlying architecture has become more fragmented resulting in difficulty visualizing the full IT Stack.

The internet delivers IT infrastructure services from hyperscale data centers at distant geographic locations. Companies are moving towards cloud-native delivery, resulting in modern distributed applications creating a perfect storm of complexity with constantly emerging technologies, hybrid-cloud infrastructures, and businesses expecting delivery of more features faster.

Companies are consuming these services – like microservices and containers – as distributed functions across layers of infrastructure and platform services. Consumers expect regular, continuous feature improvements through new releases.

To meet these requirements, IT service providers and enterprises must aggressively manage business service performance, improve stability, and predict & prevent performance degradation and outages—all in the context of the rapidly changing and evolving IT landscape. This requires closely observing and monitoring metrics and datasets related to service performance to optimize system availability, particularly during upgrades and code launches.

Observability seems like the hot new topic in the IT world, but the reality is it has been with us for a long time. Only recently, however, has it entered the IT realm, combining with monitoring to offer a more powerful approach to business service performance management. System observability and monitoring play critical roles in achieving system dependability — they may be interdependent but they’re not the same thing. Let’s understand the differences between monitoring and observability, and how they are both critical for enhanced end to end visibility.

What is monitoring?

In Enterprise IT, monitoring is the process of instrumenting specific components of infrastructure and applications to collect data – usually metrics, events, logs, and traces – and interpreting that data against thresholds, known patterns, and error conditions to turn the data into meaningful and actionable insights.

Monitoring is focused on the external behavior of a system, specifically those data targeted for collection. Monitoring is most effective in relatively stable environments, where key performance data and normal vs abnormal behavior is known. When enterprise IT was predominantly run in an organization’s own data center, monitoring was an appropriate way to approach managing the environment.

The introduction of public and private clouds, the adoption of DevOps, the emergence of new technologies and the massive scale of data brought on by digital transformation, the proliferation of mobile devices and IoT has created a situation where monitoring is no longer an effective approach for IT Operations.

What is observability?

The concept of Observability was introduced by R. Kalman in 1960 in the context of control systems theory. In control systems theory, observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs. In essence, it’s a method for learning about what you don’t know from what you do know. The relationship between the known and the unknown can be represented mathematically.

So given enough known, external data and time to do the mathematical calculations, the internal, unknown state of the system can be determined. This approach is well suited for modern, Enterprise IT, as distributed infrastructure components operate through multiple abstraction layers. This makes it impractical and challenging to understand the health of complex services by selecting specific components to instrument for telemetry and looking for threshold breaches, events, etc.

The challenge to implementing observability in IT has been the volume, variety, and velocity of external data, combined with having the computational power and domain knowledge needed to analyze and make sense of it in real-time. Effective IT Operations teams now need observability platforms that can consume vast quantities of data from a variety of sources and submit that data to immediate intensive computational analysis. Fortunately, such platforms, like BMC Helix Operations Management, are now available.

Comparing Observability and Monitoring

For simple systems, traditional monitoring is effective and can provide some measure of insight into a system’s health. Consider a single server machine. It can be easily monitored using metrics and parameters such as hardware energy consumption, temperature, data transfer rates, and processing speed. These parameters are known to be highly correlated with the health of internal system components.

Now consider a large, complex, business service. It is made up of multiple applications that span public and private clouds, a diversity of distributed infrastructure, and maybe even a mainframe. There are too many systems, some not directly accessible, that if monitored without knowledge of the key performance data, systems and error conditions, will generate too much uncontextualized data and, in turn, unnecessary alerts, data, and false flags.

In the second case, an observability and AIOps approach is needed. Rather than selecting the data to monitor and examine the behavior of that data relative to trends, known errors, etc., all available data from all systems should be consumed. Aggregated into a high-performance data store, it should be combined with a comprehensive topology of all assets, systems, and applications that builds a comprehensive model of relationships and dependencies.

On this foundational observability layer, high-performance, domain-informed AI and ML algorithms can be applied to determine which externally observable data are correlated with which services and infer the health of those services from their behavior. This is the power of an observability and AIOps approach, such as that used by BMC Helix Operations Management.

Coda: Observability in DevOps

The concept of observability is prominent in DevOps software development lifecycle (SDLC) methodologies. In earlier waterfall and agile frameworks, developers built new features and product lines while separate testing and operations teams tested for software dependability. This siloed approach meant that infrastructure operations and monitoring activities were beyond development’s scope. Projects were developed for success and not for failure: debuggability of the code was rarely a primary consideration. Infrastructure dependencies and application semantics were not adequately understood by the developers. Therefore, apps and services were built with low inherent dependability. Monitoring failed to yield sufficient information about the known-unknowns, let alone the unknown-unknowns, of distributed infrastructure systems.

The prevalence of DevOps has transformed SDLC. Monitoring goals are no longer limited to collecting and processing log data, metrics, and distributed event traces; monitoring is now used to make the system more observable. The scope of observability therefore encompasses the development segment and is facilitated by people, processes, and technologies operating across the SDLC pipeline.

Collaboration among cross-functional Devs, ITOps, Site Reliability Engineers (SRE) and QA personnel is critical when designing a highly performant and resilient system. Communication and feedback between developers and operations teams is necessary to achieve observability targets of the system that will help QA yield correct and insightful monitoring during the testing phase. As a result, DevOps teams can test systems and solutions for true real-world performance. Continuous iteration based on performance feedback can further enhance the ability to identify potential issues in the systems before the impact reaches end-users.

Observability offers actionable intelligence for optimizing performance, giving DevOps, SREs, and IT Operations increased agility by staying ahead of any potential service degradation or outages. Observability is not limited to technologies but also covers the approach, organizational culture, and priorities in reaching appropriate observability targets, and hence, value of monitoring initiatives.

After years of experimentation, business organizations are adopting cloud computing at scale. They have remained skeptical of their ability to manage regulatory compliance and security of sensitive information assets.

As they transition mission-critical IT workloads and apps to the cloud, their security posture is possibly a tradeoff between cost and performance of the cloud service. This is partly because government institutions mandate vastly different measures and policies on cloud computing. These mandates aren’t optional—the related fines and lawsuits are not the only implications of failure to compliance.

Today’s internet browsers are increasingly aware of their rights to data privacy and online security. Organizations that fail to protect user information stored in the cloud due to inadequate security measures as mandated by regulatory compliance therefore also compromise user trust and brand loyalty.

Since these regulations lay down the bare minimum requirements on security in the cloud, it’s important to understand cloud compliance regulations and follow the industry proven best practices on cloud security and governance.

Cloud compliance stats

Compliance of cloud-based solutions is one of the leading challenges facing organizations that aim to migrate existing workloads to the cloud. According to recent research surveys:

• 94% of IT and security professionals believe that compliance is a top priority for their organization. At the same time, 45% are also not concerned about penalties for noncompliance.
• More than 50% of the organizations face the compliance and audit challenges associated with Infrastructure as a Service (IaaS) cloud solutions.
• 32% of the organizations found incorrect access authorizations and privileges assigned to users. 60% are considered as shadow administrators.
• Under two-thirds (63%) of users pause to consider the organization’s data collection and storage practices before sharing sensitive information with them.
• Data classification due to cloud computing makes real and true encryption a challenge, according to 65% of organizations.

Cloud compliance regulations

Let’s begin the discussion with a quick review of the common cloud compliance regulations applicable to organizations in different industry verticals:

• HIPAA (Health Insurance Portability and Accountability Act) mandates security of electronic healthcare information, confidentiality and privacy of health related information, and information access for insurance.
• PCI DSS (Payment Card Industry Data Security Standard) is a set of security standards that enable all organizations to accept, process, store and transmit credit card and financial information.
• GLBA (Gramm-Leach-Bliley Act) requires organizations to communicate how user information is shared and protected, provide right to opt-out and apply specific mandated protections.
• PIPEDA (Personal Information Protection and Electronic Documents Act) provides rules for organizations to handle user information in conducting commercial activities.
• EU GDPR (General Data Protection Regulation), the most stringent privacy and security regulations, mandate an exhaustive set of requirements on organizations handling data of European Union (EU) residents. GDPR imposes harsh penalties for noncompliance.
• SOX (Sarbanes–Oxley Act) mandates requirements on financial disclosures, audits, and controls of information systems processing financial information.
• U.S. State Breach Laws: All 50 U.S. states require organizations to notify individuals in event of security breaches involving their personally identifiable information.
• NIST (National Institute of Standards and Technology) is the organization that provides guidelines on technology related matters such as standards, security, innovation, and economic competitiveness.
• FedRAMP (Federal Risk and Authorization Management Program) is a standardized program for security assessment and evaluation of cloud-based systems.

How to achieve cloud compliance

Cloud compliance regulations are constantly changing and updated to meet the growing demands of information security and user privacy. Adhering to the exhaustive set of cloud compliance regulations seems like a daunting task but we’ve put together a few important tips to successfully achieve compliance in the cloud:

Know your compliance regulations

Compliance is not easy but getting to know the applicable regulations is the first step toward achieving compliance. Understanding the regulations and optimizing the compliance infrastructure may require external assistance through consultants and experts, which is costly—but not as expensive as noncompliance.

Know your responsibilities

Cloud vendors typically only offer a model of shared responsibility as it pertains to security and compliance. It’s important to fully understand your own responsibilities and adopt the measures necessary to guarantee compliance from your end.

Manage information access & controls

Monitor how your data in the cloud is accessed and controlled. Look out for identity and access control lapses or anomalous behavior. Adopt the principle of least privilege access: users are allowed to access only the information and resources necessary, and no more.

Conduct audits routinely

Examine cloud compliance regularly. Identify the shortcomings of your IT environment as well as the organizational culture and workforce behavior, which may involve practices directly and indirectly violating compliance regulations.

Know how your data is stored

IT workloads are shared dynamically between hardware resources that make up a cloud environment. Especially for hybrid and multi-cloud environments, make sure that your IT asset distribution is optimized for minimal security risk.

Encrypt, encrypt, encrypt

Always encrypt sensitive business information, which means that the data remains secure even when it is compromised. Apply multiple layers of security where necessary and viable.

Related reading

• BMC Security & Compliance Blog
• BMC Multi-Cloud Blog
• What Is GRC? Governance, Risk, and Compliance Explained
• SecOps vs InfoSec: An IT Security Comparison
• Multi-Cloud vs Hybrid Cloud: What’s The Difference?
• The State of SaaS Today: Growth Trends & Statistics

DevOps organizations monitor their CI/CD pipeline across three groups of metrics:

• Automation performance
• Speed
• Quality

With continuous delivery of high-quality software releases, organizations are able to respond to changing market needs faster than their competition and maintain improved end-user experiences. How can you achieve this goal?

Let’s discuss some of the critical aspects of a healthy CI/CD pipeline and highlight the key metrics that must be monitored and improved to optimize CI/CD performance.

(This article is part of our DevOps Guide. Use the right-hand menu to go deeper into individual practices and concepts.)

CI/CD brief recap

But first, what is CI/CD and why is it important?

Continuous Integration (CI) refers to the process of merging software builds on a continuous basis. The development teams divide the large-scale project into small coding tasks and deliver the code updates iteratively, on an ongoing basis. The builds are pushed to a centralized repository where further automation, QA, and analysis takes place.

Continuous Delivery (CD) takes the continuously integrated software builds and extends the process with automated release. All approved code changes and software builds are automatically released to production where the test results are further evaluated and the software is available for deployment in the real world.

Deployment often requires DevOps teams to follow a manual governance process. However, an automation solution may also be used to continuously approve software builds at the end of the software development (SDLC) pipeline, making it a Continuous Deployment process.

(Read more about CI/CD or set up your own CI/CD pipeline.)

Metrics for optimizing the DevOps CI/CD pipeline

Now, let’s turn to actual metrics that can help you determine how mature your DevOps pipeline is. We’ll look at three areas.

Agile CI/CD Pipeline

In regard to delivering high quality software, infusing performance and security into the code from the ground up, developers should be able to write code that is QA-ready.

DevOps organizations should introduce test procedures early during the SDLC lifecycle—a practice known as shifting left—and developers should respond with quality improvements well before the build reaches production environments.

DevOps organizations can measure and optimize the performance of their CI/CD pipeline by using the following key metrics:

• Test pass rate. The ratio between passed test cases with the total number of test cases.
• Number of bugs. The number of issues that cause performance issues at a later stage.
• Defect escape rate. The number of issues identified in the production stage compared to the number of issues identified in pre-production.
• Number of code branches. Number of feature components introduced into the development project.

Automation of CI/CD & QA

Automation is the heart of DevOps and a critical component of a healthy CI/CD pipeline. However, DevOps is not solely about automation. In fact, DevOps thrives on automation adopted strategically—to replace repetitive and predictable tasks by automation solutions and scripts.

Considering the lack of skilled workforce and the scale of development tasks in a CI/CD pipeline, DevOps organizations should maximize the scope of their automation capabilities while also closely evaluating automation performance. They can do so by monitoring the following automation metrics:

• Deployment frequency. Measure the throughput of your DevOps pipeline. How frequently can your organization deploy by automating the QA and CI/CD processes?
• Deployment size. Does automation help improve your code deployment capacity?
• Deployment success. Do frequent deployments cause downtime and outages, or other performance and security issues?

Infrastructure Dependability

DevOps organizations are expected to improve performance without disrupting the business. Considering the increased dependence on automation technologies and a cultural change focused on rapid and continuous delivery cycles, DevOps organizations need consistency of performance across the SDLC pipeline.

Dependability of infrastructure underlying high performance CI/CD pipeline responsible for hundreds (at times, thousands) of delivery cycles on a daily basis is therefore critical to the success of DevOps. How do you measure the dependability of your IT infrastructure?

Here are a few metrics to get you started:

• MTTF, MTTR, MTTD: Mean Time to Failure/Repair/Diagnose. These metrics quantify the risk associated with potential failures and the time it takes to recover to optimal performance. Learn more about reliability calculations and metrics for infrastructure or service performance.
• Time to value. Another key metric is the speed of Continuous Delivery cycle release performance. It refers to the time taken before a complete written software build is released into production. The delaying duration may be caused by a number of factors, including infrastructure resources and automation capabilities available to test and process the build, as well as the governance process necessary for final release.
• Infrastructure utilization. Evaluate the performance of every service node, server, hardware, and virtualized IT components. This information not only describes the computational performance available for CI/CD teams but also creates vast volumes of data that can be studied for security and performance issues facing the network infrastructure.

With these metrics reliably in place, you’ll be ready to understand how close to optimal you really are.

Related reading

• BMC DevOps Blog
• Deployment Pipelines (CI/CD) in Software Engineering
• Continuous Delivery Metrics
• SRE vs DevOps: What’s The Difference?
• DevOps Team Structure
• Choosing IT Metrics That Matter

When cloud computing was introduced to the masses, new startups and innovative startups were among the early adopters. Cloud vendors such as Amazon, Microsoft, and Google offered a myriad of cloud resources designed to run different types of IT workloads. The flexibility and variety of choice sharpened the appetite for a cloud-first business paradigm:

• Legacy applications and workloads were quickly relocating to the cloud.
• IT began building containerized apps and delivering services to a global user base via the internet.

The growing cloud adoption trend was quickly faced by IT management and governance challenges. According to research, solving the cloud governance challenge is the top priority for SMBs investing in cloud solutions. Large enterprises are equally concerned: 84% are worried about managing cloud spending.

Fortunately, large vendors such as Amazon Web Services (AWS) offer a vast library of cloud management and governance tools. In this article, we will explore the three categories of AWS cloud management solutions:

• Enable: Built-in governance control tools.
• Provision: AWS cloud management tools that allow users to allocate and use resources efficiently based on defined policies.
• Operate: Maximize the performance of your AWS cloud systems. Streamline governance and control, and ensure compliance.

(This tutorial is part of our AWS Guide. Use the right-hand menu to navigate.)

Enable tools

AWS Control Tower

Manages multiple AWS accounts and teams for your AWS cloud environment. Security, compliance, and visibility protocols extend to all accounts that are provisioned with a few simple clicks with the AWS Control Tower tool.

Benefits:

• Easy provisioning and configuration of multiple AWS accounts.
• Automate policy management: enforce rules, Service Control Policies (SCPs).
• Gain full dashboard visibility into accounts and policies.

AWS Organizations

Grow and scale your AWS environment by programmatically provisioning accounts, allocating resources, organizing workflows for account groups and simplifying the billing process for grouped accounts.

Benefits:

• Easily and quickly scale your AWS cloud environment.
• Central audit of scalable cloud environments.
• Simplified identity and access control systems.
• Optimize resource provisioning and reduce duplication with AWS Resource Access Manager (RAM) and AWS License Manager.

AWS Well-Architected Tool

Review existing workloads and compare your IT environment to the AWS architectural best practices. The tool uses the AWS Well-Architected Framework that allows users to develop secure IT networks optimized for multi-cloud environments.

Benefits:

• Free AWS cloud architecture guidance.
• Cloud workload monitoring for compliance to AWS architectural best practices.
• Identify performance bottlenecks, monitor workloads, and track changes.

Provisioning Tools

AWS CloudFormation

AWS CloudFormation provides a common language to provision foundational assets in your cloud instance. Using a basic text file, CloudFormation enables you to model and provision each asset required.

Benefits:

• Model your infrastructure from a single source: a text file
• Standardize the infrastructure for your entire organization in a simplified way
• Provisions can be automated and deployed over and over again without being rebuilt
• Demystify infrastructure by treating it like what it is: code

AWS Service Catalog

Enables users to oversee a robust index of services primed for use on AWS. With services that incorporate everything from virtual machine images, servers, applications and databases, AWS Service Catalog enables you to centrally administer programs. It empowers clients to rapidly deploy IT services they need, on-demand.

Benefits:

• Ensure your organization complies with industry standards
• Help users find IT services to deploy
• Manage IT services from one central point

AWS OpsWorks

Lets you write small instances of code to automate configurations. AWS OpsWorks main benefit is that it offers application and server management for Puppet, Chef, and Stacks; Chef and Puppet are automation platforms that allow you to use code to automate the configurations of your servers.

Using instances of Chef and Puppet designed for AWS, developers can deploy code that keeps their configurations in check. OpsWorks has three offerings:

• AWS OpsWorks for Chef Automate
• AWS OpsWorks for Puppet Enterprise
• AWS OpsWorks Stacks

AWS Trusted Advisor

AWS Trusted Advisor is a provisioning resource that provides on-demand, real-time guidance to AWS users that increases the overall performance of your AWS environment. It does this by optimizing the instance, recalibrating things that reduce cost, increase security, and more.

Benefits:

• Full access to a wide range of perks that optimize your AWS instance
• Increased security
• Fine-tuned performance
• Alerts and notifications

Operate Tools

Amazon Cloud Watch

Amazon CloudWatch provides monitoring administration services for AWS cloud resources and applications. Users benefit from the Amazon CloudWatch tool to gather and track data analytics, screen log records, set alerts, and respond to changes in your AWS assets.

Benefits:

• Amazon EC2 monitoring
• AWS resource monitoring
• Custom metrics monitoring
• Log monitoring and storage
• View data in visual reports
• React to resource changes
• Set alarms

Amazon CloudWatch can screen AWS assets, for example, Amazon EC2 occurrences, Amazon DynamoDB tables and Amazon RDS DB instances and custom metrics produced by your applications and services.

AWS CloudTrail

An important operational tool, AWS CloudTrail helps enterprise businesses achieve compliance and track user activity. The service offers governance, compliance, operational and risk auditing of your account. Cloud Trail provides a comprehensive list of actions taken throughout AWS and aligned services.

Benefits:

• User activity is recorded in a secure log
• Compliance audits become easier with pre-stored event logs generated by the system
• Find areas where your system is vulnerable and monitor or fix them
• Security automation

AWS Config

Manage and audit configurations of your AWS environments and systems. The AWS Config keeps a repository of configuration records and evaluates them against optimal specifications.

It also tracks changes and dependencies between AWS resources. It helps users monitor the many configurations of their AWS instance and services—an otherwise time-consuming process. AWS Config offers assistance monitoring, assessing, auditing and evaluating configurations in one place.

Benefits:

• Continuously monitor and track configuration changes.
• Up to date with compliance and audit requirements.
• Manage changes at scale. Troubleshooting is simplified and can be automated.

AWS Systems Manager

AWS Systems Manager gives you full control of the framework on AWS. Systems Manager offers an impactful, easy-to-use UI so you can see operational information from various sources and automate tasks needed for smooth operation. With Systems Manager, you can assemble assets by application, monitor operational system info and activate resources.

Benefits:

• Ensures security and compliance
• Includes management of hybrid environments
• Full visibility of resource groups and configurations lets you have greater control
• Perfect for automation, easy-to-use
• Detect problems more quickly

Visit the AWS Management Tools homepage for more tools and detailed descriptions.

Third-party tools for managing AWS

In addition to the tools created by AWS, a number of third-party vendors offer resources for provisioning, ops management, monitoring and configurations.

RightScale

RightScale is a multi-use tool that helps with operations management and provisioning. This tool is also used for monitoring governance and optimizing for cost. This cloud management platform offers users the ability to manage all their clouds from one UI.

SCALR

Similar to RightScale, SCALR has a number of functions that are helpful for users in an AWS environment. The aim of this service is to increase productivity, reduce cost, enhance security, and prevent common concerns such as vendor lock-in. All the while, offering a flexible environment for users on a public, private, or hybrid cloud.

Hybridfox

Hybridfox is a popular Chrome add-on that works with a number of IaaS/PaaS providers, including AWS. It can be used with public and private clouds. It’s perfect for users who have multiple cloud environments because it allows for switching between them seamlessly.

Cloudability

Cloudability is a full-service cloud suite that offers users migration assistance, configuration management, and operations management. Cloudability helps to ensure governance and compliance needs are met, while offering a full suite of services to AWS users.

Ylastic

Ylastic is a cloud management service that focuses on managing user instances of AWS in an intuitive way and offering data analytic and backup options. Ylastic touches operations management, configuration management, security, compliance and more.

While the differences between some of these tools may seem small, something like red-flag resolution and alerts could make all the difference for enterprise business leaders. In many instances, it comes down to personal preference.

Overall, when purchasing any new services or applications, it’s important to first take inventory of the unique needs of your business, then decide on the right course of action. Apart from choosing the right services, implementing an effective cloud management strategy is also of paramount importance.

Related reading

• BMC Multi-Cloud Blog
• What’s AWS VPC? Amazon Virtual Private Cloud Explained
• Cloud Management Tools: Everything You Need To Know
• Protecting Your Data in the Cloud
• AWS Braket Quantum Computing: How To Get Started
• Cloud Governance Best Practices