DevOps
Encyclopedia
"DevOps" is an emerging set of principles, methods and practices for communication, collaboration and integration between software development
(application/software engineering) and IT operations
(systems administration/infrastructure) professionals. It has developed in response to the emerging understanding of the interdependence and importance of both the development and operations disciplines in meeting an organization's goal of rapidly producing software products and services.
The role of a DevOps professional has similarities with the Chief Engineer at the Toyota Production System
. They have responsibilities on the project success, but no formal authority over different teams involved. This requires strong technical knowledge, to convince managers of the needs. This is possible by the endorsement of the Chief Engineer by the company executives.
and Agile Infrastructure/Agile Operations movements.
) are adopted in a traditional organization with separate departments for Dev, IT operations and QA
, development and deployment
activities, that previously did not need deep cross-departmental integration with IT support or QA, now require intimate multi-departmental collaboration
. Devops concerns more than just software deployments, however. It is a set of processes and methods for thinking about communication
and collaboration between departments.
Companies with very frequent releases may require a Devops awareness or orientation. Flickr
developed Devops capability to support a business requirement of ten deployments per day; this daily deployment cycle would be much higher at organizations producing multi-focus or multi-function applications. This is referred to as continuous deployment or continuous delivery and is frequently associated with the lean startup
methodology. Working group
s, professional associations and blog
s have sprouted on the topic since 2009.
Devops integration targets product delivery, quality testing, feature development and maintenance release
s in order to improve reliability and security and faster development and deployment cycles.
The adoption of Devops is being driven by factors such as:
Devops is frequently described as a more collaborative and productive relationship between development teams and operations teams. This improved relationship and collaboration increases efficiency
and reduces the production risk associated with frequent changes. There are emerging ROI measurements and potential metrics for Devops.
Reduced change scope: Adoption of agile or iterative development, in contrast to the traditional waterfall model, means each release has less change but occurs more often. This creates a smooth rate of progressive application change vs. the large impact effect of rare deployments–each of which contains a large number of changes.
Increased release coordination: The use of a strong release coordinator to bridge the expertise and communications gap between development and operations; employment of collaboration tools such as spreadsheet
s, conference call
s, instant messaging
and corporate portals (wikis, SharePoint) to ensure thorough understanding of the change and full cooperation of all involved.
Automation: Comprehensive deployment automation ensures repeatability of deployment tasks and reduces deployment errors.
In summary, as companies seek to streamline the cumbersome transition between development and operations, they typically confront 3 different types of problems:
Release Management Problems: Companies with release management
problems are looking for better release planning than spreadsheets. They want an easy way to understand release risks, dependencies, stage gates adherence and an ensure compliance.
Release/Deployment Coordination Problems: Teams with release/deployment coordination problems are focused on better execution of release/deployment events. They want better tracking of discrete activities, faster escalation of issues, documented process control and granular reporting.
Release/Deployment Automation Problems: Companies with release/deployment automation problems usually have existing automation but want to more flexibly manage and drive this automation - without needing to enter everything manually at the command-line. Ideally, this automation can be invoked by non-operations resources in specific non-production environments.
One way to start streamlining release process is to identify which of the above problems is the overall team's highest priority.
The release co-ordinator role (also referred to as a deployment or integration co-ordinator) has emerged from the release management
or release engineering teams. This role is similar to an air traffic controller
—performing real time co-ordination activities across diverse teams to achieve a group goal (safe landing and take-off) using shared resources (airspace, flight paths, airport runways, and terminal gates).
Release co-ordination contrasts with release management, which is often focused on planning and reporting on software changes, in order to control the release of specific application changes into production. Release engineering is concerned with the systematic and technical work related to building and deploying code into environments.
Change management
is the infrastructure discipline for tracking all types of changes in the enterprise IT environment—including both application and infrastructure changes. Change management is a core part of ITIL
v3.
Software development
Software development is the development of a software product...
(application/software engineering) and IT operations
Web operations
Web operations is a domain of expertise within IT systems management that involves the deployment, operation, maintenance, tuning, and repair of web-based applications....
(systems administration/infrastructure) professionals. It has developed in response to the emerging understanding of the interdependence and importance of both the development and operations disciplines in meeting an organization's goal of rapidly producing software products and services.
The role of a DevOps professional has similarities with the Chief Engineer at the Toyota Production System
Toyota Production System
The Toyota Production System is an integrated socio-technical system, developed by Toyota, that comprises its management philosophy and practices. The TPS organizes manufacturing and logistics for the automobile manufacturer, including interaction with suppliers and customers...
. They have responsibilities on the project success, but no formal authority over different teams involved. This requires strong technical knowledge, to convince managers of the needs. This is possible by the endorsement of the Chief Engineer by the company executives.
Predecessors
Many of the ideas (and people) involved in devops came from the Enterprise Systems ManagementSystems management
Systems management refers to enterprise-wide administration of distributed systems including computer systems. Systems management is strongly influenced by network management initiatives in telecommunications....
and Agile Infrastructure/Agile Operations movements.
Devops Days
The term "devops" was popularized through a series of "Devops Days" starting in 2009 in Belgium. Since then, there have been Devops Days conferences held in India, the US, Brazil, Australia , Germany and Sweden.Devops Methodologies
When new development methodologies (such as agile software developmentAgile software development
Agile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams...
) are adopted in a traditional organization with separate departments for Dev, IT operations and QA
Quality Assurance
Quality assurance, or QA for short, is the systematic monitoring and evaluation of the various aspects of a project, service or facility to maximize the probability that minimum standards of quality are being attained by the production process...
, development and deployment
Deployment
Deployment may refer to:* Deployment flowchart, a process mapping tool used to articulate the steps and stakeholders of a given process* System deployment, transforming a mechanical, electrical, or computer system from a packaged form to an operational state* Software deployment, all of the...
activities, that previously did not need deep cross-departmental integration with IT support or QA, now require intimate multi-departmental collaboration
Collaboration
Collaboration is working together to achieve a goal. It is a recursive process where two or more people or organizations work together to realize shared goals, — for example, an intriguing endeavor that is creative in nature—by sharing...
. Devops concerns more than just software deployments, however. It is a set of processes and methods for thinking about communication
Communication
Communication is the activity of conveying meaningful information. Communication requires a sender, a message, and an intended recipient, although the receiver need not be present or aware of the sender's intent to communicate at the time of communication; thus communication can occur across vast...
and collaboration between departments.
Companies with very frequent releases may require a Devops awareness or orientation. Flickr
Flickr
Flickr is an image hosting and video hosting website, web services suite, and online community that was created by Ludicorp in 2004 and acquired by Yahoo! in 2005. In addition to being a popular website for users to share and embed personal photographs, the service is widely used by bloggers to...
developed Devops capability to support a business requirement of ten deployments per day; this daily deployment cycle would be much higher at organizations producing multi-focus or multi-function applications. This is referred to as continuous deployment or continuous delivery and is frequently associated with the lean startup
Lean Startup
"Lean startup" is a term coined by Eric Ries, his method advocates the creation of rapid prototypes designed to test market assumptions, and uses customer feedback to evolve them much faster than via more traditional product development practices, such as the Waterfall model...
methodology. Working group
Working group
A working group is an interdisciplinary collaboration of researchers working on new research activities that would be difficult to develop under traditional funding mechanisms . The lifespan of the WG can last anywhere between a few months and several years...
s, professional associations and blog
Blog
A blog is a type of website or part of a website supposed to be updated with new content from time to time. Blogs are usually maintained by an individual with regular entries of commentary, descriptions of events, or other material such as graphics or video. Entries are commonly displayed in...
s have sprouted on the topic since 2009.
Devops integration targets product delivery, quality testing, feature development and maintenance release
Maintenance release
A maintenance release is a release of a product that does not add new features or content. For instance, in computer software, maintenance releases are typically intended to solve minor problems, typically "bugs" or security issues....
s in order to improve reliability and security and faster development and deployment cycles.
The adoption of Devops is being driven by factors such as:
- Use of agile and other development processesSoftware development processA software development process, also known as a software development life cycle , is a structure imposed on the development of a software product. Similar terms include software life cycle and software process. It is often considered a subset of systems development life cycle...
and methodologies - Demand for an increased rate of production releases from application and business unit stakeholders
- Wide availability of virtualized and cloud infrastructureCloud computingCloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software, and information are provided to computers and other devices as a utility over a network ....
from internal and external providers - Increased usage of data centerData centerA data center is a facility used to house computer systems and associated components, such as telecommunications and storage systems...
automation and configuration managementConfiguration managementConfiguration management is a field of management that focuses on establishing and maintaining consistency of a system or product's performance and its functional and physical attributes with its requirements, design, and operational information throughout its life.For information assurance, CM...
tools - It has also been suggested a side-effect of the dominant, traditional US management style ("the SloanAlfred P. SloanAlfred Pritchard Sloan, Jr. was an American business executive in the automotive industry. He was a long-time president, chairman, and CEO of General Motors Corporation...
model" vs "the ToyodaEiji Toyodais a prominent Japanese industrialist, who was largely responsible for bringing Toyota Motor Corporation to profitability and worldwide prominence during his tenure as president and later chairman.-Career:...
model") guarantees the development of silos of automationAutomationAutomation is the use of control systems and information technologies to reduce the need for human work in the production of goods and services. In the scope of industrialization, automation is a step beyond mechanization...
, thus creating "the Devops gap" that then requires Devops capability to address.
Devops is frequently described as a more collaborative and productive relationship between development teams and operations teams. This improved relationship and collaboration increases efficiency
Efficiency
Efficiency in general describes the extent to which time or effort is well used for the intended task or purpose. It is often used with the specific purpose of relaying the capability of a specific application of effort to produce a specific outcome effectively with a minimum amount or quantity of...
and reduces the production risk associated with frequent changes. There are emerging ROI measurements and potential metrics for Devops.
Devops Impact on Application Releases
In many firms, application release events are high stress and high-risk activities involving multiple teams. In a Devops organization, application releases are lower risk for the following reasons:Reduced change scope: Adoption of agile or iterative development, in contrast to the traditional waterfall model, means each release has less change but occurs more often. This creates a smooth rate of progressive application change vs. the large impact effect of rare deployments–each of which contains a large number of changes.
Increased release coordination: The use of a strong release coordinator to bridge the expertise and communications gap between development and operations; employment of collaboration tools such as spreadsheet
Spreadsheet
A spreadsheet is a computer application that simulates a paper accounting worksheet. It displays multiple cells usually in a two-dimensional matrix or grid consisting of rows and columns. Each cell contains alphanumeric text, numeric values or formulas...
s, conference call
Conference call
A conference call is a telephone call in which the calling party wishes to have more than one called party listen in to the audio portion of the call. The conference calls may be designed to allow the called party to participate during the call, or the call may be set up so that the called party...
s, instant messaging
Instant messaging
Instant Messaging is a form of real-time direct text-based chatting communication in push mode between two or more people using personal computers or other devices, along with shared clients. The user's text is conveyed over a network, such as the Internet...
and corporate portals (wikis, SharePoint) to ensure thorough understanding of the change and full cooperation of all involved.
Automation: Comprehensive deployment automation ensures repeatability of deployment tasks and reduces deployment errors.
Adoption of Devops Methodologies
Many organizations divide Development and System Administration into different departments. While Development departments are usually driven by user needs for frequent delivery of new features, Operations departments focus more on availability, stability of IT services and IT cost efficiency. These two contradicting goals create a "gap" between Development and Operations, which slows down IT's delivery of business value.- Developers are often not concerned about the impact of their code on Operations. They deliver their code without involving Operations into architectural decisions or code reviews.
- Developers lack communicating configuration or environment changes necessary to run the updated code base.
- Developers apply configuration changes manually to their workstations and do not document each necessary step. Often, coming up with the necessary configuration parameters for software involves experimentation with various parameters. After reaching a working state it is often difficult to identify the minimal steps to reach the working state.
- Developers tend to use a tool set optimized for rapid development: Fast feedback on code changes, low memory consumption of runtime environment, etc. This tool set is very different from the target runtime environment in Operations where stability and performance trump flexibility requirements.
- As Developers work on desktop computers they tend to use Operating Systems optimized for desktop use. The runtime environment is usually running a server operating system.
- In development, the systems run locally on the developers workstation. In Operations, the system is often distributed amongst various servers like web server, application server, database server, etc.
- Development is driven by functional requirements usually directly related to business needs
- Operations is driven by non-functional requirements like availability, stability, performance, etc.
- Operations tries to minimize risk for delivering on non-functional requirements by avoiding change
- If frequent change is avoided, but the amount of necessary change stays constant, every change will be bigger
- Bigger changes involve more risk, as more areas are affected by any change
- In trying to avoid change, Operations slows down the flow of new features to production and therefore slowing down Development's ability to deliver features
- Operations might not be fully aware of the application's internals making it hard to correctly define the runtime environment and update procedures
- Development might not be fully aware of the runtime environment making it hard to correctly adapt the code accordingly
Needs
- More and smaller changes–mean less risk
- Giving developers more environment control
- Giving infrastructure more application-centric understanding
- Clearly articulating simple processes
- Automating as much as possible
- Collaboration between dev and ops
In summary, as companies seek to streamline the cumbersome transition between development and operations, they typically confront 3 different types of problems:
Release Management Problems: Companies with release management
Release management
The release management process is a relatively new but rapidly growing discipline within software engineering of managing software releases....
problems are looking for better release planning than spreadsheets. They want an easy way to understand release risks, dependencies, stage gates adherence and an ensure compliance.
Release/Deployment Coordination Problems: Teams with release/deployment coordination problems are focused on better execution of release/deployment events. They want better tracking of discrete activities, faster escalation of issues, documented process control and granular reporting.
Release/Deployment Automation Problems: Companies with release/deployment automation problems usually have existing automation but want to more flexibly manage and drive this automation - without needing to enter everything manually at the command-line. Ideally, this automation can be invoked by non-operations resources in specific non-production environments.
One way to start streamlining release process is to identify which of the above problems is the overall team's highest priority.
Release coordinator
A relatively new role in enterprise IT which is primarily tasked with coordinating deployments of enterprise software to pre-production environments. The need for the release coordinator has been driven by:- The need to fill the Devops "gap"
- Increased infrastructure complexity - multiple layers and platforms which form web applications
- Growth in rate of releases - due to agile and iterative development
- Distributed teams - globally deployed, outsourced and hybrid development, testing and infrastructure teams
The release co-ordinator role (also referred to as a deployment or integration co-ordinator) has emerged from the release management
Release management
The release management process is a relatively new but rapidly growing discipline within software engineering of managing software releases....
or release engineering teams. This role is similar to an air traffic controller
Air traffic controller
Air traffic controllers are the people who expedite and maintain a safe and orderly flow of air traffic in the global air traffic control system. The position of the air traffic controller is one that requires highly specialized skills...
—performing real time co-ordination activities across diverse teams to achieve a group goal (safe landing and take-off) using shared resources (airspace, flight paths, airport runways, and terminal gates).
Release co-ordination contrasts with release management, which is often focused on planning and reporting on software changes, in order to control the release of specific application changes into production. Release engineering is concerned with the systematic and technical work related to building and deploying code into environments.
Change management
Change management
Change management is a structured approach to shifting/transitioning individuals, teams, and organizations from a current state to a desired future state. It is an organizational process aimed at helping employees to accept and embrace changes in their current business environment....
is the infrastructure discipline for tracking all types of changes in the enterprise IT environment—including both application and infrastructure changes. Change management is a core part of ITIL
Itil
Itil may mean:*Atil or Itil, the ancient capital of Khazaria*Itil , also Idel, Atil, Atal, the ancient and modern Turkic name of the river Volga.ITIL can stand for:*Information Technology Infrastructure Library...
v3.