Multi-stage continuous integration
Encyclopedia
Multi-Stage Continuous integration allows for a high degree of integration to occur in parallel while vastly reducing the scope of integration problems.
, it presumes that you are already following those recommended practices.
The larger and/or more complex the project, the higher the chance that the project becomes unstable. Alerts and broken builds increase as the project grows. Progress decreases and the mainline gets increasingly unstable. The risk of build failure increases exponentially as the number and locations of developers grow.
On a frequent basis, the team will decide to go to the second phase: integration with the mainline. In this phase, the team does the same thing that an individual would do in the case of mainline development. The team's branch must have all changes from the mainline merged in (the equivalent of a workspace update), there must be a successful build and all tests must pass. Integrating with the mainline will be easier than usual because only pre-integrated features will be in it, not features-in process. Then, the team's changes are merged into the mainline which will trigger a build and test cycle on the mainline. If that passes, then the team goes back to the first phase where individual developers work on their own tasks. Otherwise, the team works on getting the mainline working again, just as though they were an individual working on mainline.
Changes propagate as rapidly as possible, stopping only when there is a problem. Ideally, changes make it to the main integration area just as frequently as when doing mainline development. The difference is that fewer problems make it all the way to the main integration area. Multi-Stage CI allows for a high degree of integration to occur in parallel while vastly reducing the scope of integration problems.
Theory
Multi-Stage CI takes advantage of a basic unifying pattern of software development: software moves in stages from a state of immaturity to a state of maturity, and the work is broken down into logical units performed by interdependent teams that integrate the different parts together over time. What changes from shop to shop is the number of stages, the number and size of teams, and the structure of the team interdependencies.Recommended Practices
Multi-stage Continuous Integration is an expansion upon Continuous integrationContinuous integration
In software engineering, continuous integration implements continuous processes of applying quality control — small pieces of effort, applied frequently...
, it presumes that you are already following those recommended practices.
The larger and/or more complex the project, the higher the chance that the project becomes unstable. Alerts and broken builds increase as the project grows. Progress decreases and the mainline gets increasingly unstable. The risk of build failure increases exponentially as the number and locations of developers grow.
Recommended Practice #1
Each developer works on their own task. As they make changes, CI is done against that team's branch. If it does not succeed, then that developer (possibly with help from her teammates) fixes the branch. When there is a problem, only that team is affected, not the whole development effort. This is similar to how stopping the line works in a modern lean manufacturing facility. If someone on the line pulls the "stop the line" cord, it only affects a segment of the line, not the whole line.On a frequent basis, the team will decide to go to the second phase: integration with the mainline. In this phase, the team does the same thing that an individual would do in the case of mainline development. The team's branch must have all changes from the mainline merged in (the equivalent of a workspace update), there must be a successful build and all tests must pass. Integrating with the mainline will be easier than usual because only pre-integrated features will be in it, not features-in process. Then, the team's changes are merged into the mainline which will trigger a build and test cycle on the mainline. If that passes, then the team goes back to the first phase where individual developers work on their own tasks. Otherwise, the team works on getting the mainline working again, just as though they were an individual working on mainline.
Changes propagate as rapidly as possible, stopping only when there is a problem. Ideally, changes make it to the main integration area just as frequently as when doing mainline development. The difference is that fewer problems make it all the way to the main integration area. Multi-Stage CI allows for a high degree of integration to occur in parallel while vastly reducing the scope of integration problems.
Advantages
Multi-stage Continuous integration has many advantages:- When unit tests fail, or a bug is discovered, developers might revert the codebase back to a bug-free state, without wasting time debuggingDebuggingDebugging is a methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece of electronic hardware, thus making it behave as expected. Debugging tends to be harder when various subsystems are tightly coupled, as changes in one may cause bugs to emerge...
. - Integration problems are detected and fixed continuously - no last minute hiatus before release dates;
- Early warning of broken/incompatible code;
- Early warning of conflicting changes;
- Immediate unit testing of all changes;
- Constant availability of a "current" build for testing, demo, or release purposes;
- The immediate impact of checking in incomplete or broken code acts as an incentive to developers to learn to work more incrementally with shorter feedback cycles.
Tools
Tools that support Multi-stage continuous integration include:- AccuRevAccuRev SCMAccuRev is a software configuration management application developed by AccuRev, Inc. and was first released in 2002.- Overview :AccuRev is a centralized version control system which uses a client/server model. Communication is performed via TCP/IP using an XML based protocol for actions, and...
- Version Control and ALM tool - Electric CloudElectric CloudElectric Cloud, Inc. is a privately held software corporation specializing in high-performance software build tools. Electric Cloud is based in Sunnyvale, California and currently offers two products. The first product, ElectricAccelerator, speeds up make, Microsoft Visual Studio, and Apache Ant...
— Build, test and deployment framework tool designed to automate the software production lifecycle - AnthillProAnthillProAnthillPro is a continuous integration server from Urbancode.Released in 2001, AnthillPro is one of the original Continuous Integration servers. Currently in its third generation, it supports distributed as well as cross-platform builds in .Net, Java, C/C++ and other languages...
- Build, dependency, release tool - Rational Team Concert ALM-Platform
See also
- Agile software developmentAgile software developmentAgile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams...
- Build automationBuild AutomationBuild automation is the act of scripting or automating a wide variety of tasks that software developers do in their day-to-day activities including things like:* compiling computer source code into binary code* packaging binary code* running tests...
- Continuous designContinuous designContinuous design is a software development practice of creating and modifying the design of a system as it is developed, rather than specifying the system completely before development starts, or in bursts at the beginning of each iteration...
- Continuous integrationContinuous integrationIn software engineering, continuous integration implements continuous processes of applying quality control — small pieces of effort, applied frequently...
- Test-driven developmentTest-driven developmentTest-driven development is a software development process that relies on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new...