----
How the Xen Project Hypervisor team manages releases
// CloudStack Blog
With the Xen Project Hypervisor 4.4 having been released a few weeks ago, the project is starting the planning cycle for version 4.5 of the Hypervisor. So I thought it is worth walking you through how we manage releases.
Welcoming Oracle's Konrad Rzeszutek Wilk as Release Manager
But first I wanted to thank George Dunlap from Citrix for successfully managing the 4.3 and 4.4 releases of the Xen Project Hypervisor. The Release Manager role is a volunteer role open to Xen Project maintainers. George, has stepped down from his role recently to find more time for coding and help bootstrap the CentOS virtualization SIG.
Konrad Rzeszutek Wilk has volunteered to take on the role for the 4.5 release. A big welcome and Thank You!
Konrad is Software Development Manager at Oracle. His group's mission is to make Linux and Xen Project virtualization better and faster. As part of this work Konrad has been the maintainer of the Xen Project subsystem in Linux, Xen Project maintainer and now also Release Manager for the 4.5 release of the Xen Project Hypervisor. Konrad has been active in the Linux and Xen communities for more than 6 years and was instrumental in adding Xen Project support to the Linux Kernel.
How The Xen Project manages releases
As is the case for many open source projects, the Xen Project community does not maintain a committed roadmap as proprietary software vendors do. Instead, we strive to accurately track development for new releases, with a predictable release cadence for major releases and maintenance releases. We aim to release the Xen Project Hypervisor every 6-7 months: historically our release cadences ranged from 9 to 18 months. Introducing the Release Manager role was instrumental to getting us to shorter and a predictable release cadence.
How Xen Project contributors operate
Our best practice recommends that contributors announce larger new features on the xen-devel mailing list. Otherwise there is a risk that several contributors may start implementing a feature in parallel, wasting time and creating friction. The Release Manager will usually follow up and ask whether the feature should be included into the releases' backlog. There is also a monthly community call to raise issues and discuss bigger features. We run a Hackathon once a year (typically in spring) and a 1/2 day face-2-face meeting before or after our Developer Summit at which all of our core developers are present. These face-2-face meetings are usually used to bootstrap the planning process for a release.
How the Xen Project creates a Roadmap
During planning and development of a release, the Release Manager sends out monthly development update e-mails on xen-devel. These mails are labelled Xen x.y Development Update: you can see examples by following the 4.4 mails. In these mails, we ask developers
- whether they have items to be tracked on the roadmap and if so add them to a backlog
- asks for status updates, when they are on the backlog
- asks how likely it is that a feature will be in our codeline (including reviews) by the next update
Towards the end of the development cycle we start to treat blocker and critical bugs in the same way: aka they are tracked by the Release Manager on he backlog. Public announcements on release status are made on our blog and the announce list at critical points in the release cycle.
Stages within a Release Cycle
The following diagram shows the key stages, gates and git branches in our release cycle.
Feature Development / Feature Freeze: Feature development starts as soon as the release branch for the previous release has been created. It ends at the Feature Freeze point, when we typically won't allow new features into the master branch. Patches that were submitted before Feature Freeze and are being reviewed may still be let in based on a risk analysis. This phase is typically 3-4 months long. The timing is adapted based on major holidays (XMas and New Years) or major conferences, such as Xen Project Developer Summits.
Hardening / Code Freeze: During the hardening phase, which typically lasts 3-4 weeks, we complete any remaining reviews of patches that were posted during development and focus on bug fixes. This phase ends with the Code Freeze point, after which each bug will be considered on a case-by-case basis and risk to the quality of the release. Major vendors will start running their complete test suites during this phase of development. This phase ends with the first release candidate (RC1).
Release Candidates / Release: We close the release cycle making a number of release candidates (RCs). Historically, we needed 4-6 RCs and create one every 1-2 weeks. During the Release Candidate phase, we run Xen Project Test Days and major vendors in the community run their test suites against RCs. Only bugs of blocker severity are fixed. This phase typically lasts 6 weeks, but may be extended if a blocker bugs are found that need to be addressed. We end this phase by creating a stable release branch.
Release Announcement: Usually we make an official Release Announcement shortly after the stable release branch has been created. But sometimes, we delay by a few days to coordinate press activities.
Improving the Process
We generally have discussions on how to improve the release process, at face-2-face meetings such as the Hackathons and Developer meetings. At previous developer meetings, we decided to focus on improving our test infrastructure and started using Coverity Scan regularly on our codebase and formed the test framework working group. In the upcoming May Hackathon, we will for example have a discussion how to handle the ever increasing traffic on our development list (which regularly hits more than 4000 messages per month).
Further Information
- On the 4.5 Roadmap (note that we are only just staring to plan; as such there is not yet much information available)
- On Maintenance Releseas
- On User Feedback
----
Shared via my feedly reader
Sent from my iPhone
No comments:
Post a Comment