A few months ago we announced our issue tracker move to GitHub issues, including a transition in how we deal with incoming issues.
We’ve since taken the time to create a process to ensure we have a solid, transparent and shared workflow for all issues in all of our trackers. This process is in line with the new process for incoming pull requests (PRs), in fact, PRs and issues are basically the same thing on GitHub. This means that we can handle PRs and issues in the samway.
We’re very pleased with the results so far. The move to GitHub combined with a new process has allowed us (Umbraco HQ) to become more organized and have greatly increased the collaboration and throughput for incoming reports.
As you may or may not know, we have a few trackers to keep on top of, 7 in total to be specific:
- Umbraco-CMS is the most active one where we build our main product - the CMS
- The two HQ supported plugins for Umbraco; Courier and Forms, have their own trackers
- Then there is, of course, Umbraco Cloud
- The documentation we write is all open source and might have issues you want to report
- Did you know that the Starter Kit that comes with Umbraco is also open source, and has an issue tracker?
- The same goes for the community site, Our Umbraco
That’s a lot of things to keep our eyes on! 😅
How do we do it?
There are two types of incoming issues and PRs: bugs and features.
- A bug is typically a defect; something that is not working and giving unexpected results.
- A feature, on the other hand, would be defined as “not a bug”, most of the time adding something new to the product.
These two types of issues have a similar but slightly different workflow from each other. In the sections below, I’ll go into details on how we handle bugs and feature requests in all of our issue trackers today and going forward. I’ll also explain the details of how we categorize and label our issues, so you get a better idea of what the status is of your favourite issue.
How we categorise Bugs
Bugs will be evaluated by impact and urgency.
- Impact defines how big the problem is and looks at the “how many” question (people, systems, dollars, etc.).
- Urgency relates to time, i.e. will the problem grow quickly or can it wait. We evaluate this by looking at how long it has existed in relation to the unavailability of the expected feature or service. The existence of a workaround might make the issue less urgent.
When the bug is no longer present in the latest version we will encourage the reporter to upgrade. If the bug has low urgency it could end up in our sprint planning or in the “up for grabs” state. Of course, when it’s urgent, we fix the bug as soon as possible. The “idea” label indicates we’ll be able to re-evaluate it later when we do periodic reviews of the list of ideas.
How we categorise Features
For features, Umbraco HQ will give feedback on whether they fit with the product and direction we’re going in and will provide guidance by updating the description of the feature. The feature can then end up in our sprint planning or in the “up for grabs” state.
We might add the “idea” label which means that it’s a great idea we want to implement, but can’t right now. This way, we’ll remember and refer to the idea when we start working on items that relate to this feature. The ideas list also get a regular review of what is still relevant and where it can fit into our planning.
Some features are not a good fit and we might encourage you to extend Umbraco instead or create a package. This is also why we always recommend you to describe your feature and send it our way before you start on the actual build.
How we label Issues
You might have noticed recently that we’ve been labelling issues in a number of ways. This is so we can keep track of the type and state of each issue.
We use several types of labels to help us guide the progress and status for each issue. These include, for example, the type of issue (bug/feature) the state of the issue, categories and an expected version in which the item will be released.
Below, I’ll go through the labels used for:
- Overall category
- Community status
- Project relation
- Release information
- State
- Current status
- Issue type
Overall categories
Or simply: category
Category labels are used to indicate different items that have a similar theme to them. In the example above, taken from the documentation repository there are items related to broken links, code examples, grammar, etc.
Community status
Or simply: Community
These dark blue labels indicate that the item either has a PR or is “up for grabs”. When we add “up for grabs” we also add “help wanted” since GitHub loves showing those in the repository overview as a number of issues needing help, see the last line here: “30 issues need help”:
Project relation
Or simply: project
Project labels are used to group together larger projects that we working on for us to be able to easily get an overview of which issues belong together.
Release information
Or simply: release
A release label is used to indicate in which release the fix/feature will appear or has appeared. This label is used to generate the release notes on each Umbraco release on Our Umbraco.
States
Most of the state labels are used for columns on our internal sprint board. However, the “hq” labels are “special”. They indicate that we either need to discuss this item internally (“hq-discussion”) or that we’ve assigned it to someone at HQ to give a reply to help move the issue along (“HQ-reply”).
We hold regular meetings at HQ where we go through the items that need to be discussed and we update the items after we come to some kind of a conclusion.
Current status
Or simply: status
Sometimes we can’t make progress on an item since we need some feedback or we have something else blocking us.
The “discussion” status is a new label we’ve introduced recently. We would usually recommend contributors to discuss new ideas on the Forum on Our while the ideas are still forming. However, we recognize that for many people, GitHub is a more natural place to start these types of discussions so we’re experimenting with how that works at the moment.
We will be evaluating discussions periodically to make sure that they don’t just fizzle out, we want them to have some kind of progress over time.
Either way, HQ members follow all the new discussions started on the Forum in the “Contributing to Umbraco CMS” section and the new discussions on GitHub.
The “idea” status is used to indicate that the current item is a great idea, but neither HQ nor the community can make progress on at the moment. This means the issue is effectively parked but we’re happy for more feedback and discussion on it. We won’t close it since we agree it’s something we want to have in place in the future.
We regularly evaluate the list of ideas to see if they still make sense and if they fit into the themes of what we’re working on in the near future.
Type of issue
Or simply: type
The “type” label is used to indicate if the current item is a bug or feature. This indication is also used in the release notes on Our Umbraco.
Gold partners
Or simply: partner
Gold partners and some support customers have the unique ability to request a bug fix as part of their support contract. In order for everyone at Umbraco HQ to keep track of these specific issues, we'll add a "partner/gold" label to these bug reports.
Any other bugs or feature requests by support customers will not be labeled as such. However, Umbraco will make a best effort to resolve bugs with the subsequent official release.
What’s next?
In the near future, we’re working on some automation to post comments when we apply certain labels to issues and PRs. These comments will clarify what the labels mean and what is expected from either the contributor or from HQ.
One important thing to remember about processes: they’re imperfect and we’ll surely run into unexpected cases. We’ll adapt and look forward to learning from your input! We’re reading all of your posts on the Forum in the “Contributing to Umbraco CMS” category so if you have feedback on the above then please let us know so that we can learn and improve where necessary.
Now if you’ll excuse me, two new issues just came in that need a response. Cheerio! 👋
