Request access
Engineering operational reviews are critical for aligning engineering efforts with business objectives, evaluating team performance, and fostering a culture of continuous improvement. When done right, these reviews provide a transparent forum where engineering leadership, the C-suite, and engineers can collaborate effectively.
However, there are many pitfalls an organization can fall into that leads to it becoming just another time sink. Here's our take on how to structure, set up, and conduct an impactful engineering operational review.
Define the Purpose and Objectives
Every operational review should have a clear purpose. You should be clear with what you want to get out of the session and, sometimes more importantly, what the session is not. This way topics of the session are relevant and impactful.
The most important thing to have clear is what you want to achieve as an organization. Typically this would be the OKRs or KPIs that you want to achieve. Without having these clear - it is difficult to really get value out of these meetings.
For example, if your organization wants to achieve higher reliability, then some key KPIs to monitor would be SLOs on Uptime, MTTRs, and possibly any defined KPIs on Quality. In general, the idea is to improve on your engineering operations - either performance of software, cost effectiveness of it, or reliability. Ideally you use this time to identify root causes in issues you identify via KPIs.
One thing to point out is that the meeting has a high risk of turning into an initiative status reporting session. You as a leader have to make the call if this is important enough but it has the risk of derailing the meeting.
Set a Regular Cadence
Operational reviews should be scheduled consistently—monthly, quarterly, or bi-annually—depending on your organization's needs. A consistent cadence fosters a rhythm for monitoring progress and ensures that issues are addressed proactively rather than reactively.
Assemble the Right Stakeholders
Identifying the right stakeholders and ensuring they attend and participate is a critical part of the success of the operational reviews. The success of the review depends on having the right mix of participants. A boat cannot be rowed in the same direction without leadership being aligned, therefore a quorum of leadership attended should be had. Attendees should include:
- Engineering Leadership: To provide insights into team performance and technical direction.
- Engineering Management: To share insights to functioning and propagate decisions taken.
- Engineers: To share ground-level insights, challenges, and innovations directly.
Structure the Agenda
A well-structured agenda ensures the review is productive and time-efficient. One thing we find helpful - this is an engineering review, topics under discussion should be technical in nature. The goal is to identify the root cause issues of whatever you are seeing, impacting your metrics and helping to drive effective change in your organization.
Here’s a suggested format:
- Introduction: This section is purely for ensuring you have a quorum of attendees.
- Open Items and Follow Up: We suggest tracking the items that come up from a session and having people report on those items that are still open. For example, if you identify a service that has some open post-mortem work and it has broken an SLA - the owner should be able to report on what is going on that they were not able to complete the work. This is about accountability.
- Review KPI and Detractors: The important point here is, given KPIs, identifying those teams or services that require discussion. If you are for example tracking costs and there has been a large spike recently, it is worth explaining that spike and what is causing it.
- Review Latest Incidents: Any good operation review should dive into the incidents that have impacted the business. The goal is to understand the root cause and share learnings to the entire team.
- SLO Reviews: Here it's the same as KPI and detractors, the idea is to review those services whose SLOs are breaking company expectations. It is worth examining exactly what happened that caused it to go into the red and what is being done to remedy it.
- Adhoc Subjects: There will always be adhoc subjects, whether it be projects ongoing, updates, or announcements. They should be placed at the end so as not to disrupt the flow of the meeting.
Preparation and Facilitation of the Meeting
Preparation is key to a successful review. Ensure that the agenda is distributed ahead of the meeting and with clear expectations on preparation. Typically this means being able to adequately and in depth explain anything that is on the agenda. For example, a root cause analysis of an incident or the increase to the change lead time of a service. When going through topics - focus should be put on detail and root cause. A general heuristic is that if the conversation is technical in nature, you are in the right direction.
Documentation and Follow Up
After the meeting, create a detailed summary that includes:
- Key insights from the review.
- Decisions made during the meeting.
- Action items with assigned owners and deadlines.
The value of an operational review lies in follow-through. Track progress on action items and revisit unresolved issues in the next review. It should be pointed out that, while the meeting should be a safe place, repeat offenders and items should be discussed. A service that repeatedly fails an SLO is either not properly calibrated or has not been prioritized.
How you can use Rely.io
Rely.io is the internal developer portal to centralize your engineering workflows and enable you to drive excellence in your organization. With our engineering performance experience, it’ll help you focus on the data that matters and drive effective change. Check out the video below to learn more and click here to get started.
Closing Thoughts
An engineering operational review is more than just a meeting—it’s a strategic tool for aligning technical efforts with business goals, fostering collaboration across teams, and driving continuous improvement. By structuring and facilitating these reviews effectively, you can ensure that engineering efforts deliver maximum impact while empowering all stakeholders to contribute meaningfully.
Ready to get started? Begin by aligning your metrics, assembling the right stakeholders, and setting a cadence that fits your organization's pace.
Happy reviewing!
See related articles
How to Structure, Set Up, and Conduct an Effective Engineering Operational Review
Rely.io Update Roundup - December 2024
Why does improving Engineering Performance feel broken?