Scorecards are the unsung heroes that help engineering leaders take their teams to new heights. Their appeal lies in the capacity to convert complex objectives into clear, measurable outcomes which help align the entire team around a common goal.

‍

A scorecard functions as a critical tool for gauging and managing the performance of a service or an app within an organization. These scorecards measure a variety of important metrics and indicators like DORA metrics, observability maturity, production readiness, and operational maturity among others.

‍

Take the DORA metric for example. It provides a holistic view of your DevOps team's performance by leveraging critical metrics like deployment frequency, number of failures, and change failure rates enabling teams to ensure that they are meeting the operational benchmarks and progressing steadily towards defined goals.

‍

Your typical implementation of scorecards is not a one-time thing. It’s not a “build-it-and-forget-it” type process and requires a cycle that includes setting goals, measuring progress, and tweaking those goals.

‍

Your typical process would look like this:

‍

1. Set the goals: Start with defining what you want to achieve and track. It could be anything from reducing deployment time, improving code quality,  mean time to resolve, etc.
2. Define Metrics: Decide on what metrics you will use to measure and get them into the software catalog whether via plugin data, automation, or public API.
3. Measure Progress. Keep track of these metrics over time.
4. Learn from Results: Analyze the results to identify areas of improvement.
5. Update Goals: Based on your learnings, you’ll update these goals to better fit your end goals.

‍

The idea is not to create a rigid system that puts pressure on your team but rather to implement scorecards that will promote a culture of continuous learning and improvement.

‍

Scorecard examples 

To effectively integrate scorecards into your internal developer portal, consider the following common scorecards that are widely recognized for their ability to promote and measure engineering excellence:

Software Development Lifecycle Efficiency Scorecards

• Time to Prod: How long on average does a change request seat wait for a review? And how long does that review take on average? And after approval, how long does it take to be deployed to production? Keeping Track of these durations allows you to identify the bottlenecks in your SDLC process.
• Sprint Burndown: What’s the amount of work left undone by the end of each sprint?
• Alert and Incident Count: How many inaccuracies are being inserted into your systems as you develop? 

Production Readiness Scorecards

• Service Stability: How frequently does the service experience downtime or performance issues?
• Monitoring: Are the right metrics being tracked and monitored? Are their values healthy? How effective are the alerting mechanisms in place for identifying and responding to issues?
• Documentation: Is there comprehensive and up-to-date documentation for the service?
• Disaster Recovery: Are there clear strategies and procedures for disaster recovery? Are your escalation policies up to date? Is an on-call rotation properly established?
• Vulnerabilities and Versioning: Has any critical vulnerability been identified in one of the packages you’re using? Is the service compliant with the SDK versioning that your services are using?

DORA Metrics Scorecards

• Deployment Frequency: How often does your team deploy code to production?
• Lead Time for Changes: How long does it take to go from code commit to code successfully running in production?
• Time to Restore Service: How quickly can the team restore a service following a downtime event?
• Change Failure Rate: What percentage of changes to production fail?

Operational Maturity Scorecards

• Configuration Management: How effectively is the service's configuration managed and applied across environments?
• Security Practices: Are best practices in place for ensuring the security of the service?
• Peer Review: How rigorously are code and infrastructure changes reviewed before deployment?

Observability Coverage Scorecards

• Logging: Does the service produce sufficient logs for debugging and understanding user behavior?
• Tracing: Are there effective tracing mechanisms in place to diagnose problems?
• Metrics: Are key performance indicators and service level indicators being measured and adhered to?

‍

These scorecards are not exhaustive, but they can serve as a starting point for engineering teams looking to measure and enhance their engineering best practices. 

‍

Tailor these scorecards to your team's unique context and objectives, periodically review them, and adapt them as necessary to maintain a trajectory toward engineering excellence.