Evaluating Managers: 5 heuristics to measure managerial impact

Am I an Effective manager? 

That is the million-dollar question every manager wants to know – we all want to be good at our jobs, we crave that satisfaction in knowing we are delivering impact and nurturing talent. Reviews, engagement surveys, and 360-degree feedback help a bit, but they come once or twice a year, and this delayed feedback loop slows down the learning process. 

Many objective performance measures with short feedback loops exist for evaluating machines. This simplicity does not translate to leadership evaluation since it involves working with people over extended periods. Thus, most resort to unwritten rules for evaluating managerial impact.

How do you know you are doing the right things as a manager? This post provides some answers; it gives line managers a self-evaluation mechanism and can be used by middle managers to guide their reports.

The heuristics

What separates bad leaders from good ones? And what makes great leaders exceptional? 

When I became a manager, I started looking for ready-made formulas to adopt; I wanted some fail-proof technique that would work in all situations and circumstances. Alas, I’ve realized this is not the case, especially after reflecting on the various leaders I have worked with across companies. 

Enhancing my experiences with my research helped me crystallize the five measures: execution, strategic vision, people management, team growth, and organizational influence.

These five signals should provide some early to medium-term signals to facilitate early course correction or doubling down on strengths.

Execution

Does your team consistently deliver promises or excuses? 

A high-performing team predictably meets its targets with few-to-zero excuses; a flailing team, on the other hand, delivers excuses with alarming regularity.

The responsibility for a team’s output lies solely with the manager. The manager’s job is to identify and fix all issues hampering team productivity. This means constantly scanning the horizon for potential disruptors and mitigating them: you anticipate and prevent roadblocks and regularly eliminate obstacles. It doesn’t end there though – you should also seed the culture by making the team autonomous and empowered to unblock itself in the future.

This signal is hard to hide and manifests within a quarter or two at most.

Questions

• Do you have mechanisms to ensure predictable high-quality outcomes? 
• Are there clear objective criteria for team performance and output? For example, SLAs, defect rates, customer satisfaction, lead time, and promises kept.
• Are estimates high, medium, or low confidence?
• Are the productivity-sapping constraints known? 
• Is there an efficient and widely accepted way of working on your team?
• How do partners perceive your team’s execution?

What you should do

Work style

Teams might balk at change due to the excruciating memory of poorly-implemented processes; alas, some groups become so anti-process that they throw the baby out with the bathwater. The absence of any process is itself a process – compute the cost of confusion arising from multiple sources of truth, meetings that could have been emails, and a blind spot on productivity bottlenecks.

You must establish feedback loops that continuously expose inefficiencies and facilitate improvements.

1. Start by understanding the current productivity bottlenecks – there is a wide plethora of research on this (e.g., Multipliers, DORA, SPACE, etc.). 
2. Experiment and tune until the pain point is removed; remember, the process is not the goal; the goal is to surface and eliminate constraints. Drop what does not work (yes, this includes obsolete processes too!).
3. Go back to step 1 to identify the next constraint: rinse and repeat.

Tip: If you only read one book on this topic, make it: The Goal: A Process of Ongoing Improvement.

Perception

Perception is subjective and prone to bias, but it is an excellent countervailing measure that prevents self-delusion. Every quarter, ask external collaborators to rate your team’s execution on a scale of 1 – 10. 

Using a ranking mitigates the subjectivity risk by forcing respondents to evaluate the team and give objective examples. If five respondents all rate your team below 7, there is an opportunity to close the gap.

Tip: Collect this sentiment from your manager, skip-level manager, peers, and partners in other roles (e.g., PM, design, support, etc.).

Estimation & Effectiveness

If estimates are unreliable, consider training the team on estimation; here is a great post. Integrate this into your team’s work style and evaluate regularly. 

Tip: any plan without a buffer for the inevitable unknown unknowns is automatically low-confidence.

Read my post on multipliers on boosting team productivity.

People management

Is your team a revolving door? Or a nursery where careers blossom?

Some attrition is inevitable – people find new opportunities, higher compensation, better WLB, prestige, etc. However, these departures can also mask a concern if no one pays close attention. Waiting for the annual employee surveys to unearth issues might be too late since the damage is already done.

This is a medium-to-long-term signal since relationships take considerable time to strengthen or weaken.

Questions

• How do you run your 1:1s?
• How well do you know each of your reports?
• Are opportunities aligned with individual career aspirations?
• What patterns emerge after reviewing your most recent three departures?
• What energizes your team? What drains your team?

What you should do

Above-average churn (> 13% for technology) and lethargy are red flags, and here are things you can do about them.

Get to know the people you work with; people are not just numbers on some spreadsheet – we have hopes, fears, desires, and wants. All great relationships are built on trust, so establishing trust is the first step.

Knowing your team makes it easier to match opportunities to interests, carve out exciting challenges, and tailor feedback delivery. 

Consider quarterly meetings to discuss your team’s identity, reflect on growth, and review values. Every team member should know what your team is about; a shared understanding of common values establishes psychological safety and promotes self-reinforcing team accountability.

Team development

Does your team have two years of experience or one year of experience repeated twice? 

A good signal for organizational stagnation is attempting to solve the same problem with the same approaches year in, year out. Take a look at your team after 6 – 18 months; is there measurable growth? Growth metrics include delivery impact, customer acquisition, scope, promotions, and so on. 

This is a long-term signal since it takes time to deliver large-scale impact; however, you should aim to have answers within a year.

Questions

• Are you still solving the same problems from one year ago?
• Have you targeted recruiting strategy at identified team gaps?
• How often do people get actionable and objective feedback?
• How has the delivered value increased in the past 3 – 4 quarters?
• How many people got promoted over the past year?
• What lessons did your team learn recently?

What you should do

Three things: review the past, take stock of the present, and plan for the future.

1. Review the past: You need to stretch more if you have made no mistakes in the past six months; if you score all the shots you take, you are standing too close to the net. Every project provides opportunities to finetune your leadership repertoire, so do a retrospective: what could have gone better? What can you change? What should you repeat? Are you carving out opportunities for high-leverage impact by developing talent?
2. Current state: A skills matrix is a great tool to quantify risk objectively. Look at boosting team resiliency by ensuring that critical operational knowledge is well-spread across the team. This invariably provides growth opportunities for more junior members while freeing up bottlenecked experts for higher-impact work. 
3. Look forward: Carving out a growth and succession plan is a great proactive exercise in team development. Forecast team development 3 – 4 quarters out. This activity should identify stretch opportunities and solidify career progression plans.

Strategic vision

Where do you see your team in a quarter, six months, and a year?

A clear vision simplifies decision-making in many ways. It defines the core priority, thus making it easy to evaluate tradeoffs, push back on non-aligned ideas, and make decisions despite incomplete information. 

Being agile does not necessitate creating chaos. Amorphous targets that continuously flip-flop are a perfect recipe for demotivation, frustration, and distrust. Estimate the energy expended on cajoling folks while rationalizing the newest reversal.

Great leaders are visionary – they set measurable aspirational targets, inspire their teams with the mission, and communicate progress effectively. 

This is a short-term signal, so aim to have a clear vision communicated to all parties (i.e., team members, stakeholders, leadership, and customers) within a quarter at most. 

Questions

• Is your team working towards a visionary goal with clear exit metrics? 
• Does every engineer understand how their work contributes to the long-term goals?
• Do workstreams cohere by providing mutually-overlapping benefits?
• Can anyone explain why a specific task is in the plan?

What you should do

Three things to consider when crafting a vision: 

• Business: What does your company need?
• Customers: What do your customers want?
• Technical: What will make your team more effective?

Work with cross-functional stakeholders to blend these three aspects into a coherent vision with objective metrics. Ensure alignment by repeatedly communicating the goals – everyone needs to know how their efforts fit into the bigger picture.

Simple communication is crucial for success, so tailoring information to the intended audience is a must. You must convey the right amount of information in the right format to the right audience. E.g., do PMs need to know about the lambda function in the East US2 region? Or do they need a high-level summary?

Do the hard work to simplify the esoteric terms – your partners will appreciate it.  

Organizational Influence

When did your team last contribute to the larger organization?

A rising tide raises all boats, and no team is an island; what we can achieve together far outstrips going at it alone. Look at improving the larger organization. There are multiple ways to contribute to the larger organization; examples include improving engineering efficiency, helping with hiring, mentoring others, and sharing best practices.

This is a short-term signal and should be evident within two quarters.

Questions

• Are you sharing best practices with your colleagues? 
• Are you trying to improve the larger organization? 
• Do you seek to learn and adopt best practices from peers? 
• Do you reinvent the wheel? 
• Do you know your first team?
• What’s your ‘brand’ in the org?

What you should do

The team is more intelligent than any individual, and what we can achieve via collaboration trumps going singly. As a manager, your first team will be your peers, and you should contribute to their success just as much as you deliver success on your direct team. 

Do not be territorial – it limits your growth and impact. Reach out to your partners and peers; set up periodic check-ins to catch up, and understand their domain, and they get to know yours as well. You’d be surprised to learn that you all face the same problems, and brainstorming sessions can lead to practical solutions. Read Lethain’s excellent Company, Team, Self article.

This concept also applies to seeking win-win outcomes with partners in different roles (PM, Design, Docs, Research, Sales, Support, Marketing, etc.). Are there ways you can be better together? Are they things you can leverage from their skills and perspectives? Move your larger organization towards more wins. 

Conclusion

Measuring a manager’s impact is hard since outcomes take time. This post provides early evaluation metrics as well as tips for course correction.

The manager takes full responsibility for the team – be it stagnation, execution woes, poor collaboration, churn, or a lack of focus. Here is a spreadsheet with all the questions and a radar chart for self-evaluation.

These are outcomes that can be measured; there are techniques for achieving these outcomes as a leader that you can deploy. 

Thanks to Ayuba, D, and a third reviewer for reviewing drafts of this post.