7 Incident Commander Skills Every On-Call Rotation Should Train

The best incident commander on your team is probably not the best debugger. The role is a distinct skill set, and it is a learnable one. If your rotation currently rewards technical depth over coordination discipline, your MTTR is higher than it needs to be by 20 to 30%. Here are the seven skills to train.

Can they frame the problem in one sentence?

At minute five of an incident, the commander must be able to say what is happening in one sentence a stakeholder would understand. Not a hypothesis; a description of the symptom.

"The checkout API is returning 5xx for roughly 12% of requests, elevated for the past 8 minutes" is a framing. "We're investigating a database issue" is not; it is a hypothesis dressed as a symptom.

Train it: give a new commander a fake incident with 10 lines of dashboard data and 90 seconds to produce the one-sentence framing. Review whether the sentence names the symptom, the scope, and the duration, without naming a cause.

Can they refuse to debug?

The single hardest habit for engineers to build as commanders is keeping their hands off the keyboard. The instinct when you see a familiar symptom is to open a query editor. The moment you do, you have downgraded yourself from commander to junior responder, because you cannot coordinate while pattern-matching.

Train it: pair a new commander with an experienced one, and score the shadow session on how many keystrokes the new commander made against a code editor, a query, or a dashboard. Target zero for the first 30 minutes. Failure to hold this discipline is the top predictor of prolonged incidents.

Can they run structured status intervals?

The rhythm of a well-run incident is a status update every 10 minutes for a SEV2, every 5 minutes for a SEV1. The commander runs the interval, whether or not anyone else has produced new information.

The structure of each update is fixed: current status, what the team is doing right now, ETA to the next update, whether severity has changed. If the commander cannot fill in one of those fields, that itself is the update, and it prompts the right next action ("we have no ETA because we do not have a hypothesis yet, so the next 10 minutes are pure diagnosis").

Train it: run a drill where the "incident" is a slow escalation with no new information for 15 minutes. Score whether the commander held the interval or drifted into silence.

Do they escalate on time?

Every escalation ladder has thresholds. A good commander watches the clock and escalates at the threshold, not at the moment they feel worried. Feeling is a lagging indicator; the clock is a leading one.

• If the primary has not acked in 5 minutes, the secondary should already be paged.
• If the incident hits 30 minutes at SEV2, the engineering manager should already be in the channel.
• If a customer-impacting SEV1 hits 60 minutes, exec comms should already have gone out.

Train it: run a drill with fake wall-clock time acceleration and score whether the commander escalated at the threshold or 5 to 10 minutes after. Late escalation is the most common failure mode and the most damaging.

Can they manage scope creep during the incident?

Halfway through most SEV2s, someone in the channel notices a second, unrelated issue. "Also, the deploy pipeline looks funny." A good commander explicitly parks it: "Noted. Not in scope for this incident. File a ticket, we will look at it after." An average commander lets the team split attention across both, and both problems take longer.

Scope creep also happens with the fix: someone proposes a broader refactor while the incident is active. The right answer is almost always no. The incident is not the time to fix everything you have wanted to fix; it is the time to stop the bleeding.

Train it: seed a drill with two unrelated distractions in the middle. Score whether the commander parked them cleanly or let them derail the response.

Can they delegate comms cleanly?

The commander should never be writing the status page update themselves. That is the comms lead's job. The commander says what to update; the comms lead writes it and posts it.

The failure mode here is quiet takeover: the commander writes the update because "it will be faster than explaining." That saves 90 seconds in the moment and costs the team the ability to develop a real comms lead for the next incident.

Train it: force the commander to give the comms lead the content in one sentence, and score whether the comms lead's output matched. If the commander cannot compress the intent to one sentence, that is the training gap.

Can they call closure without hedging?

The end of the incident is when a lot of commanders get sloppy. They say things like "I think we can call this resolved" or "does anyone see anything wrong?" That is not closure; that is asking for permission.

Real closure looks like this: "Metrics have been stable for 20 minutes. No known open risks. Declaring the incident resolved as of 15:47. Postmortem will be due Friday, owned by Maya. Channel stays open until the postmortem is drafted."

Four facts. No hedges. Everyone knows what happens next.

Train it: at the end of every drill, score the closure statement against a rubric. Did it name the resolution time, the postmortem owner, and the channel disposition? Did it hedge? A hedged closure signals to the team that the incident might reopen, which affects who feels done and who does not.

How do you actually train these skills?

Two mechanisms, both required.

• Shadow rotations. Every new commander shadows an experienced one for their first 5 incidents. During the shadow, the new commander types their intended action in a side channel, and the senior commander executes. This surfaces the gap between what a new commander thinks the right action is and what the experienced one actually does.
• Quarterly drills. A staged SEV2 with a real page, a real 30-minute response, and a debrief against the seven-skill rubric. Rotate the commander role so every senior engineer has drilled at least once a year. Non-drilled engineers should never command a SEV1.

Skip either mechanism and commanders develop by trial in real incidents, which is a costly training program for both the responders and the customers.

The mistake to avoid

Do not treat commanding as a personality thing. "Alex is just good at incidents" is a way of avoiding the work of naming what Alex is doing that others are not. Break the skill into components, train each one, and rotate the role. Otherwise you have a rotation with two people who can command a SEV1, and the moment one is on vacation, your incident response degrades in ways nobody planned for.