Model Context Protocol servers let AI assistants call tools: read logs, query metrics, restart services. During an incident, an assistant that can pull the right logs in seconds is genuinely useful. One that can restart the wrong service is a second incident. The safe way in is staged.
Stage 1: read-only tools#
Start with tools that cannot change anything:
- Service status: list apps, their containers, health and recent deploys.
- Logs: fetch the last N lines or a time window for one service, with a hard cap on size.
- Metrics: CPU, memory, error rate and latency for a service over a window.
- Recent changes: deploys and config changes in the last 24 hours, with who made them.
Most incident questions are answered by these four. "What changed?" and "what does the error log say?" cover a large share of real incidents.
Design the read tools carefully#
- Return structured, trimmed output. A tool that dumps 50,000 log lines fills the model's context and hides the signal.
- Redact secrets. Logs often contain tokens, connection strings and email addresses. Filter them in the tool, not in the prompt.
- Scope by environment. A read token for staging should not see production.
Stage 2: reversible actions with confirmation#
After the team trusts the read tools, add actions that are easy to undo: restart one container, roll back to the previous deploy, scale a worker. Require explicit human confirmation in the client for each call, and log every invocation with the requesting user.
Stage 3: everything else stays manual#
Deleting volumes, changing DNS, rotating credentials and editing firewall rules stay with humans. The cost of a wrong call is too high and the time saved is small.
Audit everything#
Every tool call should be recorded: which tool, which arguments, which user, and the result. When reviewing an incident, you want to know what the assistant saw and did.
Measure the benefit#
Track time to first useful clue during incidents before and after. If read-only tools do not help, write actions will not either.
A survey of MCP servers worth connecting for DevOps work, with notes on permissions, is in this roundup of MCP servers for DevOps.
Comments
No comments yet. Start the conversation.