Monitoring & Observability
Prometheus collects metrics and powers alerting in DevOps systems.
On this page
Use this page for fast recall. Use Full documentation when you want the complete end-to-end path.
A simple 5-step flow you can follow when using Prometheus in real work.
1) Setup
Install/run the tool and confirm version. Create a minimal config.
2) Small change
Do one small action end-to-end to prove the workflow.
3) Validate
Check output, logs, and status. Catch mistakes early.
4) Automate
Convert it into a repeatable script or pipeline step.
5) Productionize
Add safety: secrets, rollback, observability, and docs.
These are practical sequences you can copy into your own checklist or runbook.
Goal: Create alerts that map to user impact and are actionable.
- Start with SLO signals (latency, errors, saturation).
- Create recording rules for expensive queries.
- Add alert labels and runbooks.
- Test alerts with controlled failures.
- Review alert volume and iterate.
- Scraping
- PromQL
- Alert rules
Learning path (high-level):
- Metric collection
- Query writing
- Alert design
Run Prometheus
Command
docker run -p 9090:9090 prom/prometheusOpen UI
Command
http://localhost:9090Run query
Command
upShort descriptions and practical intent. Search, filter, copy, and reuse.
Showing 11
promtool check config prometheus.ymlValidate Prometheus config.
promtool check rules alerts.ymlValidate alert rule files.
curl http://localhost:9090/-/readyCheck readiness endpoint.
curl http://localhost:9090/-/healthyCheck health endpoint.
curl http://localhost:9090/api/v1/targetsList scrape targets via API.
curl http://localhost:9090/api/v1/rulesList active rules via API.
curl http://localhost:9090/api/v1/alertsList active alerts via API.
upPromQL: target is up (1) or down (0).
rate(http_requests_total[5m])PromQL: request rate over 5m.
sum(rate(http_requests_total[5m])) by (status)PromQL: rate by status.
histogram_quantile(0.95, sum(rate(request_duration_bucket[5m])) by (le))PromQL: p95 latency from histogram.
Small blocks you can drop into your terminal, config, or runbook.
Quickly list failing scrape targets (API)
bash
curl -s http://localhost:9090/api/v1/targets | headWhen things break, follow this order to stay calm and move fast.
- If targets are down: check service discovery, network, and scrape endpoint `/metrics`.
- If queries are slow: use recording rules and reduce label cardinality.
- If alerts fire constantly: add better thresholds and use multi-window burn-rate alerts.
The common mistakes that slow people down when using Prometheus.
- Copy-pasting commands without understanding inputs/outputs and side effects.
- Not documenting defaults (ports, paths, credentials) and then getting stuck in prod.
- Skipping logs and metrics when troubleshooting; always collect evidence first.
Do these tasks in order. You will feel the tool instead of just reading about it.
- Install or run the tool locally (or in Docker) and verify it works with a hello-world action.
- Create a minimal config and run the most common command 3 times (with a small change each time).
- Break something on purpose and document how you debugged it in your Notes.
Use these to test if you truly understand the basics (and can explain them clearly).
- Explain the tool’s role in a real CI/CD pipeline from commit to production.
- Describe the most common failure you’ve seen with this tool and how you fixed it.
- What would you monitor/alert on for this tool in production?