鱼皮AI导航
鱼皮AI导航
  • 主页
  • 社区
  • AI工具
  • AI知识库
  • AI提示词
  • MCP
  • Skills
  • AI学习资源
  • AI资讯
  • GitHub开源
登录/注册

prometheus-configuration

实现基础设施与应用的全栈可观测性,支持自动发现服务、多源指标采集、预计算聚合、阈值告警及长期存储对接,适用于构建高可用监控体系和 SLO 保障场景。

开发运维
访问 GitHub
文档Skills.MD
name
prometheus-configuration
description
Set up Prometheus for comprehensive metric collection, storage, and monitoring of infrastructure and applications. Use when implementing metrics collection, setting up monitoring infrastructure, or configuring alerting systems.

Prometheus Configuration

Complete guide to Prometheus setup, metric collection, scrape configuration, and recording rules.

Purpose

Configure Prometheus for comprehensive metric collection, alerting, and monitoring of infrastructure and applications.

When to Use

  • Set up Prometheus monitoring
  • Configure metric scraping
  • Create recording rules
  • Design alert rules
  • Implement service discovery

Prometheus Architecture

▼
text
复制代码
┌──────────────┐ │ Applications │ ← Instrumented with client libraries └──────┬───────┘ │ /metrics endpoint ↓ ┌──────────────┐ │ Prometheus │ ← Scrapes metrics periodically │ Server │ └──────┬───────┘ │ ├─→ AlertManager (alerts) ├─→ Grafana (visualization) └─→ Long-term storage (Thanos/Cortex)

Installation

Kubernetes with Helm

▼
bash
复制代码
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update helm install prometheus prometheus-community/kube-prometheus-stack \ --namespace monitoring \ --create-namespace \ --set prometheus.prometheusSpec.retention=30d \ --set prometheus.prometheusSpec.storageVolumeSize=50Gi

Docker Compose

▼
yaml
复制代码
version: "3.8" services: prometheus: image: prom/prometheus:latest ports: - "9090:9090" volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml - prometheus-data:/prometheus command: - "--config.file=/etc/prometheus/prometheus.yml" - "--storage.tsdb.path=/prometheus" - "--storage.tsdb.retention.time=30d" volumes: prometheus-data:

Configuration File

prometheus.yml:

▼
yaml
复制代码
global: scrape_interval: 15s evaluation_interval: 15s external_labels: cluster: "production" region: "us-west-2" # Alertmanager configuration alerting: alertmanagers: - static_configs: - targets: - alertmanager:9093 # Load rules files rule_files: - /etc/prometheus/rules/*.yml # Scrape configurations scrape_configs: # Prometheus itself - job_name: "prometheus" static_configs: - targets: ["localhost:9090"] # Node exporters - job_name: "node-exporter" static_configs: - targets: - "node1:9100" - "node2:9100" - "node3:9100" relabel_configs: - source_labels: [__address__] target_label: instance regex: "([^:]+)(:[0-9]+)?" replacement: "${1}" # Kubernetes pods with annotations - job_name: "kubernetes-pods" kubernetes_sd_configs: - role: pod relabel_configs: - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] action: keep regex: true - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path] action: replace target_label: __metrics_path__ regex: (.+) - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port] action: replace regex: ([^:]+)(?::\d+)?;(\d+) replacement: $1:$2 target_label: __address__ - source_labels: [__meta_kubernetes_namespace] action: replace target_label: namespace - source_labels: [__meta_kubernetes_pod_name] action: replace target_label: pod # Application metrics - job_name: "my-app" static_configs: - targets: - "app1.example.com:9090" - "app2.example.com:9090" metrics_path: "/metrics" scheme: "https" tls_config: ca_file: /etc/prometheus/ca.crt cert_file: /etc/prometheus/client.crt key_file: /etc/prometheus/client.key

Reference: See assets/prometheus.yml.template

Scrape Configurations

Static Targets

▼
yaml
复制代码
scrape_configs: - job_name: "static-targets" static_configs: - targets: ["host1:9100", "host2:9100"] labels: env: "production" region: "us-west-2"

File-based Service Discovery

▼
yaml
复制代码
scrape_configs: - job_name: "file-sd" file_sd_configs: - files: - /etc/prometheus/targets/*.json - /etc/prometheus/targets/*.yml refresh_interval: 5m

targets/production.json:

▼
json
复制代码
[ { "targets": ["app1:9090", "app2:9090"], "labels": { "env": "production", "service": "api" } } ]

Kubernetes Service Discovery

▼
yaml
复制代码
scrape_configs: - job_name: "kubernetes-services" kubernetes_sd_configs: - role: service relabel_configs: - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] action: keep regex: true - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme] action: replace target_label: __scheme__ regex: (https?) - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] action: replace target_label: __metrics_path__ regex: (.+)

Reference: See references/scrape-configs.md

Recording Rules

Create pre-computed metrics for frequently queried expressions:

▼
yaml
复制代码
# /etc/prometheus/rules/recording_rules.yml groups: - name: api_metrics interval: 15s rules: # HTTP request rate per service - record: job:http_requests:rate5m expr: sum by (job) (rate(http_requests_total[5m])) # Error rate percentage - record: job:http_requests_errors:rate5m expr: sum by (job) (rate(http_requests_total{status=~"5.."}[5m])) - record: job:http_requests_error_rate:percentage expr: | (job:http_requests_errors:rate5m / job:http_requests:rate5m) * 100 # P95 latency - record: job:http_request_duration:p95 expr: | histogram_quantile(0.95, sum by (job, le) (rate(http_request_duration_seconds_bucket[5m])) ) - name: resource_metrics interval: 30s rules: # CPU utilization percentage - record: instance:node_cpu:utilization expr: | 100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) # Memory utilization percentage - record: instance:node_memory:utilization expr: | 100 - ((node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100) # Disk usage percentage - record: instance:node_disk:utilization expr: | 100 - ((node_filesystem_avail_bytes / node_filesystem_size_bytes) * 100)

Reference: See references/recording-rules.md

Alert Rules

▼
yaml
复制代码
# /etc/prometheus/rules/alert_rules.yml groups: - name: availability interval: 30s rules: - alert: ServiceDown expr: up{job="my-app"} == 0 for: 1m labels: severity: critical annotations: summary: "Service {{ $labels.instance }} is down" description: "{{ $labels.job }} has been down for more than 1 minute" - alert: HighErrorRate expr: job:http_requests_error_rate:percentage > 5 for: 5m labels: severity: warning annotations: summary: "High error rate for {{ $labels.job }}" description: "Error rate is {{ $value }}% (threshold: 5%)" - alert: HighLatency expr: job:http_request_duration:p95 > 1 for: 5m labels: severity: warning annotations: summary: "High latency for {{ $labels.job }}" description: "P95 latency is {{ $value }}s (threshold: 1s)" - name: resources interval: 1m rules: - alert: HighCPUUsage expr: instance:node_cpu:utilization > 80 for: 5m labels: severity: warning annotations: summary: "High CPU usage on {{ $labels.instance }}" description: "CPU usage is {{ $value }}%" - alert: HighMemoryUsage expr: instance:node_memory:utilization > 85 for: 5m labels: severity: warning annotations: summary: "High memory usage on {{ $labels.instance }}" description: "Memory usage is {{ $value }}%" - alert: DiskSpaceLow expr: instance:node_disk:utilization > 90 for: 5m labels: severity: critical annotations: summary: "Low disk space on {{ $labels.instance }}" description: "Disk usage is {{ $value }}%"

Validation

▼
bash
复制代码
# Validate configuration promtool check config prometheus.yml # Validate rules promtool check rules /etc/prometheus/rules/*.yml # Test query promtool query instant http://localhost:9090 'up'

Reference: See scripts/validate-prometheus.sh

Best Practices

  1. Use consistent naming for metrics (prefix_name_unit)
  2. Set appropriate scrape intervals (15-60s typical)
  3. Use recording rules for expensive queries
  4. Implement high availability (multiple Prometheus instances)
  5. Configure retention based on storage capacity
  6. Use relabeling for metric cleanup
  7. Monitor Prometheus itself
  8. Implement federation for large deployments
  9. Use Thanos/Cortex for long-term storage
  10. Document custom metrics

Troubleshooting

Check scrape targets:

▼
bash
复制代码
curl http://localhost:9090/api/v1/targets

Check configuration:

▼
bash
复制代码
curl http://localhost:9090/api/v1/status/config

Test query:

▼
bash
复制代码
curl 'http://localhost:9090/api/v1/query?query=up'

Reference Files

  • assets/prometheus.yml.template - Complete configuration template
  • references/scrape-configs.md - Scrape configuration patterns
  • references/recording-rules.md - Recording rule examples
  • scripts/validate-prometheus.sh - Validation script

Related Skills

  • grafana-dashboards - For visualization
  • slo-implementation - For SLO monitoring
  • distributed-tracing - For request tracing
开发运维

快捷安装

复制到终端去安装 Skill

npxskillsaddwshobson/agents--skill"prometheus-configuration"
站长:程序员鱼皮 老鱼简历 商务合作 增值电信业务经营许可证
2026 鱼皮AI导航 | 沪ICP备19026706号-6
站长:程序员鱼皮 老鱼简历 商务合作 增值电信业务经营许可证
2026 鱼皮AI导航 | 沪ICP备19026706号-6
🔥热门工具
朱雀AI检测朱雀AI检测
朱雀AI检测助手朱雀AI检测助手
LovartLovart
老鱼简历老鱼简历
WritePass官网WritePass官网
面多多面多多
CursorCursor
星月写作星月写作
朱雀AI检测助手朱雀AI检测助手
零感AI零感AI
Skills 推荐
retrieving-developer-knowledge
retrieving-developer-knowledge 用于搜索、获取和整合 Google 官方开发者文档,覆盖 Google Cloud、AI/Gemini、Android、Chrome、Web、Flutter、Go、Firebase 等平台,适用于查询 gcloud CLI 命令、API 语法、IAM 权限或架构对比等场景。
diff-intake
diff-intake 读取 PR 或分支差异,生成结构化 YAML 变更摘要,用于下游 analytics instrumentation 技能。适用于用户提交 PR 链接、分支对比或原始 diff 后需理解变更内容、评估分析覆盖或启动埋点流程的场景。
load-balancer-configurator
根据用户需求自动生成各类负载均衡器的生产级配置,覆盖 ALB、NLB、Nginx 和 HAProxy 等主流类型,适配不同协议、端口、健康检查及后端服务拓扑,支持直接集成到基础设施即代码流程中。
implement
实现 Linus Dashboard 的功能开发与集成,基于 TypeScript 和 Python 构建前后端组件,遵循严格的代码规范与架构模式,完成视图、卡片、服务及传感器等模块的创建与验证,确保高质量交付并兼容 Home Assistant 生态。
cloud-scheduler-job-creator
自动处理 Google Cloud Platform 中的云调度任务创建,提供符合最佳实践的配置方案与生产级代码生成,支持计算、存储、大数据及机器学习服务的集成部署。