NGINX with Prometheus Exporter
- NGINX Metric 에 대해 Prometheus 에서 모니터링 가능하게 함
- nginx-prometheus-exporter 활용하여 구성
- Official URL : https://github.com/nginxinc/nginx-prometheus-exporter
- NGINX Prometheus-NJS 3rd Modules 활용하여 구성
- NGINX OSS vs NGINX Plus 차이점
- NGINX OSS : stub_status 기준 모니터링 가능
- NGINX Plus : 100+ Metric 기준 모니터링 가능
사전 설정
Prometheus 구성 *
- Official URL : https://prometheus.io/docs/prometheus/latest/installation/
- Scrape config 설정 필요 ( 모니터링 대상 NGINX 추가 )
sample Config
# my global config global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
# - alertmanager:9093
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first_rules.yml"
# - "second_rules.yml"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: "prometheus"
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ["localhost:9090"]
- targets: ['10.250.11.99:9113']
Grafana Deploy
- Dashboard 활용 시 사전 Deploy 필요
Go Install (Build)
Build 시, go 설치 필요
## 설치 참고 URL : <https://go.dev/doc/install>
wget <https://go.dev/dl/go1.19.1.linux-amd64.tar.gz>
## 설치
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.19.1.linux-amd64.tar.gz
## Profile 수정
cat <<EOF >> /etc/profile
export PATH=$PATH:/usr/local/go/bin
EOF
## Profile 수정 적용
source /etc/profile
## Go 설치확인
go version
go version go1.19.1 linux/amd64
NGINX OSS
stub_status On
Sample Code
# /etc/nginx/conf.d/http-8080.conf
server {
listen 8080;
location /stub_status { ## OSS Monitoring URI
stub_status on;
}
}
NGINX Plus
API Access On
Sample Code
# /etc/nginx/conf.d/http-8080.conf
server {
listen 8080;
location /api { ## Plus API Access URI
api write=on;
allow 0.0.0.0/0; ## API ACL IP List
access_log /dev/null;
}
}
NGINX - Prometheus-NJS 활용
- NGINX Plus에서 직접 Prometheus 메트릭 엔드포인트를 노출
- Prometheus Exporter 를 활용하지 않고, 3rd Module로 Prometheus 연동
3rd Modules Install
Code
yum search nginx-plus-module-prometheus --show-duplicate
## NGINX plus R25 버전 기준
yum install nginx-plus-module-njs-r25
yum install nginx-plus-module-prometheus-25+1.3.3-1.el7.ngx.noarchInstall 결과 (LOG)
Installing : nginx-plus-module-njs-25+0.7.3-1.el7.ngx.x86_64 1/1
----------------------------------------------------------------------
The njs dynamic modules for NGINX Plus have been installed.
To enable these modules, add the following to /etc/nginx/nginx.conf
and reload nginx:
load_module modules/ngx_http_js_module.so;
load_module modules/ngx_stream_js_module.so;
Please refer to the modules documentation for further details:
<https://nginx.org/en/docs/njs/>
<https://nginx.org/en/docs/http/ngx_http_js_module.html>
<https://nginx.org/en/docs/stream/ngx_stream_js_module.html>
----------------------------------------------------------------------
Verifying : nginx-plus-module-njs-25+0.7.3-1.el7.ngx.x86_64 1/1
Installed:
nginx-plus-module-njs.x86_64 0:25+0.7.3-1.el7.ngx
Complete!
============================================================================
Installing : nginx-plus-module-prometheus-25+1.3.3-1.el7.ngx.noarch 1/1
----------------------------------------------------------------------
The Prometheus exporter NJS module for NGINX Plus has been installed.
To enable this module, add the following to /etc/nginx/nginx.conf:
load_module modules/ngx_http_js_module.so;
load_module modules/ngx_stream_js_module.so;
then configure nginx in a way similar with the provided example here:
/usr/share/doc/nginx-plus-module-prometheus/examples/nginx.conf
and reload nginx.
Metrics suitable for Prometheus will be available at the "/metrics"
location (by default).
Please refer to the module documentation for further details:
<https://docs.nginx.com/nginx/admin-guide/dynamic-modules/prometheus-njs/>
----------------------------------------------------------------------
Verifying : nginx-plus-module-prometheus-25+1.3.3-1.el7.ngx.noarch 1/1
Installed:
nginx-plus-module-prometheus.noarch 0:25+1.3.3-1.el7.ngx
Complete!
NGINX Config 설정
Sample Config (nginx.conf)
load_module modules/ngx_http_js_module.so;
load_module modules/ngx_stream_js_module.so;Sample Config (temp.conf)
js_import /usr/share/nginx-plus-module-prometheus/prometheus.js;
server {
listen 9000;
location = /metrics {
set $prom_keyval "http_upstream_keyval";
js_content prometheus.metrics;
}
}
Prometheus 연동 확인
- Config
- prometheus.yml 에 NGINX - Prometheus-NJS Module 설치 IP 기입
- Prometheus 확인
NGINX - Prometheus Exporter 구성 (Build)
Prometheus Exporter Download
Code
wget [<https://github.com/nginxinc/nginx-prometheus-exporter/archive/refs/heads/main.zip>](<https://github.com/nginxinc/nginx-prometheus-exporter/archive/refs/heads/main.zip>)
cd /root/nginx-prometheus-exporter-main/
Prometheus Exporter Build
Code
make
Prometheus Exporter Execute
Code
# NGINX Plus
./nginx-prometheus-exporter -nginx.plus -nginx.scrape-uri=http://127.0.0.1:8080/api &
# NGINX OSS
./nginx-prometheus-exporter -nginx.scrape-uri=http://127.0.0.1:8080/stub_status &
NGINX - Prometheus Exporter 구성 (Docker) *
- Official Docker Image : https://hub.docker.com/r/nginx/nginx-prometheus-exporter
Prometheus Exporter Image Pull or Build
Image Pull or Build
Code - Image Build
wget <https://github.com/nginxinc/nginx-prometheus-exporter/archive/refs/heads/main.zip>
unzip main.zip
cd nginx-prometheus-exporter
make containerCode - Docker Image Run
# NGINX Plus with Image Pull
docker run -p 9113:9113 nginx/nginx-prometheus-exporter:0.10.0 -nginx.plus -nginx.scrape-uri=http://<nginx-plus>:8080/api
# NGINX Plus with Local Image
docker run -p 9113:9113 adf2bedcc588 -nginx.plus -nginx.scrape-uri=http://10.250.11.99:8080/api
# OSS with Image Pull
$ docker run -p 9113:9113 nginx/nginx-prometheus-exporter:0.10.0 -nginx.scrape-uri=http://<nginx>:8080/stub_status
모니터링
Prometheus 에서 모니터링
Web 에서 모니터링 수행
CAP
Grafana Dashboard 연동
- Dashbard 를 통한 모니터링
Grafana Dashboard Source update
Prometheus Data 를 Grafana 에 연결해줘야함 ( https://prometheus.io/docs/visualization/grafana/#using )
CAP
GUI >> Configuration - Data Sources - Add Data Source
URL : 그라파나 IP ( 127.0.0.1 X )
Grafana Dashboard Create
GUI DashBoard 생성
Dashboard Source : https://github.com/nginxinc/nginx-prometheus-exporter/blob/main/grafana/dashboard.json
Grafana Dasnboard Create
Grafana Dashboard Update
상세 Metric 에 대해서 설정 확인 필요
Grafana Dashboard -설정 완료 화면 (Sample)
각NGINX Plus Instance에 대해 모니터링수행 가능
ETC
- OS Service 등록
Process 실행 시, shell logout 시, 모니터링 불가
code
#vi /etc/systemd/system/nginx-prometheus.service
[Unit]
Description=NGINX with Prometheus Exporter
[Service]
Type=simple
ExecStart=/home/cco/nginx-prometheus-exporter-main/nginx-prometheus-exporter -nginx.plus -nginx.scrape-uri=http://127.0.0.1:8080/api &
Restart=on-failure
[Install]
WantedBy=multi-user.targetRun Command
# systemctl start nginx-prometheus
# systemctl status nginx-prometheus
● nginx-prometheus.service - NGINX with Prometheus Exporter
Loaded: loaded (/etc/systemd/system/nginx-prometheus.service; disabled; vendor preset: disabled)
Active: active (running) since 수 2023-04-05 14:46:47 KST; 8s ago
Main PID: 15162 (nginx-prometheu)
CGroup: /system.slice/nginx-prometheus.service
└─15162 /home/cco/nginx-prometheus-exporter-main/nginx-prometheus-exporter -nginx.plus -n...
4월 05 14:46:47 nginx.master.cco systemd[1]: Started NGINX with Prometheus Exporter.
4월 05 14:46:47 nginx.master.cco nginx-prometheus-exporter[15162]: NGINX Prometheus Exporter vers...1
4월 05 14:46:47 nginx.master.cco nginx-prometheus-exporter[15162]: 2023/04/05 14:46:47 Starting...
4월 05 14:46:47 nginx.master.cco nginx-prometheus-exporter[15162]: 2023/04/05 14:46:47 Listening ...3
4월 05 14:46:47 nginx.master.cco nginx-prometheus-exporter[15162]: 2023/04/05 14:46:47 NGINX Prom...d
Hint: Some lines were ellipsized, use -l to show in full.