NGINX Prometheus Exporter for NGINX and NGINX Plus
1b6ce493
3.2 MB
5 months ago
38.3M
Readme

NGINX Prometheus Exporter

NGINX Prometheus exporter makes it possible to monitor NGINX or NGINX Plus using Prometheus.

Overview

NGINX exposes a handful of metrics via the stub_status page. NGINX Plus provides a richer set of metrics via the API and the monitoring dashboard. NGINX Prometheus exporter fetches the metrics from a single NGINX or NGINX Plus, converts the metrics into appropriate Prometheus metrics types and finally exposes them via an HTTP server to be collected by Prometheus.

Getting Started

In this section, we show how to quickly run NGINX Prometheus Exporter for NGINX or NGINX Plus.

A Note about NGINX Ingress Controller

If you’d like to use the NGINX Prometheus Exporter with NGINX Ingress Controller for Kubernetes, see this doc for the installation instructions.

Prerequisites

We assume that you have already installed Prometheus and NGINX or NGINX Plus. Additionally, you need to:

  • Expose the built-in metrics in NGINX/NGINX Plus:
    • For NGINX, expose the stub_status page at /stub_status on port 8080.
    • For NGINX Plus, expose the API at /api on port 8080.
  • Configure Prometheus to scrape metrics from the server with the exporter. Note that the default scrape port of the exporter is 9113 and the default metrics path -- /metrics.

Running the Exporter in a Docker Container

To start the exporter we use the docker run command.

  • To export NGINX metrics, run:

    $ docker run -p 9113:9113 nginx/nginx-prometheus-exporter:0.10.0 -nginx.scrape-uri=http://<nginx>:8080/stub_status
    

    where <nginx> is the IP address/DNS name, through which NGINX is available.

  • To export NGINX Plus metrics, run:

    $ docker run -p 9113:9113 nginx/nginx-prometheus-exporter:0.10.0 -nginx.plus -nginx.scrape-uri=http://<nginx-plus>:8080/api
    

    where <nginx-plus> is the IP address/DNS name, through which NGINX Plus is available.

Running the Exporter Binary

  • To export NGINX metrics, run:

    $ nginx-prometheus-exporter -nginx.scrape-uri=http://<nginx>:8080/stub_status
    

    where <nginx> is the IP address/DNS name, through which NGINX is available.

  • To export NGINX Plus metrics:

    $ nginx-prometheus-exporter -nginx.plus -nginx.scrape-uri=http://<nginx-plus>:8080/api
    

    where <nginx-plus> is the IP address/DNS name, through which NGINX Plus is available.

  • To export and scrape NGINX metrics with unix domain sockets, run:

    $ nginx-prometheus-exporter -nginx.scrape-uri=unix:<nginx>:/stub_status -web.listen-address=unix:/path/to/socket.sock
    

    where <nginx> is the path to unix domain socket, through which NGINX stub status is available.

Note. The nginx-prometheus-exporter is not a daemon. To run the exporter as a system service (daemon), configure the init system of your Linux server (such as systemd or Upstart) accordingly. Alternatively, you can run the exporter in a Docker container.

Usage

Command-line Arguments

Usage of ./nginx-prometheus-exporter:
  -nginx.plus
        Start the exporter for NGINX Plus. By default, the exporter is started for NGINX. The default value can be overwritten by NGINX_PLUS environment variable.
  -nginx.retries int
        A number of retries the exporter will make on start to connect to the NGINX stub_status page/NGINX Plus API before exiting with an error. The default value can be overwritten by NGINX_RETRIES environment variable.
  -nginx.retry-interval duration
        An interval between retries to connect to the NGINX stub_status page/NGINX Plus API on start. The default value can be overwritten by NGINX_RETRY_INTERVAL environment variable. (default 5s)
  -nginx.scrape-uri string
        A URI or unix domain socket path for scraping NGINX or NGINX Plus metrics.
        For NGINX, the stub_status page must be available through the URI. For NGINX Plus -- the API. The default value can be overwritten by SCRAPE_URI environment variable. (default "http://127.0.0.1:8080/stub_status")
  -nginx.ssl-ca-cert string
        Path to the PEM encoded CA certificate file used to validate the servers SSL certificate. The default value can be overwritten by SSL_CA_CERT environment variable.
  -nginx.ssl-client-cert string
        Path to the PEM encoded client certificate file to use when connecting to the server. The default value can be overwritten by SSL_CLIENT_CERT environment variable.
  -nginx.ssl-client-key string
        Path to the PEM encoded client certificate key file to use when connecting to the server. The default value can be overwritten by SSL_CLIENT_KEY environment variable.
  -nginx.ssl-verify
        Perform SSL certificate verification. The default value can be overwritten by SSL_VERIFY environment variable. (default true)
  -nginx.timeout duration
        A timeout for scraping metrics from NGINX or NGINX Plus. The default value can be overwritten by TIMEOUT environment variable. (default 5s)
  -prometheus.const-labels value
        A comma separated list of constant labels that will be used in every metric. Format is label1=value1,label2=value2... The default value can be overwritten by CONST_LABELS environment variable.
  -web.listen-address string
        An address or unix domain socket path to listen on for web interface and telemetry. The default value can be overwritten by LISTEN_ADDRESS environment variable. (default ":9113")
  -web.telemetry-path string
        A path under which to expose metrics. The default value can be overwritten by TELEMETRY_PATH environment variable. (default "/metrics")
  -web.secured-metrics
        Expose metrics using https. The default value can be overwritten by SECURED_METRICS variable.  (default false)
  -web.ssl-server-cert string
        Path to the PEM encoded certificate for the nginx-exporter metrics server(when web.secured-metrics=true). The default value can be overwritten by SSL_SERVER_CERT variable.
  -web.ssl-server-key string
        Path to the PEM encoded key for the nginx-exporter metrics server (when web.secured-metrics=true). The default value can be overwritten by SSL_SERVER_KEY variable.
  -version
        Display the NGINX exporter version. (default false)

Exported Metrics

Common metrics:

NameTypeDescriptionLabels
nginxexporter_build_infoGaugeShows the exporter build information.gitCommit, version
nginx_upGaugeShows the status of the last metric scrape: 1 for a successful scrape and 0 for a failed one[]

Metrics for NGINX OSS:

Stub status metrics

NameTypeDescriptionLabels
nginx_connections_acceptedCounterAccepted client connections.[]
nginx_connections_activeGaugeActive client connections.[]
nginx_connections_handledCounterHandled client connections.[]
nginx_connections_readingGaugeConnections where NGINX is reading the request header.[]
nginx_connections_waitingGaugeIdle client connections.[]
nginx_connections_writingGaugeConnections where NGINX is writing the response back to the client.[]
nginx_http_requests_totalCounterTotal http requests.[]

Metrics for NGINX Plus:

Connections

NameTypeDescriptionLabels
nginxplus_connections_acceptedCounterAccepted client connections[]
nginxplus_connections_activeGaugeActive client connections[]
nginxplus_connections_droppedCounterDropped client connections dropped[]
nginxplus_connections_idleGaugeIdle client connections[]

HTTP

NameTypeDescriptionLabels
nginxplus_http_requests_totalCounterTotal http requests[]
nginxplus_http_requests_currentGaugeCurrent http requests[]

SSL

NameTypeDescriptionLabels
nginxplus_ssl_handshakesCounterSuccessful SSL handshakes[]
nginxplus_ssl_handshakes_failedCounterFailed SSL handshakes[]
nginxplus_ssl_session_reusesCounterSession reuses during SSL handshake[]

HTTP Server Zones

NameTypeDescriptionLabels
nginxplus_server_zone_processingGaugeClient requests that are currently being processedserver_zone
nginxplus_server_zone_requestsCounterTotal client requestsserver_zone
nginxplus_server_zone_responsesCounterTotal responses sent to clientscode (the response status code. The values are: 1xx, 2xx, 3xx, 4xx and 5xx), server_zone
nginxplus_server_zone_discardedCounterRequests completed without sending a responseserver_zone
nginxplus_server_zone_receivedCounterBytes received from clientsserver_zone
nginxplus_server_zone_sentCounterBytes sent to clientsserver_zone

Stream Server Zones

NameTypeDescriptionLabels
nginxplus_stream_server_zone_processingGaugeClient connections that are currently being processedserver_zone
nginxplus_stream_server_zone_connectionsCounterTotal connectionsserver_zone
nginxplus_stream_server_zone_sessionsCounterTotal sessions completedcode (the response status code. The values are: 2xx, 4xx, and 5xx), server_zone
nginxplus_stream_server_zone_discardedCounterConnections completed without creating a sessionserver_zone
nginxplus_stream_server_zone_receivedCounterBytes received from clientsserver_zone
nginxplus_stream_server_zone_sentCounterBytes sent to clientsserver_zone

HTTP Upstreams

Note: for the state metric, the string values are converted to float64 using the following rule: "up" -> 1.0, "draining" -> 2.0, "down" -> 3.0, "unavail" –> 4.0, "checking" –> 5.0, "unhealthy" -> 6.0.

NameTypeDescriptionLabels
nginxplus_upstream_server_stateGaugeCurrent stateserver, upstream
nginxplus_upstream_server_activeGaugeActive connectionsserver, upstream
nginxplus_upstream_server_limitGaugeLimit for connections which corresponds to the max_conns parameter of the upstream server. Zero value means there is no limitserver, upstream
nginxplus_upstream_server_requestsCounterTotal client requestsserver, upstream
nginxplus_upstream_server_responsesCounterTotal responses sent to clientscode (the response status code. The values are: 1xx, 2xx, 3xx, 4xx and 5xx), server, upstream
nginxplus_upstream_server_sentCounterBytes sent to this serverserver, upstream
nginxplus_upstream_server_receivedCounterBytes received to this serverserver, upstream
nginxplus_upstream_server_failsCounterNumber of unsuccessful attempts to communicate with the serverserver, upstream
nginxplus_upstream_server_unavailCounterHow many times the server became unavailable for client requests (state 'unavail') due to the number of unsuccessful attempts reaching the max_fails thresholdserver, upstream
nginxplus_upstream_server_header_timeGaugeAverage time to get the response header from the serverserver, upstream
nginxplus_upstream_server_response_timeGaugeAverage time to get the full response from the serverserver, upstream
nginxplus_upstream_server_health_checks_checksCounterTotal health check requestsserver, upstream
nginxplus_upstream_server_health_checks_failsCounterFailed health checksserver, upstream
nginxplus_upstream_server_health_checks_unhealthyCounterHow many times the server became unhealthy (state 'unhealthy')server, upstream
nginxplus_upstream_keepalivesGaugeIdle keepalive connectionsupstream
nginxplus_upstream_zombiesGaugeServers removed from the group but still processing active client requestsupstream

Stream Upstreams

Note: for the state metric, the string values are converted to float64 using the following rule: "up" -> 1.0, "down" -> 3.0, "unavail" –> 4.0, "checking" –> 5.0, "unhealthy" -> 6.0.

NameTypeDescriptionLabels
nginxplus_stream_upstream_server_stateGaugeCurrent stateserver, upstream
nginxplus_stream_upstream_server_activeGaugeActive connectionsserver , upstream
nginxplus_stream_upstream_server_limitGaugeLimit for connections which corresponds to the max_conns parameter of the upstream server. Zero value means there is no limitserver , upstream
nginxplus_stream_upstream_server_connectionsCounterTotal number of client connections forwarded to this serverserver, upstream
nginxplus_stream_upstream_server_connect_timeGaugeAverage time to connect to the upstream serverserver, upstream
nginxplus_stream_upstream_server_first_byte_timeGaugeAverage time to receive the first byte of dataserver, upstream
nginxplus_stream_upstream_server_response_timeGaugeAverage time to receive the last byte of dataserver, upstream
nginxplus_stream_upstream_server_sentCounterBytes sent to this serverserver, upstream
nginxplus_stream_upstream_server_receivedCounterBytes received from this serverserver, upstream
nginxplus_stream_upstream_server_failsCounterNumber of unsuccessful attempts to communicate with the serverserver, upstream
nginxplus_stream_upstream_server_unavailCounterHow many times the server became unavailable for client connections (state 'unavail') due to the number of unsuccessful attempts reaching the max_fails thresholdserver, upstream
nginxplus_stream_upstream_server_health_checks_checksCounterTotal health check requestsserver, upstream
nginxplus_stream_upstream_server_health_checks_failsCounterFailed health checksserver, upstream
nginxplus_stream_upstream_server_health_checks_unhealthyCounterHow many times the server became unhealthy (state 'unhealthy')server, upstream
nginxplus_stream_upstream_zombiesGaugeServers removed from the group but still processing active client connectionsupstream

Stream Zone Sync

NameTypeDescriptionLabels
nginxplus_stream_zone_sync_zone_records_pendingGaugeThe number of records that need to be sent to the clusterzone
nginxplus_stream_zone_sync_zone_records_totalGaugeThe total number of records stored in the shared memory zonezone
nginxplus_stream_zone_sync_zone_bytes_inCounterBytes received by this node[]
nginxplus_stream_zone_sync_zone_bytes_outCounterBytes sent by this node[]
nginxplus_stream_zone_sync_zone_msgs_inCounterTotal messages received by this node[]
nginxplus_stream_zone_sync_zone_msgs_outCounterTotal messages sent by this node[]
nginxplus_stream_zone_sync_zone_nodes_onlineGaugeNumber of peers this node is connected to[]

Location Zones

NameTypeDescriptionLabels
nginxplus_location_zone_requestsCounterTotal client requestslocation_zone
nginxplus_location_zone_responsesCounterTotal responses sent to clientscode (the response status code. The values are: 1xx, 2xx, 3xx, 4xx and 5xx), location_zone
nginxplus_location_zone_discardedCounterRequests completed without sending a responselocation_zone
nginxplus_location_zone_receivedCounterBytes received from clientslocation_zone
nginxplus_location_zone_sentCounterBytes sent to clientslocation_zone

Resolver

NameTypeDescriptionLabels
nginxplus_resolver_nameCounterTotal requests to resolve names to addressesresolver
nginxplus_resolver_srvCounterTotal requests to resolve SRV recordsresolver
nginxplus_resolver_addrCounterTotal requests to resolve addresses to namesresolver
nginxplus_resolver_noerrorCounterTotal number of successful responsesresolver
nginxplus_resolver_formerrCounterTotal number of FORMERR responsesresolver
nginxplus_resolver_servfailCounterTotal number of SERVFAIL responsesresolver
nginxplus_resolver_nxdomainCounterTotal number of NXDOMAIN responsesresolver
nginxplus_resolver_notimpCounterTotal number of NOTIMP responsesresolver
nginxplus_resolver_refusedCounterTotal number of REFUSED responsesresolver
nginxplus_resolver_timedoutCounterTotal number of timed out requestresolver
nginxplus_resolver_unknownCounterTotal requests completed with an unknown errorresolver

Connect to the /metrics page of the running exporter to see the complete list of metrics along with their descriptions. Note: to see server zones related metrics you must configure status zones and to see upstream related metrics you must configure upstreams with a shared memory zone.

Troubleshooting

The exporter logs errors to the standard output. When using Docker, if the exporter doesn’t work as expected, check its logs using docker logs command.

Releases

Docker images

We publish the Docker image on DockerHub, GitHub Container and Amazon ECR Public Gallery.

As an alternative, you can choose the edge version built from the latest commit from the master branch. The edge version is useful for experimenting with new features that are not yet published in a stable release.

Binaries

We publish the binaries for multiple Operating Systems and architectures on the GitHub releases page.

Homebrew

You can add the NGINX homebrew tap with

$ brew tap nginxinc/tap

and then install the formula with

$ brew install nginx-prometheus-exporter

Building the Exporter

You can build the exporter using the provided Makefile. Before building the exporter, make sure the following software is installed on your machine:

  • make
  • git
  • Docker for building the container image
  • Go for building the binary

Building the Docker Image

To build the Docker image with the exporter, run:

$ make container

Note: go is not required, as the exporter binary is built in a Docker container. See the Dockerfile.

Building the Binary

To build the binary, run:

$ make

Note: the binary is built for the OS/arch of your machine. To build binaries for other platforms, see the Makefile.

The binary is built with the name nginx-prometheus-exporter.

Grafana Dashboard

The official Grafana dashboard is provided with the exporter for NGINX. Check the Grafana Dashboard documentation for more information.

Support

The commercial support is available for NGINX Plus customers when the NGINX Prometheus Exporter is used with NGINX Ingress Controller.

License

Apache License, Version 2.0.