Install Prometheus on Ubuntu 16.04

Install Prometheus on Ubuntu 16.04

Prometheus

Prometheus could be a powerful and open-source, ASCII text file observance system that collects metrics from your services and stores them in an exceedingly time-series information.

It proposes a multi-dimensional data model, a versatile query language, and numerous visualization image prospects through tools like Grafana.

Prometheus solely exports metrics regarding itself (e.g. the number of requests it’s received, its memory consumption, etc.). But, you’ll greatly expand Prometheus by installing exporters, optional gratia programs that generate further metrics.

Exporters—both the official ones that the Prometheus team maintains further because the community-contributed ones—provide info regarding everything from infrastructure, databases, and web servers to electronic communication systems, APIs, and more.

Some of the foremost common selections include:

• node_exporter – This produces metrics regarding infrastructure, as well as the present CPU, memory, and disk usage, further as I/O and network statistics, like the quantity of bytes scan from a disk or a server’s average load.
• blackbox_exporter – This generates metrics derived from inquisitory protocols like protocol HTTP and HTTPS to see end convenience, time interval, and more.
• mysqld_exporter – This gathers metrics associated with a MySQL server, like the quantity of executed queries, average response time interval, and cluster replication standing.
• rabbitmq_exporter – This outputs metrics regarding the RabbitMQ electronic communication system, as well as the number of messages printed, the number of messages able to be delivered, and also the size of all the messages within the queue.
• nginx-vts-exporter – This provides metrics regarding associate Nginx web server victimization the Nginx VTS module, as well as the number of open connections, the quantity of sent responses (grouped by response codes), and also the total size of sent or received requests in bytes.

You will notice a lot of complete list of each official and community-contributed exporters on Prometheus’ web site.

You’ll install, configure, and secure Prometheus and Node Exporter to come up with metrics that may build it easier to observe your server’s performance.

Prerequisites

Before following this tutorial check that you have:

• One Ubuntu 16.04 Droplet, established by following the Initial Server Setup with Ubuntu sixteen.04 tutorial, as well as a sudo non-root user and a firewall.
• Nginx installed by following the primary 2 steps of the way to Install Nginx on Ubuntu 16.04 tutorial.

Step 1 — Create Service Users

For the safety functions, we’ll begin by making 2 new user accounts, prometheus and node_exporter. We’ll use these accounts throughout the tutorial to isolate the possession on Prometheus’ core files and directories.

Create these 2 users, and use the –no-create-home and –shell /bin/false choices in order that these users cannot log into the server.

[pastacode lang=”bash” manual=”%24%20sudo%20useradd%20–no-create-home%20–shell%20%2Fbin%2Ffalse%20prometheus%0A%24%20sudo%20useradd%20–no-create-home%20–shell%20%2Fbin%2Ffalse%20node_exporter” message=”” highlight=”” provider=”manual”/]

Before downloading the Prometheus binaries, create the essential directories to store Prometheus’ files and data.

The upcoming standard Linux conventions, we’ll generate a directory in /etc for Prometheus’ configuration files and a directory in /var/lib for its data.

[pastacode lang=”bash” manual=”%24%20sudo%20mkdir%20%2Fetc%2Fprometheus%0A%24%20sudo%20mkdir%20%2Fvar%2Flib%2Fprometheus” message=”” highlight=”” provider=”manual”/]

Set user and group ownership for new directories of prometheus user.

[pastacode lang=”bash” manual=”%24%20sudo%20chown%20prometheus%3Aprometheus%20%2Fetc%2Fprometheus%0A%24%20sudo%20chown%20prometheus%3Aprometheus%20%2Fvar%2Flib%2Fprometheus” message=”” highlight=”” provider=”manual”/]

With our users in place, we will currently download prometheus so produce the specified configuration file to run for 1st time.

Step 2 — Download Prometheus

Download and take the present stable version of Prometheus into your home directory. You’ll notice the newest binaries together with their checksums on the Prometheus transfer page.

[pastacode lang=”bash” manual=”%24%20cd%20~%0A%24%20curl%20-LO%20https%3A%2F%2Fgithub.com%2Fprometheus%2Fprometheus%2Freleases%2Fdownload%2Fv2.0.0%2Fprometheus-2.0.0.linux-amd64.tar.gz” message=”” highlight=”” provider=”manual”/]

Next, use the sha256sum command to come up with a checksum of the downloaded file:

[pastacode lang=”bash” manual=”%24%20sha256sum%20prometheus-2.0.0.linux-amd64.tar.gz” message=”” highlight=”” provider=”manual”/]

Compare the output from this command with the checksum on the Prometheus download page to confirm that your file is each real and not corrupted.

Output

[pastacode lang=”bash” manual=”e12917b25b32980daee0e9cf879d9ec197e2893924bd1574604eb0f550034d46%20%20prometheus-2.0.0.linux-amd64.tar.gz” message=”” highlight=”” provider=”manual”/]

If the checksums do not match, take away the downloaded file and repeat the preceding steps to re-download the file.

Now, take out the downloaded archive.

[pastacode lang=”bash” manual=”%24%20tar%20xvf%20prometheus-2.0.0.linux-amd64.tar.gz” message=”” highlight=”” provider=”manual”/]

This can produce a directory known as prometheus-1.7.1.linux-amd64 containing 2 binary files (prometheus and promtool), consoles, console_libraries, and several other example files.

Copy the 2 binaries to the /usr/local/bin directory.

[pastacode lang=”bash” manual=”%24%20sudo%20cp%20prometheus-2.0.0.linux-amd64%2Fprometheus%20%2Fusr%2Flocal%2Fbin%2F%0A%24%20sudo%20cp%20prometheus-2.0.0.linux-amd64%2Fpromtool%20%2Fusr%2Flocal%2Fbin%2F” message=”” highlight=”” provider=”manual”/]

Set the user and cluster possession on the binaries to the Prometheus user created in Step 1.

[pastacode lang=”bash” manual=”%24%20sudo%20chown%20prometheus%3Aprometheus%20%2Fusr%2Flocal%2Fbin%2Fprometheus%0A%24%20sudo%20chown%20prometheus%3Aprometheus%20%2Fusr%2Flocal%2Fbin%2Fpromtool” message=”” highlight=”” provider=”manual”/]

Copy the consoles and console_libraries directories to /etc/prometheus.

[pastacode lang=”bash” manual=”%24%20sudo%20cp%20-r%20prometheus-2.0.0.linux-amd64%2Fconsoles%20%2Fetc%2Fprometheus%0A%24%20sudo%20cp%20-r%20prometheus-2.0.0.linux-amd64%2Fconsole_libraries%20%2Fetc%2Fprometheus” message=”” highlight=”” provider=”manual”/]

Set the user and cluster ownership on the directories to the prometheus user. Using the -R flag will confirm that ownership is set on the files inside the directory as well.

[pastacode lang=”bash” manual=”%24%20sudo%20chown%20-R%20prometheus%3Aprometheus%20%2Fetc%2Fprometheus%2Fconsoles%0A%24%20sudo%20chown%20-R%20prometheus%3Aprometheus%20%2Fetc%2Fprometheus%2Fconsole_libraries” message=”” highlight=”” provider=”manual”/]

Lastly, take away the leftover files from your home directory as they’re not required.

[pastacode lang=”bash” manual=”%24%20rm%20-rf%20prometheus-2.0.0.linux-amd64.tar.gz%20prometheus-2.0.0.linux-amd64″ message=”” highlight=”” provider=”manual”/]

Now Prometheus is installed, we’ll produce its configuration and service files in preparation of its initial run.

Step 3 — Configure Prometheus

In the /etc/prometheus directory, use nano or your favorite text editor to make a configuration file named Prometheus.yml. For now, this file can contain barely enough info to run Prometheus for the primary time.

[pastacode lang=”bash” manual=”%24%20sudo%20nano%20%2Fetc%2Fprometheus%2Fprometheus.yml” message=”” highlight=”” provider=”manual”/]

Warning: Prometheus’ configuration file uses the YAML format, that strictly forbids tabs and needs 2 areas for indentation. Prometheus can fail to start out if the configuration file is incorrectly formatted.

In the global settings, outline the default interval for scraping metrics. Note that Prometheus can apply these settings to each exporter unless a personal exporter’s own settings override the globals.

[pastacode lang=”haml” manual=”global%3A%0A%20%20scrape_interval%3A%2015s” message=”Prometheus config file part 1 – /etc/prometheus/prometheus.yml” highlight=”” provider=”manual”/]

This scrape_interval value tells Prometheus to gather metrics from its exporters every fifteen seconds, that is long enough for many exporters.

Now, add Prometheus itself to the list of exporters to scrape from with the subsequent scrape_configs directive:

[pastacode lang=”haml” manual=”…%0Ascrape_configs%3A%0A%20%20-%20job_name%3A%20’prometheus’%0A%20%20%20%20scrape_interval%3A%205s%0A%20%20%20%20static_configs%3A%0A%20%20%20%20%20%20-%20targets%3A%20%5B’localhost%3A9090’%5D” message=”Prometheus config file part 2 – /etc/prometheus/prometheus.yml” highlight=”” provider=”manual”/]

Prometheus uses the job_name to label exporters in queries and on graphs, therefore make certain to choose one thing descriptive here.

And, as Prometheus exports necessary information regarding itself that you simply will use for observance performance and debugging, we’ve overridden the global scrape_interval directive from 15 seconds to 5 seconds for a lot of frequent updates.

Lastly, Prometheus uses the static_configs and targets directives to see wherever exporters square measure running. Since this explicit exporter is running on an identical server as Prometheus itself, we will use localhost rather than IP address together with the default port, 9090.

Your configuration file will to currently seem like this:

[pastacode lang=”bash” manual=”global%3A%0A%20%20scrape_interval%3A%2015s%0A%0Ascrape_configs%3A%0A%20%20-%20job_name%3A%20’prometheus’%0A%20%20%20%20scrape_interval%3A%205s%0A%20%20%20%20static_configs%3A%0A%20%20%20%20%20%20-%20targets%3A%20%5B’localhost%3A9090’%5D” message=”Prometheus config file – /etc/prometheus/prometheus.yml” highlight=”” provider=”manual”/]

Save the file and exit your text editor.

Now, set the user and cluster possession on the configuration file to the Prometheus user created in Step 1.

[pastacode lang=”bash” manual=”%24%20sudo%20chown%20prometheus%3Aprometheus%20%2Fetc%2Fprometheus%2Fprometheus.yml” message=”” highlight=”” provider=”manual”/]

With the configuration complete, we’re able to take a look at prometheus by running it for the primary time.

Step 4 — Running Prometheus

Startup Prometheus because the Prometheus user, providing the trail to each the configuration file and also the data directory.

[pastacode lang=”bash” manual=”%24%20sudo%20-u%20prometheus%20%2Fusr%2Flocal%2Fbin%2Fprometheus%20%5C%0A%24%20%20%20%20–config.file%20%2Fetc%2Fprometheus%2Fprometheus.yml%20%5C%0A%24%20%20%20–storage.tsdb.path%20%2Fvar%2Flib%2Fprometheus%2F%20%5C%0A%24%20%20%20%20–web.console.templates%3D%2Fetc%2Fprometheus%2Fconsoles%20%5C%0A%24%20%20%20%20–web.console.libraries%3D%2Fetc%2Fprometheus%2Fconsole_libraries” message=”” highlight=”” provider=”manual”/]

The output contains info regarding Prometheus’ loading progress, configuration file, and connected services. It conjointly confirms that Prometheus is listening on port 9090.

Output

level=info ts=2017-11-17T18:37:27.474530094Z caller=main.go:215 msg="Starting Prometheus" version="(version=2.0.0, branch=HEAD, re
vision=0a74f98628a0463dddc90528220c94de5032d1a0)"
level=info ts=2017-11-17T18:37:27.474758404Z caller=main.go:216 build_context="(go=go1.9.2, user=root@615b82cb36b6, date=20171108-
07:11:59)"
level=info ts=2017-11-17T18:37:27.474883982Z caller=main.go:217 host_details="(Linux 4.4.0-98-generic #121-Ubuntu SMP Tue Oct 10 1
4:24:03 UTC 2017 x86_64 prometheus-update (none))"
level=info ts=2017-11-17T18:37:27.483661837Z caller=web.go:380 component=web msg="Start listening for connections" address=0.0.0.0
:9090
level=info ts=2017-11-17T18:37:27.489730138Z caller=main.go:314 msg="Starting TSDB"
level=info ts=2017-11-17T18:37:27.516050288Z caller=targetmanager.go:71 component="target manager" msg="Starting target manager...
"
level=info ts=2017-11-17T18:37:27.537629169Z caller=main.go:326 msg="TSDB started"
level=info ts=2017-11-17T18:37:27.537896721Z caller=main.go:394 msg="Loading configuration file" filename=/etc/prometheus/promethe
us.yml
level=info ts=2017-11-17T18:37:27.53890004Z caller=main.go:371 msg="Server is ready to receive requests."

If an error message comes, ensure that you’ve got used YAML syntax in your configuration file so follow the on-screen directions to resolve the matter.

Now, halt Prometheus by pressing CTRL+C, so open a brand new systemd service file.

[pastacode lang=”bash” manual=”%24%20sudo%20nano%20%2Fetc%2Fsystemd%2Fsystem%2Fprometheus.service” message=”” highlight=”” provider=”manual”/]

The service file tells systemd to run Prometheus because the Prometheus user, with the configuration file situated within the /etc/prometheus/prometheus.yml directory and to store its information within the /var/lib/prometheus directory. (The details of systemd service files square measure on the far side the scope of this tutorial)

Copy the subsequent content into the file:

[pastacode lang=”bash” manual=”%5BUnit%5D%0ADescription%3DPrometheus%0AWants%3Dnetwork-online.target%0AAfter%3Dnetwork-online.target%0A%0A%5BService%5D%0AUser%3Dprometheus%0AGroup%3Dprometheus%0AType%3Dsimple%0AExecStart%3D%2Fusr%2Flocal%2Fbin%2Fprometheus%20%5C%0A%20%20%20%20–config.file%20%2Fetc%2Fprometheus%2Fprometheus.yml%20%5C%0A%20%20%20%20–storage.tsdb.path%20%2Fvar%2Flib%2Fprometheus%2F%20%5C%0A%20%20%20%20–web.console.templates%3D%2Fetc%2Fprometheus%2Fconsoles%20%5C%0A%20%20%20%20–web.console.libraries%3D%2Fetc%2Fprometheus%2Fconsole_libraries%0A%0A%5BInstall%5D%0AWantedBy%3Dmulti-user.target” message=”Prometheus service file – /etc/systemd/system/prometheus.service” highlight=”” provider=”manual”/]

Finally, save the file and shut your text editor.

To use the recently created service, reload systemd.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20daemon-reload” message=”” highlight=”” provider=”manual”/]

You will currently begin Prometheus victimization the subsequent command:

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20start%20prometheus” message=”” highlight=”” provider=”manual”/]

To check that Prometheus is running, check the service’s status.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20status%20prometheus” message=”” highlight=”” provider=”manual”/]

The output tells you Prometheus’ status, main method symbol (PID), memory use, and more.

If the service’s status is not active, follow the on-screen directions and re-trace the preceding steps to resolve the matter before continued the tutorial.

Output

● prometheus.service - Prometheus
   Loaded: loaded (/etc/systemd/system/prometheus.service; disabled; vendor preset: enabled)
   Active: active (running) since Fri 2017-07-21 11:40:40 UTC; 3s ago
 Main PID: 2104 (prometheus)
    Tasks: 7
   Memory: 13.8M
      CPU: 470ms
   CGroup: /system.slice/prometheus.service
...

When you are able to move, press Q to quit the status command.

Lastly, change the service to start on boot.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20enable%20prometheus” message=”” highlight=”” provider=”manual”/]

Now that Prometheus is up and running, we will install an extra exporter to come up with metrics regarding our server’s resources.

Step 5 — Downloading Node Exporter

To expand Prometheus on the far side metrics regarding itself solely, we’ll install an extra exporter known as Node exporter. Node exporter provides careful info regarding the system, as well as CPU, disk, and memory usage.

First, download the present stable version of Node Exporter into your home directory. You’ll notice the newest binaries together with their checksums on Prometheus’ download page.

[pastacode lang=”bash” manual=”%24%20cd%20~%0A%24%20curl%20-LO%20https%3A%2F%2Fgithub.com%2Fprometheus%2Fnode_exporter%2Freleases%2Fdownload%2Fv0.15.1%2Fnode_exporter-0.15.1.linux-amd64.tar.gz” message=”” highlight=”” provider=”manual”/]

Use the sha256sum command to create a checksum of the downloaded file:

[pastacode lang=”bash” manual=”%24%20sha256sum%20node_exporter-0.15.1.linux-amd64.tar.gz” message=”” highlight=”” provider=”manual”/]

Now, verify the downloaded file’s integrity by comparing its checksum with the one on the download page.

Output

7ffb3773abb71dd2b2119c5f6a7a0dbca0cff34b24b2ced9e01d9897df61a127  node_exporter-0.15.1.linux-amd64.tar.gz

If the checksums do not match, take away the downloaded file and repeat the preceding steps.

Now, unload downloaded archive.

[pastacode lang=”bash” manual=”%24%20tar%20xvf%20node_exporter-0.15.1.linux-amd64.tar.gz” message=”” highlight=”” provider=”manual”/]

This can produce a directory known as node_exporter-0.14.0.linux-amd64 containing a computer file named node_exporter, a license, and a notice.

Copy the binary to the /usr/local/bin directory and set the user and cluster possession to the node_exporter user that you simply created in Step 1.

[pastacode lang=”markup” manual=”%24%20sudo%20cp%20node_exporter-0.15.1.linux-amd64%2Fnode_exporter%20%2Fusr%2Flocal%2Fbin%0A%24%20sudo%20chown%20node_exporter%3Anode_exporter%20%2Fusr%2Flocal%2Fbin%2Fnode_exporter” message=”” highlight=”” provider=”manual”/]

Lastly, take away the leftover files from your home directory as they’re not required.

[pastacode lang=”bash” manual=”%24%20rm%20-rf%20node_exporter-0.15.1.linux-amd64.tar.gz%20node_exporter-0.15.1.linux-amd64″ message=”” highlight=”” provider=”manual”/]

Now that you’ve got put in Node Exporter, you’ll take a look at it out by running it so we will produce a service file for it in order that it starts on boot.

Step 6 — Running Node Exporter

The steps for running Node Exporter is the same as those for running Prometheus itself. Begin by making the Systemd service file for Node Exporter.

[pastacode lang=”bash” manual=”%24%20sudo%20nano%20%2Fetc%2Fsystemd%2Fsystem%2Fnode_exporter.service” message=”” highlight=”” provider=”manual”/]

This service file tells your system to run Node Exporter because the node_exporter user with the default set of collectors enabled.

Copy the subsequent content into the service file:

[pastacode lang=”bash” manual=”%5BUnit%5D%0ADescription%3DNode%20Exporter%0AWants%3Dnetwork-online.target%0AAfter%3Dnetwork-online.target%0A%0A%5BService%5D%0AUser%3Dnode_exporter%0AGroup%3Dnode_exporter%0AType%3Dsimple%0AExecStart%3D%2Fusr%2Flocal%2Fbin%2Fnode_exporter%0A%0A%5BInstall%5D%0AWantedBy%3Dmulti-user.target” message=”Node Exporter service file – /etc/systemd/system/node_exporter.service” highlight=”” provider=”manual”/]

Collectors outline that metrics Node Exporter can generate. You’ll see Node Exporter’s complete list of collectors—including enabled by default and that deprecated—in the Node Exporter README file.

If you ever have to be compelled to override the default list of collectors, you’ll use the –collectors.enabled flag, like:

[pastacode lang=”bash” manual=”…%0AExecStart%3D%2Fusr%2Flocal%2Fbin%2Fnode_exporter%20–collectors.enabled%20meminfo%2Cloadavg%2Cfilesystem%0A…” message=”Node Exporter service file part – /etc/systemd/system/node_exporter.service” highlight=”” provider=”manual”/]

The preceding example would tell Node Exporter to create metrics victimization solely the meminfo, loadavg, and filesystem collectors. You’ll limit the collectors to but few or several you would like, however, note that there are not any blank areas before or when the commas.

Save the file and shut your text editor.

Finally, reload systemd to use the recently created service.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20daemon-reload” message=”” highlight=”” provider=”manual”/]

At the end of the scrape_configs block, add a brand new entry known as node_exporter.

[pastacode lang=”bash” manual=”…%0A%20%20-%20job_name%3A%20’node_exporter’%0A%20%20%20%20scrape_interval%3A%205s%0A%20%20%20%20static_configs%3A%0A%20%20%20%20%20%20-%20targets%3A%20%5B’localhost%3A9100’%5D” message=”Prometheus config file part 1 – /etc/prometheus/prometheus.yml” highlight=”” provider=”manual”/]

Because this Exporter is additionally running on an identical server as Prometheus itself. We will use localhost rather than IP address once more together with Node Exporter’s default port, 9100.

Your whole configuration file ought to seem like this:

[pastacode lang=”bash” manual=”global%3A%0A%20%20scrape_interval%3A%2015s%0A%0Ascrape_configs%3A%0A%20%20-%20job_name%3A%20’prometheus’%0A%20%20%20%20scrape_interval%3A%205s%0A%20%20%20%20static_configs%3A%0A%20%20%20%20%20%20-%20targets%3A%20%5B’localhost%3A9090’%5D%0A%20%20-%20job_name%3A%20’node_exporter’%0A%20%20%20%20scrape_interval%3A%205s%0A%20%20%20%20static_configs%3A%0A%20%20%20%20%20%20-%20targets%3A%20%5B’localhost%3A9100’%5D%20%20%20″ message=”Prometheus config file – /etc/prometheus/prometheus.yml” highlight=”” provider=”manual”/]

Save the file and exit your text editor once you are able to continue.

Finally, restart Prometheus to place the changes into impact.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20restart%20prometheus” message=”” highlight=”” provider=”manual”/]

Once again, verify that everything is running properly with the status command.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20status%20prometheus” message=”” highlight=”” provider=”manual”/]

If the service’s status is not set to active, follow the on-screen directions and re-trace your previous steps before moving on.

Output

prometheus.service - Prometheus
   Loaded: loaded (/etc/systemd/system/prometheus.service; disabled; vendor preset: enabled)
   Active: active (running) since Fri 2017-07-21 11:46:39 UTC; 6s ago
 Main PID: 2219 (prometheus)
    Tasks: 6
   Memory: 19.9M
      CPU: 433ms
   CGroup: /system.slice/prometheus.service

We currently have Prometheus and Node Exporter installed, configured, and running. As a final precaution before connecting to the online interface, we’ll enhance our installation’s security with basic protocol HTTP authentication to confirm that unauthorized users cannot access our metrics.

Step 8 — Securing Prometheus

Prometheus doesn’t embrace in-built authentication or the other general purpose security mechanism. On the one hand, this suggests you are obtaining an extremely versatile system with fewer configuration restraints; on the opposite hand, it means that it’s up to you to confirm that your metrics and overall setup square measure sufficiently secure.

For simplicity’s sake, we’ll use Nginx to feature basic HTTP protocol authentication to our installation, that each Prometheus and its most popular data visualization tool, Grafana, totally support.

Start by putting in apache2-utils, which is able to offer you access to the htpasswd utility for generating password files.

[pastacode lang=”bash” manual=”%24%20sudo%20apt-get%20update%0A%24%20sudo%20apt-get%20install%20apache2-utils” message=”” highlight=”” provider=”manual”/]

Now, generate a password file by telling htpasswd wherever you wish to store the file and that username you want to use for authentication.

Note: htpasswd can prompt you to enter and re-confirm the password you want to go together with this user. Also, build note of each the username and password you enter here, as you will need them to log into Prometheus in Step 9.

The results of this command could be a newly-created file known as .htpasswd, situated within the /etc/nginx directory, containing the username and a hashed version of the password you entered.

Next, set up Nginx to use the newly-created passwords.

First, build a Prometheus-specific copy of the default Nginx configuration file in order that you’ll revert back to the defaults later if you run into a haul.

[pastacode lang=”bash” manual=”%24%20sudo%20cp%20%2Fetc%2Fnginx%2Fsites-available%2Fdefault%20%2Fetc%2Fnginx%2Fsites-available%2Fprometheus” message=”” highlight=”” provider=”manual”/]

Then, open the new configuration file.

[pastacode lang=”bash” manual=”%24%20sudo%20nano%20%2Fetc%2Fnginx%2Fsites-available%2Fprometheus” message=”” highlight=”” provider=”manual”/]

Locate the location/ block below the server block. It must look like:

[pastacode lang=”bash” manual=”…%0A%20%20%20%20location%20%2F%20%7B%0A%20%20%20%20%20%20%20%20try_files%20%24uri%20%24uri%2F%20%3D404%3B%0A%20%20%20%20%7D%0A…%0A” message=”/etc/nginx/sites-available/default” highlight=”” provider=”manual”/]

As we are going to be forwarding all traffic to Prometheus, replace the try_files directive with the subsequent content:

[pastacode lang=”bash” manual=”…%0A%20%20%20%20location%20%2F%20%7B%0A%20%20%20%20%20%20%20%20auth_basic%20%22Prometheus%20server%20authentication%22%3B%0A%20%20%20%20%20%20%20%20auth_basic_user_file%20%2Fetc%2Fnginx%2F.htpasswd%3B%0A%20%20%20%20%20%20%20%20proxy_pass%20http%3A%2F%2Flocalhost%3A9090%3B%0A%20%20%20%20%20%20%20%20proxy_http_version%201.1%3B%0A%20%20%20%20%20%20%20%20proxy_set_header%20Upgrade%20%24http_upgrade%3B%0A%20%20%20%20%20%20%20%20proxy_set_header%20Connection%20’upgrade’%3B%0A%20%20%20%20%20%20%20%20proxy_set_header%20Host%20%24host%3B%0A%20%20%20%20%20%20%20%20proxy_cache_bypass%20%24http_upgrade%3B%0A%20%20%20%20%7D%0A…” message=”/etc/nginx/sites-available/prometheus” highlight=”” provider=”manual”/]

These settings make sure that users can demonstrate at the beginning of every new session. In addition, the reverse proxy can direct all requests handled by this block to Prometheus.

When you are finished creating changes, save the file and shut your text editor.

Now, deactivate the default Nginx configuration file by removing the link thereto within the /etc/nginx/sites-enabled directory, and activate the new configuration file by generating a link to it.

[pastacode lang=”bash” manual=”%24%20sudo%20rm%20%2Fetc%2Fnginx%2Fsites-enabled%2Fdefault%0A%24%20sudo%20ln%20-s%20%2Fetc%2Fnginx%2Fsites-available%2Fprometheus%20%2Fetc%2Fnginx%2Fsites-enabled%2F” message=”” highlight=”” provider=”manual”/]

Before restarting Nginx, check the configuration for errors victimization the subsequent command:

[pastacode lang=”bash” manual=”%24%20sudo%20nginx%20-t” message=”” highlight=”” provider=”manual”/]

The output must indicate that the syntax is ok and also the test is successful. If you receive an error message, follow the on-screen directions to repair the matter before continuing to the following step.

An output of Nginx configuration tests

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Then, reload Nginx to include all of the changes.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20reload%20nginx” message=”” highlight=”” provider=”manual”/]

Verify that Nginx is up and running.

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20status%20nginx” message=”” highlight=”” provider=”manual”/]

If your output does not indicate that the service’s status is active, follow the on-screen messages and re-trace the preceding steps to resolve the difficulty before continued.

[pastacode lang=”bash” manual=”%E2%97%8F%20nginx.service%20-%20A%20high%20performance%20web%20server%20and%20a%20reverse%20proxy%20server%0A%20%20%20Loaded%3A%20loaded%20(%2Flib%2Fsystemd%2Fsystem%2Fnginx.service%3B%20enabled%3B%20vendor%20preset%3A%20en%0A%20%20%20Active%3A%20active%20(running)%20since%20Mon%202017-07-31%2021%3A20%3A57%20UTC%3B%2012min%20ago%0A%20%20Process%3A%204302%20ExecReload%3D%2Fusr%2Fsbin%2Fnginx%20-g%20daemon%20on%3B%20master_process%20on%3B%20-s%20r%0A%20Main%20PID%3A%203053%20(nginx)%0A%20%20%20%20Tasks%3A%202%0A%20%20%20Memory%3A%203.6M%0A%20%20%20%20%20%20CPU%3A%2056ms%0A%20%20%20CGroup%3A%20%2Fsystem.slice%2Fnginx.service” message=”Output” highlight=”” provider=”manual”/]

At this time, we have a fully-functional and secured Prometheus server. Therefore we will log into the web interface to start observing metrics.

Step 9 — Testing Prometheus

Prometheus provides a basic web interface for observance the status of itself and its exporters, queries, and generating graphs. But, because of the interface’s simplicity, the Prometheus team recommends to install and use Grafana for all the difficult than testing and debugging.

In this tutorial, we’ll use the inbuilt web interface to confirm that Prometheus and Node Exporter are up and running, and we’ll conjointly take a glance at straightforward queries and graphs.

To begin, purpose your web browser to http://your_server_ip.

In the HTTP protocol authentication dialogue box, enter the username and password you selected in Step 8.

Once logged in, you will see the Expression Browser, wherever you’ll execute and visualize custom queries.

Before performing any expressions, verify the status of each Prometheus and Node Explorer by clicking initial on the status menu at the highest of the screen so on the Targets menu possibility. As we’ve organized Prometheus to scrape each itself and Node Exporter, you must see each target listed within the UP state.

If either Exporter is missing or displays an error message, check the service’s status with the subsequent commands:

[pastacode lang=”bash” manual=”%24%20sudo%20systemctl%20status%20prometheus%0A%24%20sudo%20systemctl%20status%20node_exporter” message=”” highlight=”” provider=”manual”/]

The output for each service must report a status of Active: active (running). If a service either is not active in the least or is active however still not operating properly. Follow the on-screen directions and re-trace the previous steps before continuing.

Next, to form certain that the exporters are operating properly, we’ll execute some expressions against Node Exporter.

First, click on the Graph menu at the highest of the screen to come back to the Expression Browser.

In the Expression field, type node_memory_MemAvailable and press the Execute button to update the Console tab with the quantity of memory your server has.

By default, Node Exporter reports this quantity in bytes. To convert to megabytes, we’ll use math operators to divide by 1024 double time.

In the Expression field, enter node_memory_MemAvailable/1024/1024 so press the Execute button.

The Console tab can currently show the ends up in megabytes.

If you wish to verify the results, execute the free command from your terminal. (The -h flag tells free to report back in an exceedingly human-readable format, giving a quantity in megabytes.)

[pastacode lang=”bash” manual=”%24%20free%20-h” message=”” highlight=”” provider=”manual”/]

This output contains details regarding memory usage, as well as on the available memory displayed within the on the available column.

Output

              total        used        free      shared  buff/cache   available
Mem:           488M        144M         17M        3.7M        326M        324M
Swap:            0B          0B          0B

In addition to basic operators, the Prometheus query language conjointly provides several functions for aggregating results.

In the Expression field, type avg_over_time(node_memory_MemAvailable[5m])/1024/1024 and click on on the Execute button. The result is going to be common on the available memory over the last five minutes in megabytes.

Now, click on the Graph tab to show the executed expression as a graph rather than as text.

 

Finally, whereas still on this tab, hover your mouse over the graph for extra details regarding any specific purpose on the graph’s X and Y axes.

If you want to find out a lot of regarding making expressions in Prometheus’ intrinsic web interface, see the Querying Prometheus portion of the official documentation.

Conclusion

• In this tutorial we have a tendency to downloaded, configured, secured, and tested a whole Prometheus installation with one further Exporter.
• If you want to find out a lot of regarding, however, Prometheus works below the hood, take a glance at How to Query Prometheus on Ubuntu fourteen.04. (Since you have already got Prometheus put in, you’ll skip the primary step.)
• To see what else Prometheus will do, visit the official Prometheus documentation.
• And, to learn more about extending Prometheus, check out the list of available exporters as well as the official Grafana website.