NGINX Virtual Machine Building with cloud-init

The joy and burden of building new servers

The initial build of a new application server requires multiple steps. First, the operating system must be installed. Then additional software packages need to be added as required by the application. Lastly, both the operating system and software package configurations need to be customized to the needs of the runtime environment.

Traditionally, building new servers was a manual process. A system administrator had a run book with all the steps required and would perform each task one by one. If the admin had multiple servers to build, the same steps were repeated over and over.

Looking at Installing NGINX Plus on Debian or Ubuntu as an example, an administrator needs to complete around 14 tasks. If these tasks must be performed manually every time a new VM is launched, much time can be wasted.

One way to reduce this burden is to build out one server, snapshot it, and then clone that snapshot for every additional server that needs to be deployed. All hypervisor platforms have this capability. The problem with snapshots, however, is that it captures the state of a server at a specific moment in time. If new versions of the operating system or software packages are released, your golden image becomes obsolete.

Rather than taking a picture of a server after it's been configured, another approach is to automate the execution of the configuration tasks themselves. Every time a new server is launched, the configuration is built from scratch, including the latest versions of the operating system and software packages. Now every new server arrives fresh instead of a clone of a configuration created months ago.

Introducing cloud-init

In this article, you will learn how to automate the process of building out a new NGINX Plus server using cloud-init.

The beauty of cloud-init is that all public cloud providers already include it as a built-in feature. You can even build local VMs on your own hardware or laptop. Just put your configuration tasks into a simple text file and it can be used to build new VMs anywhere cloud-init is available.

Automating NGINX Plus Installation

Let's take a look at how cloud-init can automate the process of installing NGINX Plus on a Ubuntu server. Below is the "simple text file" (formatted as YAML) that describes what tasks need to be performed. Notice that we are not just replicating each command-line operation into a shell script. This is much better than that. The contents are more declarative in nature, where we describe our desired final state rather than providing the imperative steps to get there. Now let's walk through each section of this file to see how it works.

#cloud-config

write_files:
    - content: |
        -----BEGIN CERTIFICATE-----
        MIIETDCCAzSgAwIBAgIRAN0W2fO6kAXvYOhjggkZ608wDQYJKoZIhvcNAQELBQAw
        Zj7Kh4dgUD+jOI5rJgn5DGm7inpiJf5eksH+scrOVTP+WuJHs8VzhH0UUqAwM3Tx
        <REDACTED>
        iIyDaR6Z9XvtFWiVyjfWv/R9hutbc7QLpbbcC/JIAmrUnskTlFu1Mz00p7Jis8f4
        7X4xxTENEFzwPfrXIF/bxpg+X75IRb1aYOXpadxuUrckd7Rb46P4FPPxVzEH7TA7
        -----END CERTIFICATE-----
      path: /etc/ssl/nginx/nginx-repo.crt
    - content: |
        -----BEGIN PRIVATE KEY-----
        MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCtKV0irhiXnx4U
        uz1v2neP9ew857iFpZ9FyrsvYqrwaRLdxsBkrU8CgYBRYEknMdfl8OhTcvyYFSMV
        <REDACTED>
        PjQ5U2Kaf3gMia5v4nGkhyKPuvasyvEeupwkIGDvRDmc+/AVtQQbJSJa7O0X1fwE
        aIDBc9S5hiWeqIOK8489IA==
        -----END PRIVATE KEY-----
      path: /etc/ssl/nginx/nginx-repo.key

apt:
  conf: |
    Acquire::https::pkgs.nginx.com::Verify-Peer "true";
    Acquire::https::pkgs.nginx.com::Verify-Host "true";
    Acquire::https::pkgs.nginx.com::SslCert     "/etc/ssl/nginx/nginx-repo.crt";
    Acquire::https::pkgs.nginx.com::SslKey      "/etc/ssl/nginx/nginx-repo.key";
   
  sources:
    nginx:
      source: "deb http://nginx.org/packages/mainline/ubuntu $RELEASE nginx"
      keyid: 2FD21310B49F6B46
    nginx-plus:
      source: "deb https://pkgs.nginx.com/plus/ubuntu $RELEASE nginx-plus"
      keyid: 2FD21310B49F6B46
    app-protect:
      source: "deb https://pkgs.nginx.com/app-protect/ubuntu $RELEASE nginx-plus"
      keyid: 2FD21310B49F6B46
    app-protect-dos:
      source: "deb https://pkgs.nginx.com/app-protect-dos/ubuntu $RELEASE nginx-plus"
      keyid: 2FD21310B49F6B46
    security-updates:
      source: "deb https://pkgs.nginx.com/app-protect-security-updates/ubuntu $RELEASE nginx-plus"
      keyid: A5F6473795E778F4
      keyserver: https://cs.nginx.com/static/keys/app-protect-security-updates.key

packages:
# - nginx
 - nginx-plus
# - nginx-plus-module-njs
# - nginx-ha-keepalived
# - app-protect

runcmd:
 - systemctl enable --now nginx

final_message: "The system is finally up, after $UPTIME seconds"

The write_files section (Lines 3-21)

The NGINX software repository requires mTLS client authentication. Two files are provided to customers to authenticate: nginx-repo.crt and nginx-repo.key. These two files need to be copied to the new server for software installation. The write_files section creates these files with the content included inline in the YAML file.  Notice that each line of content needs to be indented to produce valid YAML.  I use awk to indent each line like so:

awk '{$1="        "$1}1' nginx-repo.crt

Being able to create these files using cloud-init is nice because everything is in one YAML file. Of course, you will need to replace the content subsections with the content of your own nginx-repo.crt and nginx-repo.key files.

The apt section (Lines 23-46)

With Ubuntu Linux, the apt command is used to install software. To install NGINX Plus we configure apt with two things:

1. The conf subsection -- Tells apt where the mTLS digital certificate files are that we created in the previous section.
2. The sources subsection -- Adds the NGINX software repositories to apt. Each repository has a keyid that refers to a verification key in Ubuntu's keyserver. Notice that the "security-updates" keyid is not in Ubuntu's keyserver so a custom keyserver is provided.

The packages section (Lines 48-53)

Here is where we specify which software packages should be installed. In this case I only need NGINX Plus so I've commented out the others. In this section you can install any package available to Ubuntu that you need.

The runcmd section (Lines 55-56)

After install, we just need to run one command to start up NGINX and make sure it automatically starts when the server boots in the future.

The final_message section (Line 58)

The final_message is a string that is written to the /var/log/cloud-init-output.log file after the automated install. In this case, I use the $UPTIME variable to record how long it took for the installation process to complete.

Using a cloud-init file when creating a new VM

Now that we have a cloud-init file ready to go, let's go ahead and use it to configure a brand new VM. Here are some examples of providing your YAML file when launching a new VM instance:

On AWS EC2 using the Web UI

When using the Web UI to launch a new instance, click on Advanced Details at the bottom of the page. Scroll down until you see a section labeled, "User data - optional."  You can either paste your YAML into the box provided or click the "Choose file" to upload the file from your computer.

On Google Cloud Platform (GCP) using the gcloud CLI

In addition to a Web UI, cloud platforms also provide a command line interface for launching new VM instances. Here is an example using GCP's CLI:

​gcloud compute instances create mynginx --zone=us-central1-a --image-project=ubuntu-os-cloud --image-family=ubuntu-2204-lts --metadata-from-file user-data=plus.yaml

Here the metadata-from-file parameter is used to pass our YAML file named "plus.yaml"

On your laptop using multipass

Canonical, the makers of Ubuntu, provide a tool named Canonical Multipass for running VMs on your Windows, Mac, or Linux laptop or desktop system. When launching a new instance with multipass, you can pass our YAML file like so:

multipass launch -n mynginx --cloud-init plus.yaml

Viewing the cloud-init-output log file

While the cloud-init automated installation is in progress, output for each task is written to /var/log/cloud-init-output.log

You can view this file after the new instance is booted to verify that each step has completed successfully. Here is a snippet of the end of this log file:

Cloud-init v. 24.1.3-0ubuntu3 running 'modules:final' at Fri, 31 May 2024 02:24:41 +0000. Up 35.57 seconds.
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  nginx-plus
0 upgraded, 1 newly installed, 0 to remove and 51 not upgraded.
Need to get 3700 kB of archives.
After this operation, 7378 kB of additional disk space will be used.
Get:1 https://pkgs.nginx.com/plus/ubuntu noble/nginx-plus amd64 nginx-plus amd64 32-1~noble [3700 kB]
Fetched 3700 kB in 1s (2513 kB/s)
Selecting previously unselected package nginx-plus.
(Reading database ... 71839 files and directories currently installed.)
Preparing to unpack .../nginx-plus_32-1~noble_amd64.deb ...
----------------------------------------------------------------------

Thank you for using NGINX!

Please find the documentation for NGINX Plus here:
/usr/share/nginx/html/nginx-modules-reference.pdf

NGINX Plus is proprietary software. EULA and License information:
/usr/share/doc/nginx-plus/

For support information, please see:
https://www.nginx.com/support/

----------------------------------------------------------------------
Unpacking nginx-plus (32-1~noble) ...
Setting up nginx-plus (32-1~noble) ...
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
Processing triggers for man-db (2.12.0-4build2) ...

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
Synchronizing state of nginx.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable nginx
The system is finally up, after 44.67 seconds

By the final_message the automated install competed in 44.67 seconds. Much faster than if the administrator had to perform the same tasks manually!

Conclusion

There are other automation tools out there, like Ansible from Red Hat and Terraform from Hashicorp. These platforms come with a learning curve and require setup and configuration before they can be used.

Using cloud-init to automate the build-out of new NGINX Plus servers offers the following advantages:

1. Included with all public cloud compute offerings. Launching new VMs with cloud-init support comes by default.
2. A declarative interface simplifies describing the configuration tasks that need to be automated.
3. All instructions go into a single YAML text file that is easy to share across all your compute platforms.
4. Instead of using a stale VM golden image, cloud-init does a fresh install and config every time you spin up a new VM.

Hopefully this introduction to using cloud-init to build new NGINX Plus VM instances has been useful. Please feel free to leave any questions or feedback you may have in the comments below.