Deploy Elasticsearch with basic authentication using Ansible

I believe there's no need to say anything about Elasticsearch. It's a well-known search engine developed in Java that provides full-text search.

If you need an Elasticsearch instance for your project and you don't want to manually install it or pay for a managed service, then you might want to consider deploying it with Ansible in less than 5 minutes.

OK. Let's be completely honest, it might take more than 5 minutes if you don't know anything about Ansible but the deployment process itself takes just about 5 minutes.

I won't cover details about installing Ansible on your local machine. You can find all you need in the official Ansible docs, but if you are using Ubuntu you can install it like this:

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible

The next step is to define hosts. Create hosts file:

vim /etc/ansible/hosts

And specify your first host:

[servers]
elastic-001 ansible_host=IP_ADDRESS ansible_port=PORT_NUMBER ansible_python_interpreter=/usr/bin/python3

In our case, we have only one host called elastic-001 with a specified IP address and port number. Change IP_ADDRESS value to the IP address of your server and PORT_NUMBER can be any random port number above 40000.

We don't want to use the default SSH port for our server, so we are setting a random value. That will reduce the risk of attacks by hackers and bots.

You can get your server anywhere you want as long as you have full SSH access. So anything like DigitalOcean, Linode, or really cheap and good Hetzner will suffice. Just make sure that you have at least 4GB of memory available.

Step-by-step instructions

1. Use git to clone the following repository:

https://github.com/gnikolovski/ansible-playbook-elasticsearch

2. Change directory to ansible-playbook-elasticsearch, open a terminal and install necessary contributed roles:

ansible-galaxy install geerlingguy.htpasswd
ansible-galaxy install geerlingguy.security
ansible-galaxy install geerlingguy.firewall
ansible-galaxy install elastic.elasticsearch
ansible-galaxy install geerlingguy.nginx

3. Open the vars/elastic.yml file and update variables. You probably want to change everything in the gnikolovski.system and gnikolovski.apps sections. You'll need the server_user_password later on when executing subsequent runs.

In the geerlingguy.htpasswd section update the name and password. Those are credentials for basic authentication.

In the geerlingguy.security section update the security_ssh_port value. This will replace the default SSH port. You'll need this value later on when executing subsequent runs. That's our PORT_NUMBER value.

In the elastic.elasticsearch section update the Elasticsearch version to any available 6.x or 7.x version.

And finally, inside the geerlingguy.nginx section update the server_name value. That's the URL of your Elasticsearch instance.

4. Now you can run the Ansible playbook that will configure the server and deploy Elasticsearch:

ansible-playbook playbook.yml \
  --limit elastic-001 \
  --extra-vars "@vars/elastic.yml" \
  --extra-vars "ansible_user=root ansible_ssh_port=22 ansible_python_interpreter=/usr/bin/python3"

Playbook execution recap should look something like this: 

5. If you change something in your variables file (vars/elastic.yml), and you have to run the playbook again use the following command:

ansible-playbook playbook.yml \
  --limit elastic-001 \
  --extra-vars "@vars/elastic.yml" \
  --extra-vars "ansible_user=VALUE_OF_server_user ansible_ssh_port=VALUE_OF_security_ssh_port ansible_python_interpreter=/usr/bin/python3" \
  --ask-become-pass

As you can see we are using the default SSH port only during the first run. For all subsequent runs, we must specify our custom SSH port.

Server configuration

Apart from using contributed roles like geerlingguy's roles and elastic.elasticsearch role, we have our own custom roles for configuring servers.

The gnikolovski.apps role will install apps like Vim, Midnight Commander, Git, Zip, and the gnikolovski.system role will do stuff like creating a new user and setting up necessary directories.

The geerlingguy.firewall role will set up the firewall, which will block direct access to Elasticsearch's default port. We don't want to expose port 9200, but instead, we want to use Nginx as a reverse proxy. This will also allow us to add basic authentication.

Always make sure that your instance is secured at least with basic authentication because Elasticsearch doesn’t ship with built-in authentication. Putting Nginx as a reverse proxy in front of Elasticsearch is probably the easiest way to do that.

… and that’s it! Now go on and deploy some Elasticsearch instances.