Etcd Database

torero supports etcd as one of its database storage options. An etcd database is necessary if you plan on running torero in a distributed core/runner architecture. Etcd enables all of your torero instances access the same database and stay in sync. For more information on the other storage options that are available, please reference the overall store guide

Installing Etcd

torero requires etcd v3.5 which can be installed by following the guide provided on their website.

Hardware Specifications

Most torero installations will run smoothly with hardware specifications utilizing machines even smaller than the Small cluster example shown on etcd's website in the 'Example hardware configurations' section.

Clustering Etcd

For improved resiliency, several etcd servers can be configured in a cluster as outlined on etcd's clustering guide.

Configuring torero To Connect To Etcd

For torero to connect to etcd, you will first need to set the configuration variable of TORERO_STORE_BACKEND to etcd. For more information on changing torero's settings, please see the Config Variables documentation page.

With etcd enabled, you will also want to set TORERO_STORE_ETCD_HOSTS to the hostname of your etcd node. If you have multiple etcd nodes, you can enter a space separated list e.g. hostname1:port hostname2:port.

Etcd Security

It is highly recommended that torero connects to etcd with TLS by setting TORERO_STORE_ETCD_USE_TLS to true and configuring your etcd cluster to use TLS. Two different TLS authentication mechanisms are supported as is outlined below.

Client To Server Transport Security

When TLS is enabled, torero will use 'Client-to-server transport security' by default. This authentication method allows torero to verify the identity of the etcd server while providing transport security. Set TORERO_STORE_ETCD_CLIENT_CERT_AUTH to false to utilize this authentication method.

When using this TLS method, your torero instance will only require TORERO_STORE_ETCD_CA_CERTIFICATE_FILE to point to a CA certificate (ca.crt) and your etcd cluster will require a signed key pair (server.crt, server.key).

Once you have all of your certificates ready, you can launch your server with a command that is similar to what is shown below.

>_ etcd --name infra0 --data-dir infra0 \
  --cert-file=/path/to/server.crt --key-file=/path/to/server.key \
  --advertise-client-urls=https://127.0.0.1:2379 --listen-client-urls=https://127.0.0.1:2379

Now, ensure that your torero instance has the following configuration variables set

  • TORERO_STORE_BACKEND = etcd
  • TORERO_STORE_ETCD_HOSTS = host:port
  • TORERO_STORE_ETCD_USE_TLS = true
  • TORERO_STORE_ETCD_CLIENT_CERT_AUTH = false
  • TORERO_STORE_ETCD_CA_CERTIFICATE_FILE = /path/to/ca.crt

You can now run a command like torero get services to check if your connection to your new database is successful. A failure to connect may result in a timeout.

The method described here is very similar to the 'Example 1: Client-to-server transport security with HTTPS' section on etcd's security page.

Client To Server Authentication With Client Certificates

A more secure option for TLS authentication would be to use 'Client To Server Authentication With Client Certificates' where both the torero and the etcd server can verify each-other's identity while providing transport security. Set TORERO_STORE_ETCD_CLIENT_CERT_AUTH to true to utilize this authentication method.

When using this TLS method, your torero instance will require TORERO_STORE_ETCD_CA_CERTIFICATE_FILE to point to a CA certificate (ca.crt), TORERO_STORE_ETCD_CERTIFICATE_FILE to point to a public certificate file (client.crt), and TORERO_STORE_ETCD_PRIVATE_KEY_FILE to point to a private key (client.crt). Your etcd cluster will require the same CA certificate (ca.crt) and its own signed key pair (server.crt, server.key). This allows your client to verify that the server is to be trusted during the TLS handshake.

Once you have all of your certificates ready, you can launch your server with a command that is similar to what is shown below.

>_ etcd --name infra0 --data-dir infra0 \
  --client-cert-auth --trusted-ca-file=/path/to/ca.crt --cert-file=/path/to/server.crt --key-file=/path/to/server.key \
  --advertise-client-urls https://127.0.0.1:2379 --listen-client-urls https://127.0.0.1:2379

Now, ensure that your torero instance has the following configuration variables set

  • TORERO_STORE_BACKEND = etcd
  • TORERO_STORE_ETCD_HOSTS = host:port
  • TORERO_STORE_ETCD_USE_TLS = true
  • TORERO_STORE_ETCD_CLIENT_CERT_AUTH = true
  • TORERO_STORE_ETCD_CA_CERTIFICATE_FILE = /path/to/ca.crt
  • TORERO_STORE_ETCD_CERTIFICATE_FILE = /path/to/client.crt
  • TORERO_STORE_ETCD_PRIVATE_KEY_FILE = /path/to/client.key

You can now run a command like torero get services to check if your connection to your new database is successful. A failure to connect may result in a timeout.

The method described here is very similar to the 'Example 2: Client-to-server authentication with HTTPS client certificates' section on etcd's security page.