Logging

This file contains information about how logging is performed in torero as well as the standards and conventions used when writing logs in code. Logging functions are available inside of the internal/logger package.

Log File

Server

When torero is launched in server mode (torero server), the log file will be located at TORERO_LOG_SERVER_DIR/torero.log. If TORERO_LOG_SERVER_DIR is not specified, a default location of /var/log/torero/torero.log will be used.

Client

When the torero CLI client is used, the log file will be located at TORERO_APPLICATION_WORKING_DIR/torero.log. This location is used when users run CLI commands to accommodate torero CLI users that do not have /var/log access.

Console Logs

If you wish to have console logs displayed to the terminal when you run torero you can append --verbose to whichever command you are running. For example, running torero server --verbose will launch the server with console logs.

Format

By default, logs will be displayed in the format shown below: 2024-02-14T09:18:28-06:00 INF grpc server is starting up.

If you would like to have your logs displayed in JSON, you can set the environment variables TORERO_LOG_FILE_JSON=true or TORERO_LOG_CONSOLE_JSON=true. Some clients may prefer JSON so that they can more easily feed their logs into 3rd party software such as log aggregation programs.

JSON logs will resemble what is shown below: {"level":"info","time":"2024-02-14T09:18:28-06:00","message":"grpc server is starting up"}

By default, console logs will have some color formatting. To remove the color, please set the environment variable TORERO_TERMINAL_NO_COLOR=true.

Log File Rotations

We have two options for log file rotation and will have to choose which one to implement. 1. Encourage customers to use logrotate 2. Offer our own log rotation logic like IAP does

Logging Levels

torero supports many different logging levels to denote the importance of a log as well as the severity of an error. The log level that is used by torero will be set by an environment variable called TORERO_LOG_LEVEL. The default value if TORERO_LOG_LEVEL is unset is INFO. Both file and console logs will match the value set by TORERO_LOG_LEVEL.

Supported logging levels include - TRACE - DEBUG - INFO - WARN - ERROR - FATAL - DISABLED

torero will display logs for the level set in TORERO_LOG_LEVEL as well as any logs of higher importance. The list above is ordered from least important to most important.

For example, if INFO logs are selected, a user will be shown logs for INFO, WARN, ERROR, and FATAL. They will not see DEBUG and TRACE logs.

Logging Levels in Code Conventions

Trace

Used to describe which functions are being called within torero. All public functions will contain a trace log unless the function call is already being logged by higher log level. Private function trace logging is up to developer discresion.

Trace logs will describe the package and sub package if one exists, the receiver's type if one is present, and then the name of the function being called. Any important inputs being passed to the function should also be logged. Function inputs with sensitive data should not be logged.

package grpc


func (c LocalClient) CreateRepository(ctx context.Context, in *v1.Repository) (*v1.Repository, error) {
    logger.Trace()
    ...

Debug

Contain inputs that are sent to the various ‘systems and programs’ that torero calls.

Example uses include - Input to ssh-keyscan call - Inputs and name of Python script - Input and name of Ansible playbook - Database calls

See example below

logger.Debug("running command: `%s`", cmd.String())

Info

Info messages describe major actions that torero is taking but should be high level enough that they do not produce many messages and therefore spam.

Example uses include - Whenever an API is called (core package) - When configuration is being read/set on bootup

See example of core package logs below

func (h *Handler) CreateRepository(ctx context.Context, in *v1.Repository) (*v1.Repository, error) {
    logger.Info("CreateRepository called with input: %v", in)
    logger.Trace()

Warn

Used whenever torero observes something that is concerning but has not necessarily caused an error yet.

Error

An error log should be present every single time that there is error handling.

If an err variable is present, it should be passed as the first argument to logger.Error. Otherwise, nil can be passed. Error logs should be given a descriptive message in addition to the err variable to assist with later debugging.

For example

if err != nil {
    logger.Error(err, "The directory %s could not be created", directory)
    ...

Fatal

Used when we know that we want torero to terminate immediately. Fatal will usually be calledduring a bad startup. A call to logger.Fatal() will automatically terminate the torero process.

If an err variable is present, it should be passed as the first argument to logger.Fatal. Otherwise, nil can be passed. Fatal logs should be given a descriptive message in addition to the err variable to assist with later debugging.