How to configure Apache2 to produce JSON logs?

To log JSON format data in Apache2, you can customize the logging configuration in the Apache configuration file (httpd.conf or a site-specific configuration file like 000-default.conf in Ubuntu/Debian-based systems). Below is a step-by-step guide to set up JSON logging.

1. Enable mod_log_config Ensure the mod_log_config module is enabled, as it's required to customize log formats. This module should be enabled by default on most Apache installations.

2. Define the JSON Log Format In your Apache configuration file, define a custom log format in JSON. You can add this within the <VirtualHost> block for a specific site, or globally.

LogFormat "{ \
  \"time\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \
  \"remote_ip\": \"%a\", \
  \"host\": \"%v\", \
  \"method\": \"%m\", \
  \"url\": \"%U\", \
  \"query\": \"%q\", \
  \"protocol\": \"%H\", \
  \"status\": \"%>s\", \
  \"bytes_sent\": \"%B\", \
  \"referer\": \"%{Referer}i\", \
  \"user_agent\": \"%{User-Agent}i\", \
  \"response_time_microseconds\": \"%D\", \
  \"forwarded_for\": \"%{X-Forwarded-For}i\", \
  \"http_version\": \"%H\", \
  \"request\": \"%r\" \
}" json

3. Apply the Log Format To apply the log format, use the CustomLog directive.

CustomLog ${APACHE_LOG_DIR}/access.log json

4. Restart Apache After making changes to the configuration file, restart Apache to apply the changes.

sudo systemctl restart apache2

Apache2 VirtualHost with JSON logs

<VirtualHost *:80>
    ServerAdmin [email protected]
    ServerName example.com
    DocumentRoot /var/www/html

    # Define the JSON Log Format
    LogFormat "{ \
      \"time\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \
      \"remote_ip\": \"%a\", \
      \"host\": \"%v\", \
      \"method\": \"%m\", \
      \"url\": \"%U\", \
      \"query\": \"%q\", \
      \"protocol\": \"%H\", \
      \"status\": \"%>s\", \
      \"bytes_sent\": \"%B\", \
      \"referer\": \"%{Referer}i\", \
      \"user_agent\": \"%{User-Agent}i\", \
      \"response_time_microseconds\": \"%D\", \
      \"forwarded_for\": \"%{X-Forwarded-For}i\", \
      \"http_version\": \"%H\", \
      \"request\": \"%r\" \
    }" json

    # Apply the JSON Log Format to access logs
    CustomLog ${APACHE_LOG_DIR}/access.log json

    ErrorLog ${APACHE_LOG_DIR}/error.log
    LogLevel warn

    <Directory /var/www/html>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>
</VirtualHost>

Save size in Apache2 JSON logs

Since the JSON format in Apache2 logs adds a space overhead as the property names are repeated for each line. This can increase the size of the log files, to mitigate this, you can simply compress the names of the fields in a JSON object.

LogFormat "{ \
  \"t\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \         # Time
  \"ip\": \"%a\", \                             # Remote IP address
  \"h\": \"%v\", \                              # Hostname
  \"m\": \"%m\", \                              # HTTP method (GET, POST, etc.)
  \"u\": \"%U\", \                              # URL path requested
  \"q\": \"%q\", \                              # Query string
  \"p\": \"%H\", \                              # Protocol (HTTP/1.1, HTTP/2.0, etc.)
  \"s\": \"%>s\", \                             # Status code
  \"b\": \"%B\", \                              # Bytes sent
  \"r\": \"%{Referer}i\", \                     # Referer
  \"ua\": \"%{User-Agent}i\", \                 # User-Agent
  \"rt\": \"%D\", \                             # Response time in microseconds
  \"xff\": \"%{X-Forwarded-For}i\", \           # X-Forwarded-For header (if behind a proxy)
  \"hv\": \"%H\", \                             # HTTP version
  \"rq\": \"%r\" \                              # First line of the request
}" json

Stream Apache2 logs to Logdy

Logdy is a tool for consuming messy logs and outputting it in a nice Web UI, all with a single command.

$ tail -f /var/log/file.log | logdy
# Enter http://localhost:8080

Logdy UI

If you're interested, read more about Logdy or jump directly to Quick start. You can also see it in action: UI Demo.

On most Linux systems, Apache2 will store its logs in /var/log/apache2, however these locations can vary so make sure you're accessing a correct location. Run Logdy (assuming it's added to PATH)

$ tail -f /var/log/apache2/access.log | logdy