English | 简体中文 | 繁體中文 | Русский язык | Français | Español | Português | Deutsch | 日本語 | 한국어 | Italiano | بالعربية

Detailed Explanation of Nginx Log Configuration and Log Splitting

Log configuration

Logs are very beneficial for statistics and troubleshooting. This article summarizes the nginx log-related configurations such as access_log, log_format, open_log_file_cache, log_not_found, log_subrequest, rewrite_log, error_log.

nginx has a very flexible log recording mode. Each level of configuration can have its own access log. The log format is defined by the log_format command.

ngx_http_log_module is used to define the request log format.

1. access_log directive

Syntax:

access_log path [format [buffer=size [flush=time]]];
access_log path format gzip[=level] [buffer=size] [flush=time];
access_log syslog:server=address[,parameter=value] [format];
access_log off;

Default value: access_log logs/access.log combined;

Configuration section: http, server, location, if in location, limit_except

gzip compression level.

buffer sets the size of the memory cache area.

flush keeps the longest time saved in the cache area.

Do not log: access_log off;

Log using the default combined format: access_log logs/access.log or access_log logs/access.log combined;

2. log_format instruction

Syntax: log_format name string …;

Default value: log_format combined "…"

Configuration section: http

name represents the format name, string represents the equivalent format. log_format has a default combined log format that does not need to be set, equivalent to the combined log format of Apache, as shown below:

log_format combined '$remote_addr - $remote_user [$time_local] "
                  " "$request" $status $body_bytes_sent "
                  " "$http_referer" "$http_user_agent" "" 

If nginx is located after the load balancer, squid, and nginx reverse proxy, the web server cannot directly obtain the real IP address of the client. $remote_addr gets the IP address of the reverse proxy. The reverse proxy server can add X in the http header information of the forwarded request.-Forwarded-For information, used to record the client IP address and the server address requested by the client.

log_format porxy '$http_x_forwarded_for - $remote_user [$time_local] "
               " "$request" $status $body_bytes_sent "
               " "$http_referer" "$http_user_agent" ""

The comments on the variables that can be included in the log format are as follows:

$remote_addr, $http_x_forwarded_for (reverse) Records the client IP address
$remote_user Records the client user name
$request Records the URL and HTTP protocol of the request
$status Records the request status
$body_bytes_sent The number of bytes sent to the client, excluding the size of the response header; This variable is compatible with the "%B" parameter in the Apache module mod_log_config.
$bytes_sent The total number of bytes sent to the client.
$connection The serial number of the connection.
$connection_requests The number of requests obtained through a connection.
$msec The time of log writing. The unit is second, and the precision is millisecond.
$pipe If the request is sent through HTTP pipelined (pipelined), the pipe value is "p", otherwise it is "."
$http_referer records the page from which the access is linked
$http_user_agent Records information about the client browser
$request_length Length of the request (including the request line, request headers, and request body).
$request_time Request processing time, in seconds, with millisecond precision; from the first byte read from the client to the log write after the last character is sent to the client.
$time_iso8601 ISO8601Local time in the Standard Log Format.
$time_local Local time in the Common Log Format.

[warning] The response headers sent to the client have the prefix 'sent_http_'. For example, $sent_http_content_range. [/warning]

Example as follows:

http {
 log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '"$status" $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for" '
                    '"$gzip_ratio" $request_time $bytes_sent $request_length';
 log_format srcache_log '$remote_addr - $remote_user [$time_local] "$request" '
                '"$status" $body_bytes_sent $request_time $bytes_sent $request_length '
                '[$upstream_response_time] [$srcache_fetch_status] [$srcache_store_status] [$srcache_expire]';
 open_log_file_cache max=1000 inactive=60s;
 server {
 server_name ~^(www\.)?(.+)$;
 access_log logs/$2-access.log main;
 error_log logs/$2-error.log;
 location /srcache {
 access_log logs/access-srcache.log srcache_log;
 }
 }
}

3. open_log_file_cache directive

Syntax:

open_log_file_cache max=N [inactive=time] [min_uses=N] [valid=time];
open_log_file_cache off;

Default: open_log_file_cache off;

Configuration sections: http, server, location

For each log entry, the file will be opened first, then the log will be written, and then closed. open_log_file_cache can be used to set the log file cache (default is off), in the format as follows:

Parameter comments as follows:

  • max: Set the maximum number of file descriptors in the cache. If the cache is full, the LRU algorithm will be used to close the descriptors.
  • inactive: Set the survival time, default is10s
  •  min_uses: Set the minimum number of times the log file must be used within the inactive period before the log file descriptor is cached, default is1times
  • valid: Set the check frequency, default60s
  •  off: Disable caching

 Example as follows:

open_log_file_cache max=1000 inactive=20s valid=1m min_uses=2;

4. log_not_found directive

Syntax: log_not_found on | off;

Default: log_not_found on;

Configuration sections: http, server, location

Whether to record non-existent errors in the error_log. Default is yes.

5. log_subrequest directive

Syntax: log_subrequest on | off;

Default: log_subrequest off;

Configuration sections: http, server, location

Whether to record the access logs of sub-requests in the access_log. Default is not recorded.

6. rewrite_log directive

Provided by ngx_http_rewrite_module module. Used to record rewrite logs. It is recommended to enable it for debugging rewrite rules. Nginx Rewrite Rule Guide

Syntax: rewrite_log on | off;

Default: rewrite_log off;

Configuration sections: http, server, location, if

When enabled, notice level rewrite logs will be recorded in the error log.

7. error_log directive

Syntax: error_log file | stderr | syslog:server=address[,parameter=value] [debug | info | notice | warn | error | crit | alert | emerg];

Default: error_log logs/error.log error;

Configuration section: main, http, server, location

Configure error logs.

--------------------------------------------------------------------------------

Log cutting

By default, nginx logs are all written into one file, which becomes very large and inconvenient to view and analyze. It is a good practice to cut logs by date, usually we do statistics on a daily basis. Let's talk about nginx log cutting below.

1. Define log rotation strategy

# vim nginx-log-rotate
/data/weblogs/*.log {
  nocompress
  daily
  copytruncate
  create
  notifempty
  rotate 7
  olddir /data/weblogs/old_log
  missingok
  dateext
  postrotate
    /bin/kill -HUP `cat /var/run/nginx.pid 2> /dev/null` 2> /dev/null || true
  endscript
}

[warning]/data/weblogs/*.log when using wildcards,/data/weblogs/All matching log files under the directory will be cut. If you want to cut a specific log file, specify the file. [/warning]

2. Set up a scheduled task

59 23 * * * root ( /usr/sbin/logrotate -f /PATH/TO/nginx-log-rotate)

such that every day23point59minute execution log cutting.

That's all for this article. I hope it will be helpful to everyone's learning and that everyone will support the Yell Tutorial more.

Declaration: The content of this article is from the Internet, and the copyright belongs to the original author. The content is contributed and uploaded by Internet users spontaneously. This website does not own the copyright, has not been manually edited, and does not assume any relevant legal responsibility. If you find any content suspected of copyright infringement, please send an email to: notice#oldtoolbag.com (Please replace # with @ when sending an email to report, and provide relevant evidence. Once verified, this site will immediately delete the infringing content.)

You May Also Like