9.4

Edit

HTTP request analysis

Sample log format:

$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"

The fields mean the following:

  • $remote_addr: the IP from which the request was made.
  • $remote_user: the HTTP authenticated user. For most apps, the field is empty as modern apps do not use HTTP-based authentication.
  • [$time_local]: the timestamp based on the server timezone.
  • “$request”: the HTTP request type GET, POST, and so on, plus the requested path without arguments and the HTTP protocol version.
  • $status: the HTTP code for the response from the server.
  • $body_bytes_sent: the size of the server response in bytes.
  • “$http_referer”: the referral URL (if any).
  • “$http_user_agent”: the user agent as seen by the server.

Sort access by response codes

cat /var/log/nginx/access.log | cut -d '"' -f3 | cut -d ' ' -f2 | sort | uniq -c | sort -rn
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

Get most visited 404 pages

awk '($9 ~ /404/)' /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn

Get 502 (bad-gateway) pages

awk '($9 ~ /502/)' /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -r

See who is requesting broken links (or URLs resulting in 502)

awk -F\" '($2 ~ "/workfusion/login"){print $1}' /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -r

Most requested URLs

awk -F\" '{print $2}' /var/log/nginx/access.log | awk '{print $2}' | sort | uniq -c | sort -r

Most requested URLs containing XYZ

awk -F\" '($2 ~ "ref"){print $2}' /var/log/nginx/access.log | awk '{print $2}' | sort | uniq -c | sort -r

Maximum response size

awk '($9 ~ /200/)' /var/log/nginx/access.log  | awk '{print $10}' | sort -nr | head -n 1

::note Only HTTP 200 OK response are taken into consideration. :::

Average response size

echo $(( `awk '($9 ~ /200/)' /var/log/nginx/access.log | awk '{print $10}' | awk '{s+=$1} END {print s}'` / `awk '($9 ~ /200/)' /var/log/nginx/access.log  | wc -l` ))

Top IP for requests

top -b | tail -f /var/log/nginx/access.log | cut -d ' ' -f 1
tail -f /var/log/nginx/access.log | cut -d ' ' -f 1 | logtop
← PreviousNext →