Log Analysis
Analyze web server, system, and application logs to find flags and attacker activity in CTF forensics.
Log Analysis
Log files record what happened on a system — every HTTP request, every failed login, every error. CTF challenges give you logs and ask you to find attacker activity, reconstruct a timeline, or locate a flag hidden in the data.
The tools are simple (grep, awk, sort) but the skill is knowing what to search for and how to correlate entries across multiple log files. Always start with a direct flag search — grep -i "flag{" — before doing deeper analysis.
Investigation Flow
Types of Log Files
| Log File | Location | Contains |
|---|---|---|
| Apache access | /var/log/apache2/access.log | HTTP requests |
| Apache error | /var/log/apache2/error.log | Server errors |
| Nginx access | /var/log/nginx/access.log | HTTP requests |
| Auth log | /var/log/auth.log | SSH, sudo, login |
| Syslog | /var/log/syslog | General system |
| Kernel | /var/log/kern.log | Kernel messages |
| Cron | /var/log/cron.log | Scheduled jobs |
Apache/Nginx Access Log Format
93.184.216.34 - alice [10/Oct/2024:13:55:36 -0700] "GET /flag.txt HTTP/1.1" 200 1234 "-" "Mozilla/5.0"
↑ IP ↑ user ↑ timestamp ↑ method ↑ path ↑ code ↑ bytesIn Apache Combined Log Format: $1=IP, $4=timestamp, $7=path, $9=status code, $10=bytes transferred.
Investigation Methodology
Search for the flag directly
grep -i "flag{" access.log
grep -iE "CTF\{|flag\{" *.logIdentify suspicious IPs
# Count requests per IP — high volume = likely attacker
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20Analyze attacker's requests
grep "192.168.1.100" access.log
awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -20Decode encoded payloads
python3 -c "
from urllib.parse import unquote
print(unquote('/search?q=%3Cscript%3Ealert(1)%3C%2Fscript%3E'))
"Reconstruct the timeline
cat auth.log syslog | sort -k1,3 | head -100
grep "192.168.1.100" access.log | awk '{print $4,$5,$7}' | sortGrep Patterns
awk for Log Parsing
# Count requests per IP
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20
# Count requests per URL
awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -20
# Show requests in time range
awk '$4 >= "[10/Oct/2024:13:00:00" && $4 <= "[10/Oct/2024:14:00:00"' access.log
# Sum bytes transferred per IP
awk '{sum[$1] += $10} END {for (ip in sum) print sum[ip], ip}' access.log | sort -rnFinding Attacker Activity
Auth Log Analysis
grep "Failed password" auth.log
grep "Accepted password\|Accepted publickey" auth.log
grep "sudo:" auth.log
grep "10\.0\.0\.100" auth.log | head -50
# Count failed logins per username
grep "Failed password" auth.log | awk '{print $9}' | sort | uniq -c | sort -rnDecoding URL-Encoded Requests
python3 -c "
from urllib.parse import unquote
log_entry = '/search?q=%3Cscript%3Ealert(1)%3C%2Fscript%3E'
print(unquote(log_entry))
"
# → /search?q=<script>alert(1)</script>
# Extract and decode all URL params from access log
grep "GET" access.log | awk '{print $7}' | python3 -c "
import sys
from urllib.parse import unquote
for line in sys.stdin:
print(unquote(line.strip()))
" | grep -i flagFinding Hidden Flags
# Flag might be in a URL parameter
grep -iE "flag=|key=|secret=" access.log | head -20
# Or in the response body (if logged)
grep "flag{" error.log
# Base64 in logs
grep -E "[A-Za-z0-9+/]{20,}={0,2}" access.log | \
while read line; do
echo "$line" | grep -oE '[A-Za-z0-9+/]{20,}={0,2}' | \
while read b64; do
decoded=$(echo "$b64" | base64 -d 2>/dev/null)
echo "$decoded" | grep -qi "flag" && echo "Found: $decoded"
done
doneChecklist
-
grep -iE "flag\{|CTF\{"in all log files - Find unusual IPs (high request volume)
- Look for injection payloads (SQLi, XSS, traversal)
- Decode URL-encoded payloads
- Check
auth.logfor brute force or unusual logins - Reconstruct timeline of suspicious activity
- Decode base64 blobs found in logs
- Sum bytes per IP — large transfers reveal data exfiltration
Last updated on