There are times when a webserver which you’re in charge of administrating has a traffic spike and you need to try to identify whether that is malicious or standard traffic.
There is always the possibility to check the log files, but this isn’t always easy if there are 100s of sites on a server with an individual log file for each domain name.
This is when the tool ngrep can come in handy! Ngrep allows you to watch real-time the traffic as it is being served by the Apache web server in a similar vein to how tcpdump works for network packets.
The best way to see how this works is to see it in action, so below is an example of me accessing Steve’s most recent blog post:
ngrep -l -q -d eth0 "^GET" tcp and port 80
interface: eth0 (xxx.xx.xxx.xx/xxx.xxx.xxx.xxx)
filter: (ip or ip6) and ( tcp and port 80 )
match: ^GET
T xxx.xxx.xxx.xxx:38110 -> xxx.xxx.xxx.xxx:80 [AP]
GET /expertise/blog/2012/12/12/retro-blog-christmas-1982-zx-spectrum/ HTTP/1.0..Host: forlinux.co.uk..User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.91 Safari/537.11..Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8..Accept-Encoding: gzip,deflate,sdch..Accept-Language: en-US,en;q=0.8..Accept-Charset:
ISO-8859-1,utf-8;q=0.7,*;q=0.3..Via: 1.1 (squid/2.6.STABLE21)..X-Forwarded-For: 10.0.0.7..Cache-Control: max-age=259200..Connection: keep-alive....
As you can see this shows me accessing the page from our office, not particularly useful in isolation but if you saw this scrolling your terminal output very quickly and all from one IP address it might indicate that you have specific IP address accessing a lot of pages.
You can search through this based on any parameters which you can see in the above output. Another example is me downloading a file using wget.
ngrep -l -q -d eth0 "User-Agent: Wget" tcp and port 80
interface: eth0 (xxx.xx.xxx.xx/xxx.xxx.xxx.xxx)
filter: (ip or ip6) and ( tcp and port 80 )
match: User-Agent: Wget
T xxx.xxx.xxx.xxx:38110 -> xxx.xxx.xxx.xxx:80 [AP]
GET /downloads/script HTTP/1.0..User-Agent: Wget/1.12 (linux-gnu)..Accept: */*..Host: forlinux.co.uk..Via: 1.0 (squid/2.6.STABLE21)..X-Forwarded-For: 10.0.0.7.
.Cache-Control: max-age=259200..Connection: keep-alive....
More information on this application can be viewed at the site http://ngrep.sourceforge.net/.
