Monday, August 11, 2014

Grok nginx access logs with Logstash – grokking twice

I had a case where I wanted to see the directory and files being served up. In this case, doing so would provide an approximation of what players were doing inside the web/mobile game. I had already set up Logstash to parse nginx access logs, but this was some additional parsing. Well, here’s the best way to do it: Just grok the same field again. Below is my actual Logstash filter file. The second “grok {…… }” section is what I added.


filter {

if [program] == “nginx-access” {


grok {

match => [ "message" , "%{IPORHOST:remote_addr} - %{USERNAME:remote_user} \[%{HTTPDATE:time_local}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer$

}


grok {

match => [ "message" , ".*GET /%{WORD:ActionType}/%{WORD:Action}?.*" ]

}

geoip {

source => “remote_addr”

database => “/opt/logstash/vendor/geoip/GeoLiteCity.dat”

}

}

}


I used this to write the additional grok parse: http://ift.tt/1sPTniB





No comments:

Post a Comment