Greetings, sir. Could you please help me with parsing json log file.
Example of log string:
{"ip": "176.122.232.00", "host": "cdn.tv.com", "path": "/dtv/c_1199622700411804.ts", "status": "200", "user_agent": "YX_XY/1.0.1.0.3364", "length": 1392120, "date": "2016-10-09T20:00:00+03:00"}
I'm using this format:
goaccess -f access.log --log-format='%^: "%h", %^:%^, %^: "%R", %^: %s, %^: "%u",%^: %b, %^: "%dT%t+%^"' --date-format='%Y-%m-%d' > /report.html
And getting
Parsing... [1 734 743] [867 371/s]s]
GoAccess - version 0.8.5 - Dec 5 2014 06:36:01
Fatal error has occurred
Error occured at: goaccess.c - main - 832
Nothing valid to process
What's wrong with my log-format?
Thanks.
konungrl
All 3 comments
This should work:
goaccess -f access.log --log-format='%^: "%h",%^: "%v",%^: "%U",%^: "%s",%^: "%u",%^: %b,%^: "%dT%t+%^"%^' --date-format='%Y-%m-%d' --time-format='%T' --http-protocol=no --http-method=no
allinurl
on 10 Oct 2016
Thanks for reply,
I've got this errors
goaccess: unrecognized option '--time-format=%T' (probably i have new version)
Then i removed option time-format and got 2 more errors.
goaccess: option '--http-protocol' doesn't allow an argument
goaccess: option '--http-method' doesn't allow an argument
When i removed all of this options i finally got my report.
By the way, could you please explain this log format for the for the further generations? :)
konungrl
on 10 Oct 2016
For those unrecognized options, please upgrade to the latest version.
Great question, one of the most important parts in the log format is the character that follows %^ or any other specifier. The parser will look for that character, so for instance, if I have %^:, it means that I want the parser to skip everything until a : is found. That helps a lot so I don't have to use a bunch of %^ to ignore fields. The upcoming version, will handle this better by displaying which tokens did not match the log format, which should make things easier.
For a list of available log specifiers, please take a look at this.
allinurl
on 10 Oct 2016
Related issues
narayan1599
·
3Comments
domainoverflow
·
3Comments
SergioDG-YCC
·
3Comments
canepa
·
3Comments
ArunDahiya1
·
3Comments
Most helpful comment
For those unrecognized options, please upgrade to the latest version.
Great question, one of the most important parts in the log format is the character that follows
%^or any other specifier. The parser will look for that character, so for instance, if I have%^:, it means that I want the parser to skip everything until a:is found. That helps a lot so I don't have to use a bunch of%^to ignore fields. The upcoming version, will handle this better by displaying which tokens did not match the log format, which should make things easier.For a list of available log specifiers, please take a look at this.