Hey everyone ! I finally decided to monitor my applications more closely with Grafana. However I’m having issues building dashboards their logs.
Their logs are currently sent over syslog (in RFC3164 format) into telegraf. But it simply puts the whole message into the message field, so I can’t use specific fields (eg. URL for httpd, source IP for DNS requests, username for SSH, …) to build graphs.
I’ve read about grok patterns, but I have no idea how to use them.
Would someone have any pointer on how I could make sense out of these logs for later use ?
I have a similar setup (all hosts sending logs through syslog protocol to a central collector), but the collector is graylog. A few years back it used to use Grok expressions, but now it has its own filter syntax. My notes on extractors/grok patterns are still there (unfold
details). Can’t help you much more than that, sorry!It does help thank you ;)
I’ve found that you can use custom grok patterns to parse logs just as grayling extractors do. I’m still trying to figure it out, but so far I could start parsing logs using a
[[processor.parser]]block. I’ll document my findings when I get it working as I want it.