-
Notifications
You must be signed in to change notification settings - Fork 33
Open
Description
I am using below grok_pattern for parsing Nginx logs
<source>
@type tail
path /data/nginxlogs/access.log
pos_file /var/log/td-agent/nginx-access.log.pos
tag es.nginx.acces
<parse>
@type multiline_grok
<grok>
pattern Started %{IPORHOST:client_ip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] \"%{WORD:method} %{NOTSPACE:request_page} HTTP/%{NUMBER:http_version}\" %{NUMBER:server_response} %{NUMBER:bytes} \"%{URI:referrer}\" %{QS:user_agent}
# grok_pattern %{MY_SUPER_PATTERN}
# custom_pattern_path /etc/td-agent/MY_SUPER_PATTERN
# multiline_start_regexp /^[^\s]/
</grok>
keep_time_key true
</parse>
</source>
<filter es.nginx.acces>
@type record_transformer
<record>
hostname ${hostname}
</record>
</filter>
<match es.**>
type "aws-elasticsearch-service"
type_name "access_log"
logstash_format true
logstash_prefix "ngrokcontainer"
include_tag_key true
tag_key "@log_name"
flush_interval 1s
<endpoint>
url http://testing-logs-ab-east-1.es.amazonaws.com
region us-east-1
</endpoint>
</match>
Kibana :
| message | 3.231.213.28 - - [18/Apr/2020:22:20:16 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-" |
|---|---|
| Its not parsing properly, but when I check this grok debugger its working fine. | |
| Please let me know how I can fix this issue |
Metadata
Metadata
Assignees
Labels
No labels