I have record in log file with records in JSON:
{"created":1610314252.6277733,"process":13,"levelname":"INFO","message":"Requesting url: http://smshost:8080/push with method: POST and params {'data': {'message': 'Record changed: go to:XXXX', 'resource_id': '1824908f-efae-474e-aa3e-579cabe67517', 'subscriber': '000000000000', 'service': 'SERVICE'}, 'json': None}","message_id":268562185}
Filebeat send this data to logstash
filebeat.prospectors:
- paths:
- /opt/apps/mobilizer/*.log
input_type: log
json.keys_under_root: true
json.add_error_key: true
json.message_key: log
Output data:
{
"@timestamp":"2021-01-11T09:53:48.062Z",
"@metadata":{"beat":"filebeat","type":"doc","version":"6.8.13"},
"levelname":"INFO",
"source":"/opt/apps/mobilizer/logs/test.json",
"offset":114,
"message":"Requesting url: http://smshost:8080/push with method: POST and params {'data': {'message': 'Record changed: go to:XXXX', 'resource_id': '1824908f-efae-474e-aa3e-579cabe67517', 'subscriber': '000000000000', 'service': 'SERVICE'}, 'json': None}",
"beat":{"name":"skif-HP-240-G7-Notebook-PC","hostname":"skif-HP-240-G7-Notebook-PC","version":"6.8.13"},
"host":{"name":"skif-HP-240-G7-Notebook-PC"},
"message_id":268562185,
"error":{"message":"Key 'log' not found","type":"json"},
"log":{"file":{"path":"/opt/apps/mobilizer/logs/test.json"}},
"process":13,"created":1.6103142526277733e+09
}
But I want to parse the "message" and get the dictionary data after "paran". (Either in logstash or filebeat ):
{
...
"message":"Requesting url: http://smshost:8080/push with method: POST and params ",
"data": {"message": "Record changed: go to:XXXX", "resource_id": "1824908f-efae-474e-aa3e-579cabe67517', "subscriber": "000000000000', "service": "SERVICE"},
"json": "None",
...
}
How can I do this? Where is it better to do in logstash or in filebeat?