0

I want to parse this json using logstash.

{"name":"bob","last":"builder", "atts":"{\"a\":111, \"b\":222}"}

{ "name" => "bob", "last" => "builder" "atts" => { "a" => 111, "b" => 222} }

1
  • Please add more details about your use case. So then we can help you out. And also add the sample of what you have tried. Commented Aug 21, 2019 at 21:23

1 Answer 1

2

Two options!

Parsing JSON using Logstash

If you want to parse JSON using logstash- would refer to the logstash plugin here:

https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html

To achieve this- you'd be toying with the filter part of your logstash.conf:

filter {
   json {
     source => "message"
   }
}

there are more examples of json decoding in that link.

Parsing JSON using Filebeat

Your other option would be to decode json on the filebeat side before it gets into logstash. Relevant links:

https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html

https://discuss.elastic.co/t/parse-json-data-with-filebeat/80008/5

https://discuss.elastic.co/t/parse-json-data-with-filebeat/80008/7

https://discuss.elastic.co/t/how-to-read-json-file-using-filebeat-and-send-it-to-elasticsearch/91802

Here's a sample filebeat.yml for this situation:

filebeat.inputs:
  - type: log
    paths:
      - 'path to the log directory you want to track'
    enter code here
    input_type: log
    json.keys_under_root: true
    json.add_error_key: true
    fields:
        log_type: 'type of log'

    processors:
    - decode_json_fields:
        fields: ["message"]
        process_array: true

    - add_tags:
        tags:
            - 'tag in elastic'

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

setup.template.settings:
  index.number_of_shards: 1

output.logstash:
  # The Logstash hosts
  hosts: ["where logstash is running"]
  index: 'your index'

  codec.json:
    pretty: true
    escape_html: false

#================================ Processors =====================================
# Configure processors to enhance or manipulate events generated by the beat.
processors:
- decode_json_fields:
    fields: ["message"]
    process_array: true
json.keys_under_root: true
json.add_error_key: true

and

processors:
    - decode_json_fields:
        fields: ["message"]
        process_array: true

does the trick.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.