Logstash explode string to array with keys

Question

This is my pipeline output:

"attributes" => "Width:150,200;Height:200;Size:L"

I'd like to have an output:

"attributes" => [
    "Width"  => [
        150,
        200
    ],
    "Height" => 200,
    "Size"   => "L"
]

I tried using the mutate filter

mutate {
    split => ["attributes", ";"]
}

Which transforms the data this way

"attributes" => [
    [0] "Width:150",
    [1] "Height:200"
    [2] "Size:L"
]

Is there a way to transform it via logstash filters?

alpert · Accepted Answer · 2016-05-30 10:51:22Z

3

You can use kv filter:

kv {
    source => "attributes"
    field_split => ";"
    value_split => ":"
    target => "attributes"
}

ruby {
    # splits all fields so all fields are array
    code => "event['attributes'].each do |cusField| event['attributes'][cusField[0]] = cusField[1].split(',') end"
    # splits only if field contains ','
    # code => "event['attributes'].each do |cusField| event['attributes'][cusField[0]] = cusField[1].split(',') if cusField[1].include? ',' end"
}

edited May 30, 2016 at 10:51

answered May 23, 2016 at 11:05

alpert

4,6751 gold badge21 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Jan Richter Over a year ago

Just a tiny addition, if I have "Width:100,150;Height..." would I be able to explode even values by comma to array?

alpert Over a year ago

You can conbine this with a mutate you used in question.

Jan Richter Over a year ago

I get a new field name, is there a way to preserve it in "attributes" ? Also the how to target the mutation with a new fieldname (which I dont know prior to the split)

alpert Over a year ago

Updated my answer for preserving in "attributes". You can refer [attributes][Weight] in mutate later.

Jan Richter Over a year ago

The thing is I don't know how many attributes there will be (width, height, size and maybe a hundred more) is there a way to rename the fields in some sort of loop?

|

Basem Mohammed · Accepted Answer · 2017-02-23 12:36:16Z

-2

You can use split directly

 filter {
      split {
            field => "attributes"
          }
      }

Here is the documentations: https://www.elastic.co/guide/en/logstash/5.2/plugins-filters-split.html#plugins-filters-split-remove_field

answered Feb 23, 2017 at 12:36

Basem Mohammed

32 bronze badges

1 Comment

PhaedrusTheGreek Over a year ago

The split filter splits events into multiple events

Collectives™ on Stack Overflow

Logstash explode string to array with keys

2 Answers 2

9 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

9 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related