use field in index name for elasticsearch plugin logstash

Question

I am trying to have elasticsearch index based on field so I can get an index for each source (allowing for secure access to each index).

I tried something along the lines of

output {
    stdout { codec => rubydebug }
    elasticsearch {
       index => [SERVER]"-%{+YYYY.MM.dd}"
    }
}

as well as

output {
    stdout { codec => rubydebug }
    elasticsearch{
       index => "[SERVER]-%{+YYYY.MM.dd}"
    }
}

and neither work : first errors, second tries to create the index with [SERVER] in it then errors due to uppercase, this might not be supported as I can't find it anywhere in the docs, but I was wondering if anyone has gotten something like this functional for their own ELK stacks?

baudsp · Accepted Answer · 2016-06-15 15:34:46Z

2

The right syntax for this is "%{SERVER}-%{+YYYY.MM.dd}"

According to the documentation :

[The index to write] can be dynamic using the %{foo} syntax.

edited Jun 15, 2016 at 15:34

answered Jun 15, 2016 at 15:14

baudsp

4,1091 gold badge20 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

CodeBlake Over a year ago

how did i miss that... smh

CodeBlake Over a year ago

i tried output { stdout { codec => rubydebug } elasticsearch{ index => "%{SERVER}-%{+YYYY.MM.dd}" } }

CodeBlake Over a year ago

the above still didnt work, are you sure it can be mapped to fields?

baudsp Over a year ago

Well it worked in my case... Could you give more information => What happened this time (errors, index created with the wrong name...)? And maybe your configuration?

CodeBlake Over a year ago

that is the exact output configuration, in my filter i extract a field named SERVER using grok, it is simply sending the entire string %{SERVER} where it should be replacing the variable causing ES to reject it ( the field definitely exists i can see it in the output when i am using an index name that works)

|

Collectives™ on Stack Overflow

use field in index name for elasticsearch plugin logstash

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related