Logstash kafka topic as part of elasticsearch indexname

Question

I want to use kafka topic as part of es-indexname. Here is my example, but this does not work:

input {
  kafka {    
    ...
    decorate_events => true
  }
}

filter {    
  mutate {
    add_field => {"[@metadata][index]" => "[kafka][topic]"}
  }
}

output {
  elasticsearch {
    index => "kafka-%{[@metadata][index]}-%{+YYYY.MM.dd}"
    ...
  }
}

Can someone find the problem in this logstash.conf?

Кирилл Полищук · Accepted Answer · 2017-07-05 11:48:44Z

2

hello this is working config:

input {
  kafka {    
    ...
    decorate_events => true
  }
}

filter {    
  mutate {
    add_field => {"[@metadata][index]" => "%{[kafka][topic]}"}
  }
}

output {
  elasticsearch {
    index => "kafka-%{[@metadata][index]}-%{+YYYY.MM.dd}"
    ...
  }
}

answered Jul 5, 2017 at 11:48

Кирилл Полищук

8456 silver badges4 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Logstash kafka topic as part of elasticsearch indexname

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related