Execute complex command from bash script

Question

I'm working with kafka and I want to monitor topics. Basically I output several topics to stdout and to file for later review. For single topic monitoring I came up with the following command:

${KAFKA_HOME}/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic mytopic | \
    while IFS= read -r line; do 
      printf '[%s | %20s] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "mytopic" "${line}";
    done | tee -a kafka.out

However, there're many topics to consume, so I'm trying to do something like

consumer=${KAFKA_HOME}/bin/kafka-console-consumer.sh
mapfile -t topics < <(${KAFKA_HOME}/bin/kafka-topics.sh --zookeeper localhost:2181 --list)
for i in ${topics[@]}; do
  xterm -T ${i} -e "my_command" &
done
tail -F kafka.out

where my_command is the above command. It doesn't work. I suspect I'm messing up with quotes, unfortunately I can't figure out how to apply them correctly

Why not make the first command an executable (eg show_topic that takes the topic as a parameter. Then your xterm line becomes a simple xterm -e "show_topic $i" and you avoid the whole quoting mess. — Stephen Harris
– Stephen Harris, Commented Aug 24, 2016 at 13:00

Stéphane Chazelas · Accepted Answer · 2016-08-24 14:58:20Z

If you want to store some text verbatim, it's usually easier with this syntax:

mycommand=$(cat << 'EOF'
"${KAFKA_HOME}/bin/kafka-console-consumer.sh" --zookeeper localhost:2181 --topic "$TOPIC" |
  while IFS= read -r line; do 
    printf '[%s | %20s] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$TOPIC" "${line}";
  done | tee -a kafka.out
EOF
)

In ksh93, bash or zsh, you can also make it:

mycommand=$(<<'EOF'
....
EOF

In ksh93 that optimises out the exec of cat and the fork, while with bash only the exec (in zsh, it makes no difference).

In mksh:

mycommand=<<'EOF'
...
EOF

And then:

export TOPIC KAFKA_HOME
for TOPIC in "${topics[@]}"; do
  xterm -T "$TOPIC" -e "$mycommand" &
done

Here, I'd use ts to time-stamp the input (or do it in gawk or perl if ts is not available) instead of that very inefficient while read loop.

ts '[%F %T | mytopic]'

Stack Exchange Network

Execute complex command from bash script

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Execute complex command from bash script

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions