while loop returns one additional empty row

Question

I am attempting to write a bash script that list the tables in my local postgres db

while read line
do 
   myarray+=("$line")
   ## debug
   echo $line
done  < <(psql -h ... -U ... -t -c "select distinct table_name from information_schema.columns where table_schema='myschema' ;")

I have created 36 tables but the above code outputs a 37th line that is completely blank. I checked the number of elements like so echo ${#myarray[@]} ; 37 elements.

If I run the query select distinct table_name from information_schema.columns where table_schema='myschema' ;, I get 36 records back.

I am using the process substitution approach in the while loop as it seems faster but I think the error comes from there. Any ideas?

The loop seems like overkill. Can't you just say myarray=($(psql ...))? — Nick Barnes
– Nick Barnes, Commented Jul 12, 2017 at 22:09
Have you checked the output of psql? Does it have two newlines at the end or maybe \r\n? Can you post the output of psql ... | hd (hex dump)? — alpinist
– alpinist, Commented Jul 12, 2017 at 22:15
Unless you're expecting blank lines in the middle of the output (and you want to keep them), then you could just add (warning: bashism) [[ "$line" =~ ^[[:space:]]*$ ]] && continue at the top of the loop. — Mike Holt
– Mike Holt, Commented Jul 12, 2017 at 22:23

Nick · Accepted Answer · 2017-07-12 23:23:54Z

2

In Postgres code, there is fputc('\n', fout); added as a final unconditional step, for all output formats, so you need to filter it out, e.g.:

while read line
do 
   myarray+=("$line")
   ## debug
   echo $line
done  < <(psql -h ... -U ... -t -c "select distinct table_name from information_schema.columns where table_schema='myschema' ;" \
           | egrep -v "^$")

Also, keep in mind, that if in the future, you (or your colleague) will put something to ~/.psqlrc (for example, \timing on) it might break your code. To protect from it, use psql's option --no-psqlrc (or -X).

edited Jul 12, 2017 at 23:23

answered Jul 12, 2017 at 23:17

Nick

2,54315 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alan Over a year ago

Wow. That makes complete sense. Accepted and voted up. Thank you!

Collectives™ on Stack Overflow

while loop returns one additional empty row

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related