read raw input lines and output single array

Question

I have a directory with files in it. I would like to create an array from that list of files. I thought it would be pretty easy, like:

ls mydir | jq -R '[.]'
[
  "file1"
]
[
  "file2"
]
[
  "file3"
]

The only thing I could figure out is this:

ls mydir | jq -sR '[split("\n")[]|select(.|length>0)]'
[
  "file1",
  "file2",
  "file3"
]

Is there a better way?

i am merging the contents of each file in that directory. each file is known as a dataset. the dataset name is the filename. so, once these files are merged, i am injecting metadata into that merge which declares which datasets comprises this merge. that injection is this array. so, two things. merging contents, injecting dataset origin metadata. the files have all been created by me, they have predictable filenames. — Greg
– Greg, Commented Dec 14, 2020 at 17:10

Inian · Accepted Answer · 2020-12-14 15:19:37Z

2

You'd have to be extra careful in dealing with Unix filenames in general. They can contain almost any character in a filename, including whitespace, newlines, commas, pipe symbols, and pretty much anything else you'd ever try to use as a delimiter except NUL. Your best bet is to separate the names with the NUL character, which is the only character that can't be part of a valid filename and split on it with jq

Use the native shell printf to separate entries on \0 and delimit it back

printf '%s\0' * | jq -Rn 'inputs | split("\u0000")'

or for just files

for file in *; do 
  [ -f "$file" ] && printf '%s\0' "$file"
done | jq -Rn 'inputs | split("\u0000")'

edited Dec 14, 2020 at 15:19

answered Dec 14, 2020 at 14:55

Inian

87k15 gold badges166 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Greg Over a year ago

ls mydir | jq -nR '[inputs]'

peak · Accepted Answer · 2020-12-14 20:17:42Z

1

Using find opens up other possibilities:

find . -type f -maxdepth 1 -print0 |
  jq -Rs 'split("\u0000") | map(sub("./";""))'

answered Dec 14, 2020 at 20:17

peak

119k21 gold badges185 silver badges218 bronze badges

1 Comment

Greg Over a year ago

thanks @peak, the inputs is what was illuding me.

Cory Gross · Accepted Answer · 2024-03-30 05:26:12Z

0

This works. Slightly simpler than what you have:

ls mydir | jq -sR 'split("\n")[0:-1]'

answered Mar 30, 2024 at 5:26

Cory Gross

37.3k18 gold badges73 silver badges81 bronze badges

Collectives™ on Stack Overflow

read raw input lines and output single array

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related