2

I have multi-column data that I wish to reformat such that each column follow "on top" of each other in one single column.

Example input:

1  2  3  4
1  2  3  4

Desired output: (fixed old typo where a "1" and a "3" was missing)

1
1
2
2
3
3
4
4

(Occasionally I would like to add spaces between the columns too)

Edit: The comment about "spaces between" was unclear. I sometimes want an empty line between the stacked columns. Adapting the Ed Morton's accepted answer below, this could be achieved by

awk '
{for (i=1; i<=NF; i++) a[i]=a[i] $i ORS}
END {for (i=1; i<=NF; i++) printf "%s", a[i](i==NF?"":"\n")}
' file
Improve this question
3
  • 1
    What do you mean mit space between the columns? Commented Jul 23, 2022 at 20:18
  • If you want to convert spaces to newlines, use tr. But your request is unclear. Commented Jul 24, 2022 at 12:31
  • @FelixJN Yes, that works for stacking the file content row-wise but unfortunately not column-wise. Commented Jul 25, 2022 at 11:56

4 Answers 4

Reset to default
2

Using datamash to transpose the data and then tr for converting spaces to newlines (while -squeezing multiple spaces to one).

datamash -t' ' transpose <infile | tr -s ' ' '\n'
Improve this answer
1
  • 1
    datamash -W --output-delimiter $'\n' transpose <file Commented Jul 25, 2022 at 18:27
1

This may be what you want, using any awk in any shell on every Unix box:

$ awk '
    { for (i=1; i<=NF; i++) a[i] = a[i] $i ORS }
    END { for (i=1; i<=NF; i++) printf "%s", a[i] }
' file
1
1
2
2
3
3
4
4
Improve this answer
0

How about GNU grep:

grep -o '\S*' infile

Or, expanding (an equivalent to) \S:

grep -o '[^ \t\r\n\v\f]*' infile

If the order of columns needs to be retained, try (with GNU sort)

$ awk '{for (i=1;i<=NF;i++){ print i,$i }}' infile | 
      sort -snbk 1,1 | 
      awk '{$1=""}1'

1
1
2
2
3
3
4
4

Or, using only awk, and a process similar to transposing columns:

awk '{ for(i=1;i<=NF;i++) { f[i]=(f[i]==""?"":f[i] ORS) $i; if(NF>n) n=NF}}
       END { for(i=1; i<=n; i++) { print f[i] }  }
    '  infile
Improve this answer
0
-2

This answer relies on switching the order in which the columns and rows are printed, in effect, generating a transposition of the original data.

The same output was updated in OP's question and as expected, it was initially incorrect.

for i in `seq 4`; do 
    awk -v field=$i '{ print $field }' infile; 
done

1
1
2
2
3
3
4
4
Improve this answer
5
  • This is correct but very inefficient since it loops through the file however many times there are fields. For e.g. a multi-gigabyte file with a thousand columns this would become very important. (But I didn't downvote.) Also see Why is using a shell loop to process text considered bad practice? since you're running awk in a shell loop here. Commented Jul 25, 2022 at 21:06
  • @Wildcard Thank you for your comment. That potential issue was discussed in the now deleted comments for this answer. The reality is that even with thousands of columns (~1 million elements) this approach takes less than 1 minute. In order to have even a 1gb file similar to the sample input, you need to store roughly 100 million elements. Is that a realistic use case for this question? Commented Jul 25, 2022 at 21:43
  • By the way, I thought this answer was a quirky and concise way to "transpose" the elements, at least compared to the other more performant answers, but it seems to me some people are more concerned about saving system calls. Commented Jul 25, 2022 at 21:46
  • My experience as a sysadmin at some major companies informs my opinion: the people who copy and paste answers from Stack Exchange into production code don't consider whether their use case is realistic or whether their data size calls for a different approach. If you're writing super inefficient code that shouldn't be used for anything beyond toy examples, and you don't put vividly clear, bolded warnings on it, it's going to end up in production code. That doesn't make it "your fault" (copy-pasters are irresponsible), but it is the rationale for downvoting such answers. Commented Jul 25, 2022 at 22:51
  • 1
    That's fair, but in my personal experience writing answer on SE sites, there is no need to target the lowest common denominator among readers. Additionally, I think we provide answers just because it is fun, without trying to cover edge cases or high performance requirements. For example, the datamash answer, which is really good, takes ~1 minute for 10,000,000 values and the accepted answer, using awk, takes 29 minutes 51 seconds, so it is clear that performance was never a large priority. Every algorithm without constant time complexity will reach a point in which it is highly inefficient. Commented Jul 26, 2022 at 0:31

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.