0

I have a file looks like below:

1 
4 5 6 7 19
20
22
24 26 27 
29
30
31
32 
34 
40 
50 
56 
58
100
234 235 270 500
1234 1235 1236 1237
2300
2303
2304
2307
2309

As it is clear there are some rows with more than 1 column and some others with only one single column. I'd like to join together single-column rows such that there are at most 4 columns on each combined row. So the output should look like this:

1  
4 5 6 7 19
20 22
24 26 27 
29 30 31 32
34 40 50 56 
58 100
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307
2309

Any suggestions on how do this, considering that the real data is big?

Improve this question

4 Answers 4

Reset to default
4

A little bit idiomatic but working with gnu awk:

awk '{printf "%s",(NF==1?$0 FS:(c==0?"":RS) $0 RS)} \
{(NF==1?++c:c=0)} \
c==4{printf "\n";c=0} \
END{printf "\n"}' file

#Output
1 
4 5 6 7 19
20 22 
24 26 27
29 30 31 32 
34 40 50 56 
58 100 
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307 
2309

Explanation:
awk variables:
NF=Number of Fields
FS=Field Separator = space by default
RS=Record Separator= new line by default.
c=counter

Line1: {printf "%s",(NF==1?$0 FS:(c==0?"":RS) $0 RS)}: nested ternary if operations 

#Single ternary if operation:
condition?true action:false action
#Nested if operations:  
condition1?true action 1:(condition2:true action2:false action2) #nested ternary if operations   
-------------------------[            ^ false action1 ^        ]

This can be explained in pseudocode like: 

if NF==1 then print $0 and print FS   
else (if c==0 then print "" else print RS) and print $0 and print RS again

Line 2: {(NF==1?++c:c=0)} : Another ternary if operation that can be expressed as: 

If NF==1 (line has one field) 
then increase counter c by one 
else reset counter c.

Line 3 : c==4{printf "\n";c=0} Classic awk syntax : condition{action} 

If counter c==4 then print a new line and reset counter c

Line 4: END{printf "\n"}' file : This justs prints a new line at the end of the script.

Improve this answer
4
  • 2
    Ugh, this might be a bit more understandable with at least a bit of explanation added, or at least a more friendly formatting? Just saying. :) Commented Jun 1, 2017 at 12:20
  • @ilkkachu OK - Added some basic explanation and a bit better formatting. Thanks for your comment. Commented Jun 1, 2017 at 13:57
  • Are extra interfield spaces and trailing spaces a big deal? Your solution have them both :) - I have run it just now. Should we care about it or it is unimportant in this case? I was care about it in the my solution. Commented Jun 1, 2017 at 14:50
  • 1
    @MiniMax Default field separator of awk (FS) is actually whitespace and this include one or more spaces. A basic awk like awk '{print $2}' <<<"one two" will work either with one space between fields or with more spaces . Commented Jun 2, 2017 at 7:37
2

You may use sed to get what you want:

sed -e '
   /./!b
   /[^[:space:]]/!b
   /[^[:space:]][[:blank:]]\{1,\}[^[:space:]]/b

   :loop
      $q;N
      /\n.*\S[[:blank:]]\+\S/b
      s/\n/ /;tdummy
      :dummy
      s/[[:space:]]\{1,\}/&/3;t
   bloop
' yourfile

Explanations

  • Skip empty, blank, and lines with NF > 1.
  • Setup a do-while loop at the point where the pattern space holds a single-field line.
  • We grab the next line and check whether it has NF > 1, at which point we print the whole pattern space and go back to reading the next line.
  • Now we know that the next line also is single-field, so we go ahead and clip the newline joining these two portions in the pattern space.
  • Does the pattern space have 3 spaces chunks yet? If yes then we print the whole of pattern space and start reading the next line.
  • Otherwise, we branch back to the loop which will in turn read the next line but attach it to the existing pattern space.

Result

1
4 5 6 7 19
20 22
24 26 27
29 30 31 32
34 40 50 56
58 100
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307
2309
Improve this answer
2

Usage: ./join_rows.awk input.txt

Check shebang #!/usr/bin/awk -f, because the awk location may differ on your system.

#!/usr/bin/awk -f

BEGIN {
    count = 1;
}

{
    if (NF == 1) {
        if (count > 1 && count <= 4) printf " ";

        printf "%s", $1;
        count++;

        if (count > 4) {
            printf "\n";
            count = 1;
        }
    } else {
        if (count > 1) printf "\n";

        print;
        count = 1;
    }
}

END {
    if(count > 1) printf "\n";
}

Output:

1
4 5 6 7 19
20 22 
24 26 27  
29 30 31 32
34 40 50 56
58 100
234 235 270 500
1234 1235 1236 1237 
2300 2303 2304 2307
2309
Improve this answer
1
  • +1 for clarity. I would probably start count from zero, now I think count == 1 means that there weren't any single-column rows. Commented Jun 1, 2017 at 12:19
1

Extended gawk approach:

rearrange_columns.awk script:

#!/bin/awk -f
function printRow(a, i, v)
{
    for (i in a) {
        printf "%s ", a[i]
    }
   print ""
   delete a
}
NF <= 2{
    for (i=1; i<=NF; i++) { 
        a[++c] = $i 
        if (length(a) == 4) {
            c = 0 
            printRow(a) 
        }
    }
}
NF > 2{
    if (length(a) > 0) {
        c = 0
        printRow(a)
    }
    print $0 
}
END{ print }

Usage:

awk -f rearrange_columns.awk yourfile

The output:

1 
4 5 6 7 19
20 22 
24 26 27 
29 30 31 32 
34 40 50 56 
58 100 
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307 
2309
Improve this answer
4
  • I added one row with two columns and your solution broke. For example, add one column to the 9-th row, new row will be 32 87. I got this string in the output: 29 30 31 32 87 34 40 50 56 58 100 Commented Jun 1, 2017 at 13:34
  • @MiniMax, be more precise. You added one row with two columns in what position? Commented Jun 1, 2017 at 13:36
  • I was writing the row number - "to the 9-th row". Row which have one column with 32 value. Commented Jun 1, 2017 at 13:41
  • @MiniMax, ok, fixed. Thanks for checking out Commented Jun 1, 2017 at 19:45

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.