0

Please find below an excerpt from one of my file.

1991;1;-7;-3;-9;-4;-7
1991;1;-7;-3;-9;-4;-7
1991;1;-7;-3;-9;-4;-7
1991;2;-14;-11;-14;-4;-14
1991;2;-14;-11;-14;-4;-14
1991;2;-14;-11;-14;-4;-14
1991;3;-7;-3;-15;5;-7
1991;3;-7;-3;-15;5;-7
1991;3;-7;-3;-15;5;-7
1991;4;-15;-9;-21;1;-16
1991;4;-15;-9;-21;1;-16
1991;4;-15;-9;-21;1;-16
1992;1;-12;-6;-19;-2;-12
1992;1;-12;-6;-19;-2;-12
1992;1;-12;-6;-19;-2;-12
1992;2;-16;-7;-22;-12;-15
1992;2;-16;-7;-22;-12;-15
1992;2;-16;-7;-22;-12;-15
1992;3;-22;-15;-25;-16;-24
1992;3;-22;-15;-25;-16;-24

I'm trying through sed or/and awk to add + 1 on the second column for the second row for the second row as long as the year in the first column remains the same.

The results would be the following:

1991;1;-7;-3;-9;-4;-7
1991;2;-7;-3;-9;-4;-7
1991;3;-7;-3;-9;-4;-7
1991;4;-14;-11;-14;-4;-14
1991;5;-14;-11;-14;-4;-14
1991;6;-14;-11;-14;-4;-14
1991;7;-7;-3;-15;5;-7
1991;8;-7;-3;-15;5;-7
1991;9;-7;-3;-15;5;-7
1991;10;-15;-9;-21;1;-16
1991;11;-15;-9;-21;1;-16
1991;12;-15;-9;-21;1;-16
1992;1;-12;-6;-19;-2;-12
1992;2;-12;-6;-19;-2;-12
1992;3;-12;-6;-19;-2;-12
1992;4;-16;-7;-22;-12;-15
1992;5;-16;-7;-22;-12;-15
1992;6;-16;-7;-22;-12;-15
1992;7;-22;-15;-25;-16;-24
1992;8;-22;-15;-25;-16;-24

I've seen countless examples on stackflow but nothing that can lead me close to a solution.

I welcome any suggestions.

Best,

7
  • edit lines with the first year (1991) in the file? Commented Mar 21, 2014 at 13:30
  • hi, I'm not sure to understand ... Commented Mar 21, 2014 at 13:32
  • edit all years or just one year? Commented Mar 21, 2014 at 13:33
  • all years, suspectus. I was thinking of using a FOR loop , which would be parsing for the year and then use "awk -F\| '{++$2;print}'" Commented Mar 21, 2014 at 13:35
  • Why list specific tools (sed, awk) when you don't actually care about which tool is used, or have reason to believe any particular tool to be appropriate for the use case? If you want a solution using standard shell tools, specify "standard shell tools" rather than assuming that specific tools will fit the problem. Commented Mar 21, 2014 at 13:39

4 Answers 4

7

If you always want the 2nd column to be 1 for the line in which the year first appears in column 1, then:

awk -F\; '$1!=l{c=0}{$2=++c}{l=$1}1' OFS=\; input

If you want to maintain whatever was in column 2:

awk -F\; '$1!=l{c=$2}{$2=c++}{l=$1}1' OFS=\; input
Sign up to request clarification or add additional context in comments.

1 Comment

+1, terseness. I normally prefer readability, but it's good to have a proper answer showing awk's strengths.
4

This could be done more tersely with awk, but pure bash works fine:

last_year=
counter_val=
while IFS=';' read -r year old_counter rest; do
  if [[ $year = "$last_year" ]]; then
    (( ++counter_val ))
  else
    counter_val=1
    last_year=$year
  fi
  printf -v result '%s;' "$year" "$counter_val" "$rest"
  printf '%s\n' "${result%;}"
done <input.txt >output.txt

8 Comments

Charles, I've tried your command line but it does not work ... It gives me "non valid arithmetical error" when running the while loop.
@AndyK, that's not an error message that actually exists anywhere in bash. Are you sure you copied it correctly? Also, are you sure your script actually starts with #!/bin/bash, not #!/bin/sh? (If you had a script starting with #!/bin/bash but you ran it with sh scriptname, that would cause the same problem)
@AndyK, Ooh. Did find a bug there -- needed quotes around the IFS value. Try again now, with IFS=';' rather than IFS=;.
I've tried it without putting it in a shell script. Let me try again within a proper shell script. Thanks for your feedback, Charles.
This is what I have after running in a bash shell ./test_toto.sh < toto2_rev.csv > toto3_rev.csv ./test_toto.sh: line 6: ((: Ann▒e: erreur de syntaxe : opérateur arithmétique non valable (error token is "▒e") ./test_toto.sh: line 6: ((: Ann▒e: erreur de syntaxe : opérateur arithmétique non valable (error token is "▒e") ./test_toto.sh: line 6: ((: Libell▒: erreur de syntaxe : opérateur arithmétique non valable (error token is "▒")
|
1

You simply want to increment your second column, and not add one to it? Do you want the second column to go from one onward no matter what the second column is?

awk -F\; '{
    if ( NR == 1 ) {
        year = $0
    }
    if ( year == $0 ) {
        for (count = 1; count < NF; count++) {
            if ( count == 2) {
                printf NR ";";
            }
            else {
                printf $count ";";
            }
        }
        print "";
    }
    else {
        print 
    }
}' test.txt

Awk is a natural program to use because it operates in assuming a loop. Plus, it's math is more natural than plain shell.

The NR means Number of Records and NF means Number of fields. A field is separated by my -F\; parameter, and the record is the line number in my file. The rest of the program is pretty obvious.

5 Comments

Good answer, but I disagree somewhat with the commentary -- since the shell specified is bash, it has perfectly natural integer math; as long as you're in a math context (with (( ))), you have all the same operators C does -- even the ternary. The same mostly applies to ksh and POSIX sh, except with the need to use $(( )) in the latter; it's been a long time since integer math was awful in shell.
David, the test.txt is the input file. Am I correct with my guess?
Yes. I grabbed your data and put it in a file called text.txt.
Thanks for your feedback. Checking again.
I tried again with my files and also with the excerpt in a file but it is not functionning...
1

Using awk, set the FS (field separator) and OFS (output field separator) to ';' and for each new year record set the val counter to the start column 2 value. Increment val for each line with that year.

awk -F';' 'BEGIN{OFS=";";y=0} 
 { if (y!=$1) 
      {y=$1;val=$2;print} 
   else 
      {val++;print $1,val,$3,$4,$5,$6,$7}}' data_file

3 Comments

your solution is working fine , Suspectus. Many thanks for being the first to answer.
Given all the other answers I have to thank you for verifying it worked! :)
Always, Suspectus, always. I value your work and the ones of the other contributors and I'm learning. No work is like a drop of water in the rain , it is a single and unique. ;)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.