Generate multiple output files for loop

Question

i'm trying to generate a new output file from each existing file in a directory of .txt files. I want to check line by line in each file for two substrings. And append the lines that match that substring to each new output file.

I'm having trouble generating the new files. This is what i currently have:

#!/bin/sh
# My first Script


success="(Compiling)\s\".*\"\s\-\s(Succeeded)"
failure="(Compiling)\s\".*\"\s\-\s(Failed)"
count_success=0
count_failure=0
for i in ~/Documents/reports/*;
do
        while read -r line;
        do
                if [[$success=~$line]]; then
                        echo $line >> output_$i
                        count_success++

                elif [[$failure=~$]]; then
                        echo $line >> output_$i
                        count_failure++
                fi
                done
        done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"
~

Any help would be appreciated, thanks

Arnaud Valmary · Accepted Answer · 2022-07-05 06:42:13Z

2

Please, use https://www.shellcheck.net/ to check your shell scripts. If you use Visual Studio Code, you could install "ShellCheck" (by Timon Wong) extension.

About your porgram.

Assume bash
Define different extensions for input and output files (really important if there are in the same directory)
Loop on report, input, files only
Clear output file
Read input file
if sequence:
- if [[ ... ]] with space after [[ and before ]]
- spaces before and after operators (=~)
- reverse operands order for operators =~
Prevent globbing with "..."

#! /bin/bash

# Input file extension
declare -r EXT_REPORT=".txt"
# Output file extension
declare -r EXT_OUTPUT=".output"

# RE
declare -r success="(Compiling)\s\".*\"\s\-\s(Succeeded)"
declare -r failure="(Compiling)\s\".*\"\s\-\s(Failed)"

# Counters
declare -i count_success=0
declare -i count_failure=0

for REPORT_FILE in ~/Documents/reports/*"${EXT_REPORT}"; do
    # Clear output file
    : > "${REPORT_FILE}${EXT_OUTPUT}"

    # Read input file (see named file in "done" line)
    while read -r line; do

        # does the line match the success pattern ?
        if [[ $line =~ $success ]]; then
            echo "$line" >> "${REPORT_FILE}${EXT_OUTPUT}"
            count_success+=1
        # does the line match the failure pattern ?
        elif [[ $line =~ $failure ]]; then
            echo "$line" >> "${REPORT_FILE}${EXT_OUTPUT}"
            count_failure+=1
        fi

    done < "$REPORT_FILE"
done

echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"

answered Jul 5, 2022 at 6:42

Arnaud Valmary

2,41511 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

tshiono Over a year ago

Very nice detailed suggestions. In addition, bash regex (ERE) doesn't support \s.

Arnaud Valmary Over a year ago

You could use [[:space:]] instead of \s or simply \ for one space

Renaud Pacalet · Accepted Answer · 2022-07-05 08:04:15Z

1

What about using grep?

success='Compiling\s".*"\s-\sSucceeded'
failure='Compiling\s".*"\s-\sFailed'
count_success=0
count_failure=0
for i in ~/Documents/reports/*; do
  (( count_success += $(grep -E "$success" "$i" | tee "output_$i" | wc -l) ))
  (( count_failure += $(grep -E "$failure" "$i" | tee -a "output_$i" | wc -l) ))
done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"

edited Jul 5, 2022 at 8:04

answered Jul 5, 2022 at 7:58

Renaud Pacalet

30.7k3 gold badges42 silver badges60 bronze badges

Collectives™ on Stack Overflow

Generate multiple output files for loop

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related