Shell Script: Using files based on part of file name

Question

I have a directory (say ~/dir/) containing several hundred files. Half of these will begin with the string "ABC", ie they will be called ABC_0.csv, ABC_1.csv, ABC_2.csv etc etc.

My goal is to write a shell script that will take each of these "ABC" files and merge them together in one bigger file which I am calling "master_ABC".

I know how to merge them, but I do not know how to write a shell script that will only take files with names beginning in "ABC" (Note: There are other files in the ~/dir/ that I have no interest in and want to avoid).

Moreover, the number of "ABC" files will vary from day to day.

nu11p01n73R · Accepted Answer · 2014-09-26 12:28:13Z

2

Use wildcard * expansion to get different files ABC_1.csv and so on

cat ABC_*.csv > master_ABC.csv

answered Sep 26, 2014 at 12:28

nu11p01n73R

26.8k3 gold badges42 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Knight71 · Accepted Answer · 2014-09-26 12:49:35Z

1

You could capture all the files in a list and then cat with append (>>) to master file

files=`ls ABC*csv`

 for f in $files
 do
   echo $f
   cat $f >> master_ABC.csv
 done

answered Sep 26, 2014 at 12:49

Knight71

2,9595 gold badges39 silver badges66 bronze badges

Comments

Amar · Accepted Answer · 2014-09-26 12:52:32Z

1

You can use wildcard * for this.

#!/bin/bash

cat ~/dir/ABC*csv > master_ABC

answered Sep 26, 2014 at 12:52

Amar

3,8451 gold badge26 silver badges26 bronze badges

Comments

afenster · Accepted Answer · 2014-09-26 12:55:22Z

1

Use ls (or find) with grep in while loop:

ls | grep 'ABC_.*\.csv$' | while read fn ; do cat $fn >> master_ABC.csv ; done

or with find (especially if you need to traverse subdirectories recursively):

find . -type f -name 'ABC*.csv' | while read fn ; do cat $fn >> master_ABC.csv ; done

Note that grep accepts a regex, while find required a wildcard string.

I'd suggest to avoid using * in such cases as it will not work for very long list of files, and will also fail if any of the file names contain a space character.

answered Sep 26, 2014 at 12:55

afenster

3,63821 silver badges26 bronze badges

Collectives™ on Stack Overflow

Shell Script: Using files based on part of file name

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related