Shell script: execute cmd on a file, with additional processing of file name

Question

So I am going to post a question about shell scripting again.

Problem Definition: For all files under a dir, ex.:

A_anything.txt, B_anything.txt, ......

I want to execute a script, say 'CMD', on each of them, with the output files named like:

A_result.txt, B_result.txt, ......

In addition, at the first line of these output file, I want to have the file name of the original one.

The 'find -exec' util seems to me unable to extract part of the file name.

Does someone know a solution to this problem, by any means(shell, python, find,etc)? Thank you!

Zsolt Botykai · Accepted Answer · 2011-01-21 14:24:42Z

2

cd /directory
for file in *.txt ; do
    newfilename=`echo "$file"|sed 's/\(.\+\)_.*/\1_result.txt/`
    echo "$file" > "$newfilename" 
    your-command $file >> "$newfilename"
done

HTH

edited Jan 21, 2011 at 14:24

answered Jan 21, 2011 at 13:39

Zsolt Botykai

52k14 gold badges90 silver badges111 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jonathan Leffler Over a year ago

+1 despite the niggling that follows. Arguably, it should be *.txt or even [A-Z]_*.txt in the for line, but that will work if the files are all in a single directory. The your-command line probably needs "$files" too. I would use a singular name $file rather than the plural $files since at any one time it contains a single file name, not several.

Jonathan Leffler · Accepted Answer · 2011-01-21 14:35:45Z

1

Well, there's more than one way to do it (including using Perl, where that's the motto), but probably I'd write it like this:

find . -name '[A-Z]_*.txt' -type f -print0 |
    xargs -0 modify_rename.sh

And then I'd write the script modify_rename.sh like this:

#!/bin/sh
for file in "$@"
do
    dirname=$(dirname "$file")
    basename=$(basename "$file" .txt)
    leadname=${file%_*}
    outname="$dirname/${leadname}_result.txt"
    # Optionally check for pre-existence of $outname
    {
    # Optionally echo "$basename.txt" instead of "$file"
    echo "$file"
    # Does this invocation of CMD write to standard output?
    # If not, adjust invocation appropriately.
    CMD "$file"
    } > "$outname"
done

The advantage of this separation into separate scripting operations is that the rename/modify operation can be checked out separately from the search process - which runs less risk of zapping your entire directory structure with bad commands.

Bash has the tools to avoid invoking basename and dirname but the notation is moderatly excruciating; I find the clarity of the command names worth having. I'd be happy if bash implemented them as built-ins. There are plenty of other ways to get the prefix of the file; this should be safe, though, even in the presence of spaces (tabs, newlines) in file or directory names because of the careful use of double quotes.

edited Jan 21, 2011 at 14:35

answered Jan 21, 2011 at 13:57

Jonathan Leffler

759k145 gold badges961 silver badges1.3k bronze badges

2 Comments

ghostdog74 Over a year ago

notation wise, once one get used to the syntax, it becomes easy.

Jonathan Leffler Over a year ago

@ghostdog74: and since I learned the basename and dirname notations more than a quarter century ago, they are trivially easy and completely clear to me, and the johnny-come-lately new-fangled notations (which are only 15 years old, but were not available in every shell) are not clear; so, I stick with my preferred dialect, aware that youngsters will likely prefer an alternative dialect, and therefore pointing out TMTWTDI.

Collectives™ on Stack Overflow

Shell script: execute cmd on a file, with additional processing of file name

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related