Despite the comments, there's no direct problem with tcsh (and believe me, I'm no fan of C shell), nor is there a problem with bash per se. The problems would be similar, in fact, if you replaced the tcsh with bash.
The problem is that what you're trying to do is actually extremely difficult to do. Let me explain...
In the bash script, you are trying to create a single string that will contain a valid command line that tcsh will interpret correctly, including preserving spaces in arguments.
Developing an answer step-by-step
Let's start with some easy stuff — arguments without spaces in them:
set -- /bin/ls /bin/sh /bin/bash # Set the arguments to bash
/bin/tcsh -c "ls -l $*"
This will work fine; it will execute the C shell and the C shell will process the string and execute:
ls -l /bin/ls /bin/sh /bin/bash
So, the problem is how to relay arguments with spaces in them to C shell reliably when the command as a whole is being specified as a single string.
You already know that this runs into problems:
mkdir "./a b c" "./d e f"
set -- "a b c" "d e f" # Two arguments with spaces
/bin/tcsh -c "ls -al $*"
On my machine, I get:
ls: a: No such file or directory
ls: b: No such file or directory
ls: c: No such file or directory
ls: d: No such file or directory
ls: e: No such file or directory
ls: f: No such file or directory
If we do the expansion manually, we can get the desired result (for this limited example) with:
mkdir "./a b c" "./d e f"
set -- "a b c" "d e f" # Two arguments with spaces
/bin/tcsh -c "ls -al 'a b c' 'd e f'"
This yields:
a b c:
total 0
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
drwxr-xr-x 4 jleffler staff 136 Aug 25 12:21 ..
d e f:
total 0
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
drwxr-xr-x 4 jleffler staff 136 Aug 25 12:21 ..
(I'm going to assume that the two directories 'a b c' and 'd e f' exist from here onwards without creating them each time.)
So, the objective must be to find a way to create a string that will be safe when interpreted by the C shell, automatically (not manually as shown). Because of the metasyntactic zoo that C shell has (lots of special characters), the full task will be hard, but let's get the easy stuff done first — spaces and no metacharacters.
For each argument, we want to add single quotes to the start and end, and ensure that any single quotes inside the string are protected. That is its own little party; the trick is to replace the embedded single quotes with the sequence '\'' where the first single quote ends the current single-quoted string, the backslash single-quote embeds a single quote, and the final single quote starts a new single-quoted string. And we want that added to the end of the current command string. So, this leads to:
set -- "a b c" "d e f" # Two arguments with spaces
cmd="ls -al"
for arg in "$@"
do escaped=$(sed -e "s/'/'\\''/g" -e "s/^/'/" -e "s/$/'/" <<< "$arg")
cmd="$cmd $escaped"
done
echo "$cmd"
tcsh -c "$cmd"
This yields (the ls line is from the echo, of course):
ls -al 'a b c' 'd e f'
a b c:
total 0
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
drwxr-xr-x 4 jleffler staff 136 Aug 25 12:21 ..
d e f:
total 0
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
drwxr-xr-x 4 jleffler staff 136 Aug 25 12:21 ..
OK, so far, so good. What about the metasyntactic zoo? Fortunately, most of the characters have no special meaning inside single quotes.
Time to add some more complex directories to the list (these will survive for the duration of the question, too). Make sure you know what names are being created; you need to understand shell quoting rather well.
As an exercise, for each directory name created during this question, write alternatives that give the same result when enclosed in single quotes, when enclosed in double quotes, and without any quotes around the whole argument.
$ mkdir '! % *' '$(pwd)' '`pwd`'
And the script is mostly unchanged — it uses the shell glob to generate the list of directory names, echoes each argument in turn, and lists the inode numbers too:
set -- *
cmd="ls -ail"
for arg in "$@"
do echo "arg: $arg"
escaped=$(sed -e "s/'/'\\''/g" -e "s/^/'/" -e "s/$/'/" <<< "$arg")
cmd="$cmd $escaped"
done
echo "cmd: $cmd"
tcsh -c "$cmd"
Hey presto:
arg: ! % *
arg: $(pwd)
arg: `pwd`
arg: a b c
arg: d e f
cmd: ls -ail '! % *' '$(pwd)' '`pwd`' 'a b c' 'd e f'
! % *:
total 0
1640119 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 .
1640040 drwxr-xr-x 7 jleffler staff 238 Aug 25 12:34 ..
$(pwd):
total 0
1640120 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 .
1640040 drwxr-xr-x 7 jleffler staff 238 Aug 25 12:34 ..
`pwd`:
total 0
1640121 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 .
1640040 drwxr-xr-x 7 jleffler staff 238 Aug 25 12:34 ..
a b c:
total 0
1640056 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
1640040 drwxr-xr-x 7 jleffler staff 238 Aug 25 12:34 ..
d e f:
total 0
1640057 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
1640040 drwxr-xr-x 7 jleffler staff 238 Aug 25 12:34 ..
Just what the doctor ordered! But we haven't yet been brutal enough: like Knuth says, you have to get into a really nasty mean mindset when you're testing code, so let's try:
$ mkdir "O'Reilly's Books"
$ mkdir "' \` \""
$ mkdir '${HOME}' '$PATH' 'He said, "Don'\''t Do It!"'
$ ls -l
total 0
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 ! % *
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 $(pwd)
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 $PATH
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 ${HOME}
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 ' ` "
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 He said, "Don't Do It!"
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 O'Reilly's Books
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 `pwd`
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 a b c
drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 d e f
$
And the result is:
arg: ! % *
arg: $(pwd)
arg: $PATH
arg: ${HOME}
arg: ' ` "
arg: He said, "Don't Do It!"
arg: O'Reilly's Books
arg: `pwd`
arg: a b c
arg: d e f
cmd: ls -ail '! % *' '$(pwd)' '$PATH' '${HOME}' '''' ` "' 'He said, "Don'''t Do It!"' 'O'''Reilly'''s Books' '`pwd`' 'a b c' 'd e f'
Unmatched `.
That's not what we wanted. Part of the trouble, though, is that sequence of 4 single quotes in the line tagged 'cmd:'; it should be ''\''. So, the sed script isn't accurate enough.
set -- *
cmd="ls -ail"
for arg in "$@"
do echo "arg: $arg"
escaped=$(sed -e "s/'/'\\\\''/g" -e "s/^/'/" -e "s/$/'/" <<< "$arg")
cmd="$cmd $escaped"
done
echo "cmd: $cmd"
tcsh -c "$cmd"
And when it is run, we get:
arg: ! % *
arg: $(pwd)
arg: $PATH
arg: ${HOME}
arg: ' ` "
arg: He said, "Don't Do It!"
arg: O'Reilly's Books
arg: `pwd`
arg: a b c
arg: d e f
arg: x.sh
cmd: ls -ail '! % *' '$(pwd)' '$PATH' '${HOME}' ''\'' ` "' 'He said, "Don'\''t Do It!"' 'O'\''Reilly'\''s Books' '`pwd`' 'a b c' 'd e f' 'x.sh'
1640231 -rw-r--r-- 1 jleffler staff 223 Aug 25 12:56 x.sh
! % *:
total 0
1640119 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
$(pwd):
total 0
1640120 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
$PATH:
total 0
1640176 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
${HOME}:
total 0
1640175 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
' ` ":
total 0
1640163 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
He said, "Don't Do It!":
total 0
1640177 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
O'Reilly's Books:
total 0
1640164 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
`pwd`:
total 0
1640121 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
a b c:
total 0
1640056 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
d e f:
total 0
1640057 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 .
1640040 drwxr-xr-x 13 jleffler staff 442 Aug 25 12:56 ..
Mean enough? Getting close. What about directory names containing backslashes?
$ mkdir "a \\' \\\` \\$ b \\\" c" # Make sure you do the exercise!
$ mkdir 'a \\'\'' \\\` \\$ b \\\" c' # Make sure you do the exercise!
$ ls -li
total 8
1640119 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 ! % *
1640120 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 $(pwd)
1640176 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 $PATH
1640175 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 ${HOME}
1640163 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 ' ` "
1640177 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 He said, "Don't Do It!"
1640164 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 O'Reilly's Books
1640121 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 `pwd`
1640243 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:58 a \' \` \$ b \" c
1640259 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:01 a \\' \\\` \\$ b \\\" c
1640056 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 a b c
1640057 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 d e f
1640231 -rw-r--r-- 1 jleffler staff 223 Aug 25 12:56 x.sh
$
And with the ls -ail changed to ls -dil, the output is:
$ bash x.sh
arg: ! % *
arg: $(pwd)
arg: $PATH
arg: ${HOME}
arg: ' ` "
arg: He said, "Don't Do It!"
arg: O'Reilly's Books
arg: `pwd`
arg: a \' \` \$ b \" c
arg: a \\' \\\` \\$ b \\\" c
arg: a b c
arg: d e f
arg: x.sh
cmd: ls -dil '! % *' '$(pwd)' '$PATH' '${HOME}' ''\'' ` "' 'He said, "Don'\''t Do It!"' 'O'\''Reilly'\''s Books' '`pwd`' 'a \'\'' \` \$ b \" c' 'a \\'\'' \\\` \\$ b \\\" c' 'a b c' 'd e f' 'x.sh'
1640119 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 ! % *
1640120 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 $(pwd)
1640176 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 $PATH
1640175 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 ${HOME}
1640163 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 ' ` "
1640177 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 He said, "Don't Do It!"
1640164 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 O'Reilly's Books
1640121 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 `pwd`
1640243 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:58 a \' \` \$ b \" c
1640259 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:01 a \\' \\\` \\$ b \\\" c
1640056 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 a b c
1640057 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 d e f
1640271 -rw-r--r-- 1 jleffler staff 223 Aug 25 13:03 x.sh
$
Working Script
set -- *
cmd="ls -ail"
for arg in "$@"
do echo "arg: $arg"
escaped=$(sed -e "s/'/'\\\\''/g" -e "s/^/'/" -e "s/$/'/" <<< "$arg")
cmd="$cmd $escaped"
done
echo "cmd: $cmd"
tcsh -c "$cmd"
Summary
The key parts to the solution are:
- Recognizing that single quotes are needed around the arguments.
- Knowing how to escape single quotes.
- Knowing how to escape backslashes.
- Being really brutal when you do your testing!
- It helps if you've done it before...
Oh futz! I forgot to test arguments containing newlines:
$ mkdir "a
> b
> c"
$ ls -li
total 8
1640119 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 ! % *
1640120 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 $(pwd)
1640176 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 $PATH
1640175 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 ${HOME}
1640163 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 ' ` "
1640177 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 He said, "Don't Do It!"
1640164 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 O'Reilly's Books
1640121 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 `pwd`
1640336 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:16 a?b?c
1640243 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:58 a \' \` \$ b \" c
1640259 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:01 a \\' \\\` \\$ b \\\" c
1640056 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 a b c
1640057 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:21 d e f
1640271 -rw-r--r-- 1 jleffler staff 223 Aug 25 13:03 x.sh
$
Well, there are some reasons why you should never try to parse the output from ls; it has generated question marks in place of the newlines (this is on Mac OS X 10.8.1 and is not GNU ls, just for those keeping score at home; other systems may behave differently).
And when the script (x.sh) is run, I get:
$ bash x.sh
arg: ! % *
arg: $(pwd)
arg: $PATH
arg: ${HOME}
arg: ' ` "
arg: He said, "Don't Do It!"
arg: O'Reilly's Books
arg: `pwd`
arg: a
b
c
arg: a \' \` \$ b \" c
arg: a \\' \\\` \\$ b \\\" c
arg: a b c
arg: d e f
arg: x.sh
cmd: ls -dil '! % *' '$(pwd)' '$PATH' '${HOME}' ''\'' ` "' 'He said, "Don'\''t Do It!"' 'O'\''Reilly'\''s Books' '`pwd`' 'a'
'b'
'c' 'a \'\'' \` \$ b \" c' 'a \\'\'' \\\` \\$ b \\\" c' 'a b c' 'd e f' 'x.sh'
ls: a: No such file or directory
1640119 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 ! % *
1640120 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 $(pwd)
1640176 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 $PATH
1640175 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 ${HOME}
1640163 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 ' ` "
1640177 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:45 He said, "Don't Do It!"
1640164 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:43 O'Reilly's Books
1640121 drwxr-xr-x 2 jleffler staff 68 Aug 25 12:34 `pwd`
b: Command not found.
c: Command not found.
$
There are multiple issues here. The sed script treated each line of the argument separately. That really isn't soluble using sed; or, perhaps more accurately, it isn't something I want to solve using sed. Æons ago, I wrote a C program escape to do the job that the sed script almost does.
#!/bin/bash
set -- *
escaped=$(escape "$@")
cmd="ls -dil $escaped"
echo "cmd: $cmd"
bash -c "$cmd"
tcsh -c "$cmd"
Note that I've added an invocation of bash in there. The output is:
cmd: ls -dil '! % *' '$(pwd)' '$PATH' '${HOME}' ''\'' ` "' 'He said, "Don'\''t Do It!"' 'O'\''Reilly'\''s Books' '`pwd`' 'a
b
c' 'a b c' 'd e f' x.sh
178474064 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 ! % *
178474065 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 $(pwd)
178474219 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 $PATH
178474218 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 ${HOME}
178474170 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 ' ` "
178474220 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 He said, "Don't Do It!"
178474131 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 O'Reilly's Books
178474066 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 `pwd`
178474998 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:40 a?b?c
178473958 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 a b c
178473959 drwxr-xr-x 2 jleffler staff 68 Aug 25 13:38 d e f
178475097 -rw-r--r-- 1 jleffler staff 115 Aug 25 13:41 x.sh
Unmatched '.
b: Command not found.
Unmatched '.
Wassup? Well, bash and other shells derived from the Bourne shell such as ksh, are OK with a string starting on one line and continuing over other lines, but the C shell and its derivatives are not. They demand a backslash before the newline. So, to work with tcsh, I'd have to upgrade escape to generate the output for the C shell. Not at all hard to do, but it would need doing. Presumably, that would be an option -c and for general purpose safety, the invocation would become:
escaped=$(escape -c -- "$@")
with the double-dash preventing misinterpretation of arguments in "$@" as options to escape itself. In part, this goes to show that it is hard to write scripts that deal with file names that contain characters outside the portable file name character set. Fortunately, I don't have to deal with the C shell very often; I don't plan to make that a part of escape because it is a change of interface (the current code does not have any options of its own, so I do not use the double-dash notation with escape). If I need it, it will become cescape to unconditionally support the C shell.
first file.