LInux shell: conditional conversion of character encoding, multiple text files

The other solutions don't care about the mixture of files, which sounds like a solution in the sense of:

for F in *.csv; do
    if [ `file -i "$F" | awk '{print $3;}'` = "charset=utf-16" ]; then
        iconv -f UTF-16 -t US-ASCII "$F" > "u.$F"
    fi
done

What makes it easier is the identity of us-ascii and utf-16 in the first few (128) characters - so if the file really is us-ascii, the conversion would not do any harm.

Pls try the following command:

iconv -f FROM-ENCODING -t TO-ENCODING *.csv

and replace FROM-ENCODING and TO-ENCODING with appropriate values.

You can use the following script, or something similar for your needs.

for file in  *.csv
do
    iconv -f FROM-ENCODING -t TO-ENCODING "$file" > "$file.new"
done

You can also use recode command.

recode FROM-ENCODING..TO-ENCODING file.csv

Finally, look at this Best way to convert text files between character sets? if you are interested in learning more about iconv and/or recode

This will convert any non-us-ascii encoded *.csv files to us-ascii:

#!/bin/bash
for f in *.csv;do
    charset=`file -i README.md |grep -o 'charset=.*'|cut -d= -f2`
    if [ "$charset" != "us-ascii" ];then
      echo "$f $charset -> us-ascii"
      iconv -f "$charset" -t us-ascii < "$f" > "$f.tmp" \
        && mv "$f.tmp" "$f"
    fi
done