Change a list of strings to lowercase

Question

I have two files, one file contains a list of strings.

+stringa +Dog +Cat
+cat +Tux +elephant

and the second file (csv) contains something like:

"123456 Abc","+Stringx +123","something"
"23456 dEf","+cat +Tux +elephant","Other something"
"34524 xyz","+stringa +Dog +Cat","third something"

the result should be:

"123456 Abc","+Stringx +123","something"
"23456 dEf","+cat +tux +elephant","Other something"
"34524 xyz","+stringa +dog +cat","third something"

How I can change the strings, that match my list of patterns, to lowercase?

My comma-separated values file have about 30 columns and about 1500 rows.

can you add some code you tried? what tools do you have? GNU sed? perl? awk? — Sundeep
– Sundeep, Commented Nov 27, 2017 at 10:34
Despite newer versions having an upper- & lower-case variable operator, bash is not a text editor. — Jeff Schaller
– Jeff Schaller ♦, Commented Nov 27, 2017 at 10:48
@Sundeep I can not find the right approach. I do not understand perl but it is available. I use Debian 8.9 (Jessie). The point is i like change only the pattern, not the whole file. I have sed, awk, tr — FaxMax
– FaxMax, Commented Nov 27, 2017 at 10:54

Sundeep · Accepted Answer · 2017-11-28 10:27:49Z

With GNU sed, assumes that you do not have any meta character in list of strings, + is not a meta character with default BRE

$ # create substitute command for each line
$ sed 's/.*/s|"&"|\\L\&|gi/' f1
s|"+stringa +Dog +Cat"|\L&|gi
s|"+cat +Tux +elephant"|\L&|gi

$ # pass those commands as sed script
$ sed -f <(sed 's/.*/s|"&"|\\L\&|gi/' f1) ip.csv
"123456 Abc","+Stringx +123","something"
"23456 dEf","+cat +tux +elephant","Other something"
"34524 xyz","+stringa +dog +cat","third something"

$ # or save them in a file and use
$ sed 's/.*/s|"&"|\\L\&|gi/' f1 > f2
$ sed -f f2 ip.csv

\L to convert string to lowercase
g for replacing all occurrences in a line, i for case-insensitive matching

If you don't have GNU sed

$ # \Q to quote metacharacters
$ # but will have issues if you have \ or $ or @
$ sed 's/.*/s|\\Q"&"|\\L$\&|gi;/' f1
s|\Q"+stringa +Dog +Cat"|\L$&|gi;
s|\Q"+cat +Tux +elephant"|\L$&|gi;

$ perl -p <(sed 's/.*/s|\\Q"&"|\\L$\&|gi;/' f1) ip.csv 
"123456 Abc","+Stringx +123","something"
"23456 dEf","+cat +tux +elephant","Other something"
"34524 xyz","+stringa +dog +cat","third something"

As noted by Stéphane Chazelas, this could lead to code injection vulnerabilities if contents of f1 is not under control

It may be worth noting that those amount to command injection vulnerabilities if the content of f1 is not under your control. — Stéphane Chazelas
– Stéphane Chazelas, Commented Nov 27, 2017 at 11:23

Stéphane Chazelas · Accepted Answer · 2017-11-27 14:07:52Z

With perl, assuming you want each word in the first file to be turned to lowercase:

perl -pe '
 BEGIN {local $/ = undef; $regex = join "|", map qr{\Q$_\E}i, split " ", <>}
 s/$regex/\L$&/g' file1.words file2.csv

local $/ = undef makes the record separator for the BEGIN block undefined so that the one invocation of <> there, slurps the whole first file (file1.words) in. We split that on whitespace (split " " is special in perl in the same way as awk -F " " is in awk), and join the resulting words with | after having regex-quoted them and made them case insensitive.

So we have a huge regexp that is something like (?i:word1)|(?i:word2)|... which we apply on each line of the second file in the rest of the code.

If it's each string in each line of the first file, then that can be simplified to:

perl -pe '
 BEGIN {chomp (@strings = <STDIN>); $regex = join "|", map qr{\Q$_\E}i, @strings}
 s/$regex/\L$&/g' < file1.strings file2.csv

There, we open the first file on stdin instead of passing it as argument. <STDIN> returns a list of its lines from which we remove the delimiters with chomp, and join with | as above.

If you don't want it to be limited to ASCII characters, add the -Mopen=locale option.

@Sundeep. That was my initial interpretation, but now that you mention it, you're probably right the lines of file1 are probably to be the strings to be matched. See edit. — Stéphane Chazelas
– Stéphane Chazelas, Commented Nov 27, 2017 at 11:45

RomanPerekhrest · Accepted Answer · 2017-11-27 15:39:59Z

AWK solution (for your current input):

Assuming that the 2nd field is of main interest and values in search file are double-quoted.

awk 'NR==FNR{ $0="\042"$0"\042"; a[$0]; next }
     $2 in a{ $2=tolower($2) }1' patterns FS=',' OFS=',' file.csv

$0="\042"$0"\042" - wrap a pattern line with double quotes while iterating through the lines of patterns file
a[$0] - capturing a pattern line into array a
$2 in a{ $2=tolower($2) } - if the 2nd field value from line of file.csv file is in the list of patterns(i.e. array a) - convert all characters in it to lowercase $2=tolower($2)

The output:

"123456 Abc","+Stringx +123","something"
"23456 dEf","+cat +tux +elephant","Other something"
"34524 xyz","+stringa +dog +cat","third something"

Stack Exchange Network

Change a list of strings to lowercase

3 Answers 3

You must log in to answer this question.

Hot Network Questions

Change a list of strings to lowercase

3 Answers 3

You must log in to answer this question.

Related

Hot Network Questions