1

I have a source file where I use a macro to do logging. Typical logging lines look like:

STDOUT_LOG(logDEBUG4) << "a single line log text" << aVariable;

the exist some multiline log commands as well:

        STDOUT_LOG(logDEBUG4)
            << bsd->sensorName
            << " trainMinValueInWin = " << value.trough
            << " trainMaxValueInWin = " << value.peak
            << " |";

The idea is to grab the log string (whether it is in single line or multiple lines) and place them in the parenthesis of the macro.

Typically the end result would be something like:

 STDOUT_LOG(logDEBUG4,"a single line log text" << aVariable);

or

        STDOUT_LOG(logDEBUG4,
            bsd->sensorName
            << " trainMinValueInWin = " << value.trough
            << " trainMaxValueInWin = " << value.peak
            << " |");

I am trying to read up on anything related to multiline matching, including while loops in perl, the /s amd /m operators and potential ways to search and replace, but I am very far from correctly matching the multiline macro. I assume I need the /s operator in order for the "." to match newline characters too, but I have not been able to successfully use it.

The beginning of the pattern is of course "STDOUT_LOG" and the end is the first occurrence of ";" however many lines later it shows up.

this attempt

 perl -ne 's/STDOUT_LOG\((.*?)\)(.*?)\;/STDOUT_LOG($1,$2)\;/gs; print;'

manages to pick up the log level ("logDEBUG4" in this case) is "sort" of OK for single line log commands, but matching of the rest of the lines fails even though I used the /s in the end.

How do I approach this problem? what other operators should I use? any good tutorials out there? ultimately is there a perl command that can save the day?

EDIT: since none of the suggestions work out of the box for me (the pattern matching fails and as a result nothing is changed), I wonder if there is some newer functionality involved here. In any case my perl version is 5.18.2. I am trying to run the commands in a linux cli as follows:

perl -ne 's/pattern/replace/gs; print;' <filename>

I hope I am not doing something wrong here

1
  • If "none of the suggestions work" I'd guess that you haven't explained the data well enough. What you show works in my answer, with the script and with the one-liner. For example -- do you have that one segment of data (one line or mulitple lines, but one log) in one file? Or do you have more logs in a single file? Please check the data you show, is it indeed what you have? I copy-pasted your example into a file and wrote my answer with it. Commented Nov 29, 2016 at 3:29

4 Answers 4

2

With $log string containing the whole log, with newlines and all,

$log =~ s/STDOUT_LOG\K \( ( [^)]+ ) \) ( [^;]+ );/($1,$2);/xs;

The [^)]+ matches up to the first closing parentheses, and likewise [^;]+ goes to the first ;. You do not need nor want the /g modifier. The /x allows spaces for readability.

The \K is a particular form of the positive lookbehind, which also discards all previous matches. So the pattern before it, used as an assertion, is not affected by the substitution. See it in perlretut. It is not essential here, but if you leave it out then STDOUT_LOG is consumed by the match and so it need be typed in as well in the replacement part. It's just cleaner this way.

Since there could be a problem with how the log is input here is a complete program

use warnings 'all';
use strict;

my $logfile = 'log.txt';
my $log = do {
    local $/; 
    open my $fh, '<', $logfile or die "Can't open $logfile: $!";
    <$fh>;
};
# print $log;

$log =~ s/STDOUT_LOG\K \( ([^)]+) \) ([^;]+);/($1,$2);/xs;

print $log;

The do block allows us to slurp the whole file $logfile into the $log variable, by unsetting $INPUT_RECORD_SEPARATOR (or $/), within the block by local. See discussions in perlvar.


If this is meant to be done with a one-liner

perl -0777 -pe 's/STDOUT_LOG\K \( ([^)]+) \) ([^;]+);/($1,$2);/xs;' log.txt

The -0777 makes it slurp the whole file into $_, and the substitution is by default done on it. The -p also prints $_ after each line is processed (and here we only have one "line" -- the whole file). See perlrun for command line switches and General Variables in perlvar for $_.


Both these assume that the shown example of a log (one-liner or the multi-line one) stands on its own, that there is a single log-string to process. For simplicity I took that to be in a file log.txt and read it from that file, into the $log variable in the script and into $_ in the one-liner.

Sign up to request clarification or add additional context in comments.

5 Comments

hi there, I am abit fuzzy about the (my $newlog = $log) =~ part. could you elaborate, please? Also, should I assume that a full single liner perl command would look like : perl -ne '(my $newlog = $log) =~ s/STDOUT_LOG\K \( ([^)]+) \) ([^;]+);/($1,$2);/xs;' <filename> ?
@nass I added some explanations, please look over it again. What you ask is an idiom of sorts -- the $log is copied into $newlog so that the substitution is done on that, leaving $log unchanged. It is not essential and I removed it, so now $log is changed in place. I added a one-liner version. Let me know how it goes now.
hi there and thank you for the help. nope unfortunately neither the one liner works. I wonder what needs escaping in bash...
@nass Why "escaping in bash" ... ? There is no relation of one-liner to that. What is inside '' is directly passed to the perl command, the shell doesn't touch it. If these are no good the question is about the data. Here is what I did. The script and one-liner work for a file log.txt containing literal text from your example. I copy-pasted STDOUT_LOG(logDEBUG4) and all following lines, concluding with << " |";. So the file log.txt has those five lines. Is this understanding of your data wrong?
@nass Just in case -- these aren't meant to be cout statements, right? For example, << " trainMinValueInWin = " << value.trough is just a literal line of text, a string of characters to process, right? This is how I understood it. So you have LOG(label) ...text...; which needs to become LOG(label, ...text...);
1

You can use a regex like this with s (single line flag):

\).*?<<(.*?);

And the replacement string:

, \1);

Working demo

8 Comments

hi there, if I use the above in perl -ne 's/\).*?<<(.*?);/$1/gs; print;' <filename> it does not work. Am I somehow not usng it right?
@nass the replacement string you are using is $1, the one I gave you is , $1);
hi there, but this only affects the replacement. For the moment the pattern matching fails - even though it works in the live demo. This is puzzling. So even with , $1); I still do not get the correct result
my perl version is : perl 5, version 18, subversion 2 (v5.18.2) hopefully there is no newer functionality used in you suggestion:). right?
Could you please try your working demo exactly in a linux cli and tell me if it runs fine for you with the command perl -ne '\).*?<<(.*?);/, \1);/gs; print; <text as in working demo> ? thank you
|
0

Here is a way to do it:

use Modern::Perl;

my @lines = (
'STDOUT_LOG(logDEBUG4) << "a single line log text" << aVariable;',
'        STDOUT_LOG(logDEBUG4)
            << bsd->sensorName
            << " trainMinValueInWin = " << value.trough
            << " trainMaxValueInWin = " << value.peak
            << " |";'
);
for (@lines) {
    s/(STDOUT_LOG\(.*?)\)(\s*)<<(.*?);/$1, $2$3);/gs;
    say;
}

Output:

STDOUT_LOG(logDEBUG4, "a single line log text" << aVariable);
        STDOUT_LOG(logDEBUG4, 
            bsd->sensorName
            << " trainMinValueInWin = " << value.trough
            << " trainMaxValueInWin = " << value.peak
            << " |");

Comments

0

In the end the command that successfully did what I wanted was the following:

perl -0777pne 's/STDOUT_LOG\((.*?)\)(.*?)<<(.*?);/STDOUT_LOG\(\1\,\2\3\);/gs' sensor.cpp

This is alot a superposition for the answers of "Federico Piazza" and "zdim". You'll notice I prepended the "STDOUT_LOG" bit in the front to really much exactly what I wanted in a large number of source files. I noticed that for correct matching one MUST include the /s operator at the end of the regex , but also add the -0777 flag in the perl executable.

see perlrun for an explanation of the runtime perl executable flags

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.