how do I parse text file into variables using regex c++?

Question

Please help me fulfill my dreams of turning this sequence into a meaningful output. :)

See regex in action, it works!: http://regex101.com/r/iM4yN2/1 Now all I need is to know how to use it. If I could put this into a multidimensional array e.g. configFile[0][0] = [Tuner,] that would work. Or if I could turn this into a comma separated list, I could then parse that again and put it into arrays and finally out to individual variables. Anyway, you don't need to spell out how to actually assign the variables, I'll create another question if I really need help with that. Mainly I need help with the use of regex functions and outputting data into SOME variable where I can access the various text on either side of the = sign per line.

regex:

^[\t ]*(.*?)\s*=[\t ]*(.*?)(#.*)?$

test string:

    ### MODULES ###
Tuner         =  
 PitchDetector = 0
PhaseLocker   = 0
FileOutput    = 1

### FILE MANAGER ###
RenameFile_AvgFreq  =  dfgsdfg dsf gdfs g #gdrgk
RenameFile_NoteName = 0
    RenameFile_Prefix   = "The String Is Good"
RenameFile_Suffix   = ""
OutputFolder        = "..\Folder\String\"

### PITCH DETECTOR ###
AnalysisChannel = 1  #int starting from 1
BlockSize             = 8  #power of 2
Overlap               = 16 #power of 2
NormalizeForDetection = 0

### TUNER ###
Smoothing = 0.68
Envelope  = 0.45

### PHASELOCKER ###
FFTSize    = 1024 #powert of 2
FFTOverlap = 54687
WindowType = 0
MaxFreq    = 5000

my variables:

//Modules
bool Tuner;
bool PitchDetector;
bool PhaseLocker;
bool FileOutput;

//File Manager
bool RenameFile_AvgFreq;
bool RenameFile_NoteName;
std::string RenameFile_Prefix;
std::string RenameFile_Suffix;
std::string OutputFolder;

//Pitch Detector
int AnalysisChannel;
int BlockSize;
int Overlap;
bool NormalizeForDetection;

//Tuner
float Smoothing;
float Envelope;

//Phaselocker
int FFTSize;
int FFTOverlap;
int FFTWindowType;
float FFTMaxFreq;

final notes: i spent a long time looking at c++ regex functions... very confusing stuff. I know how to do this in python without thinking twice.

It might be possible to use an "ini" file parsing library for this, rather than your own regex. — John Zwinck
– John Zwinck, Commented Oct 24, 2014 at 1:00
I agree with @JohnZwinck. This is not really what regex was designed for (though it may well work). configuration files are structured (whether they be .INI, .XML, or even .JSON), meaning they are quickly and easily parsed and sense-checked. — Moo-Juice
– Moo-Juice, Commented Oct 24, 2014 at 9:42
Ok, I will use ini. I'm going to try the "inih" library. If no one answers clearly step by step instructions to use a given ini parser, then I shall write a new answer for this post once I learn it. — Elan Hickler
– Elan Hickler, Commented Oct 24, 2014 at 11:27
Alright I'm back, I am still using regex! I'm satisfied with my approach and I hope this helps someone else. — Elan Hickler
– Elan Hickler, Commented Oct 25, 2014 at 1:05

Elan Hickler · Accepted Answer · 2015-12-27 04:01:25Z

Include the following:

#include <string>
#include <regex>

Declare a string and regex type:

std::string s;
std::regex e;

In your main function, assign string and regex variables and call regex function (you could assign the variables when you declare them as well):

int main()
{
    s="i will only 349 output 853 the numbers 666"
    e="(\\d+)"
    s = std::regex_replace(s, e, "$1\n", std::regex_constants::format_no_copy);

    return 0;
}

Notice how I am putting the results right back into the string (s). Of course, you could use a different string to store the result. The "std::regex_constants::format_no_copy" is a flag that tells the regex function to output only "substrings" aka group matches. Also notice how I am using double slash on the "\d+". Try double slashes if your regex pattern isn't working.

To find key/value pairs with regex, e.g. "BlockSize = 1024", you could create a pattern such as:

BlockSize\s*=\s*((?:[\d.]+)|(?:".*"))

in c++ you could create that regex pattern with:

expr = key+"\\s*=\\s*((?:[\\d.]+)|(?:\".*\"))";

and return the match with:

config = std::regex_replace(config, expr, "$1", std::regex_constants::format_no_copy);

and put it all together in a function with the ability to return a default value:

std::string Config_GetValue(std::string key, std::string config, std::string defval)
{
    std::regex expr;
    match = key+"\\s*=\\s*((?:[\\d.]+)|(?:\".*\"))";
    config = std::regex_replace(config, expr, "$1", std::regex_constants::format_no_copy);
    return config == "" ? defval : config;
}

FULL CODE (using std::stoi and std::stof to convert string to number when needed, and using auto type because right-hand side (RHS) makes it clear what the type is):

#include "stdafx.h"
#include <string>
#include <regex>
#include <iostream>

std::string Config_GetValue(std::string key, std::string config, std::string defval)
{
    std::regex expr;
    match = key+"\\s*=\\s*((?:[\\d.]+)|(?:\".*\"))";
    config = std::regex_replace(config, expr, "$1", std::regex_constants::format_no_copy);
    return config == "" ? defval : config;
}


int main()
{
    //test string
    std::string s = "    ### MODULES ###\nTuner         =  \n PitchDetector = 1\n PhaseLocker = 0 \nFileOutput    = 1\n\n### FILE MANAGER ###\nRenameFile_AvgFreq  =  dfgsdfg dsf gdfs g #gdrgk\nRenameFile_NoteName = 0\n    RenameFile_Prefix   = \"The String Is Good\"\nRenameFile_Suffix   = \"\"\nOutputFolder        = \"..\\Folder\\String\\\"\n\n### PITCH DETECTOR ###\nAnalysisChannel = 1  #int starting from 1\nBlockSize             = 1024  #power of 2\nOverlap               = 16 #power of 2\nNormalizeForDetection = 0\n\n### TUNER ###\nSmoothing = 0.68\nEnvelope  = 0.45\n\n### PHASELOCKER ###\nFFTSize    = 1024 #powert of 2\nFFTOverlap = 54687\nWindowType = 0\nMaxFreq    = 5000";

    //Modules   
    auto FileOutput    = stoi(Config_GetValue("FileOutput", s, "0"));
    auto PitchDetector = stoi(Config_GetValue("PitchDetector", s, "0"));
    auto Tuner         = stoi(Config_GetValue("Tuner", s, "0"));
    auto PhaseLocker   = stoi(Config_GetValue("PhaseLocker", s, "0"));

    //File Manager
    auto RenameFile_AvgFreq  = stoi(Config_GetValue("RenameFile_AvgFreq", s, "0"));
    auto RenameFile_NoteName = stoi(Config_GetValue("RenameFile_NoteName", s, "0"));
    auto RenameFile_Prefix   = Config_GetValue("RenameFile_Prefix", s, "");
    auto RenameFile_Suffix   = Config_GetValue("RenameFile_Suffix", s, "");
    auto OutputFolder        = Config_GetValue("FileOutput", s, "");

    //Pitch Detector
    auto AnalysisChannel       = stoi(Config_GetValue("AnalysisChannel", s, "1"));
    auto BlockSize             = stoi(Config_GetValue("BlockSize", s, "4096"));
    auto Overlap               = stoi(Config_GetValue("Overlap", s, "8"));
    auto NormalizeForDetection = stoi(Config_GetValue("NormalizeForDetection", s, "0"));

    //Tuner 
    auto Smoothing     = stof(Config_GetValue("Smoothing", s, ".5"));
    auto Envelope      = stof(Config_GetValue("Envelope", s, ".3"));
    auto TransientTime = stof(Config_GetValue("TransientTime", s, "0"));

    //Phaselocker   
    auto FFTSize       = stoi(Config_GetValue("FFTSize", s, "1"));
    auto FFTOverlap    = stoi(Config_GetValue("FFTOverlap", s, "1"));
    auto FFTWindowType = stoi(Config_GetValue("FFTWindowType", s, "1"));
    auto FFTMaxFreq    = stof(Config_GetValue("FFTMaxFreq", s, "0.0"));

    std::cout << "complete";
    return 0;
}

Could you post some information on this question about string literals and what they could help me do?
string literals will save you from needing to do the double backslash for the regexp metacharacters. See e.g. stackoverflow.com/questions/15998482/regex-w-escaping-in-c11
"oh, and using auto type because I read that you should" - not a good reason! You listed which types your vars should be, then replaced all with int from stoi - which might work for you but is not a good default recommendation. To me, auto should only be used when you know it'll do exactly what you need - not as a default. (FWIW, I only really use it in cases where the resulting type is clearly shown by the RHS - e.g. when I'm cast<>ing - or fetching from a container defined within eyeshot, etc. This gets me less redundant code but not at the expensive of adding confusion or ambiguity)

Elan Hickler · Accepted Answer · 2014-10-28 01:20:43Z

Another way of doing this is with regex_iterator:

#include <regex>
using std::regex;
using std::sregex_iterator;

void CreateConfig(string config)
{
    //group 1,2,3,4,5 = key,float,int,string,bool
    regex expr("^[\\t ]*(\\w+)[\\t ]*=[\\t ]*(?:(\\d+\\.+\\d+|\\.\\d+|\\d+\\.)|(\\d+)|(\"[^\\r\\n:]*\")|(TRUE|FALSE))[^\\r\\n]*$", std::regex_constants::icase);
    for (sregex_iterator it(config.begin(), config.end(), expr), itEnd; it != itEnd; ++it)
    {
        if ((*it)[2] != "") cout << "FLOAT -> " << (*it)[1] << " = " <<(*it)[2] << endl;
        else if ((*it)[3] != "") cout << "INT -> " << (*it)[1] << " = " <<(*it)[3] << endl;
        else if ((*it)[4] != "") cout << "STRING -> " << (*it)[1] << " = " <<(*it)[4] << endl;
        else if ((*it)[5] != "") cout << "BOOL -> " << (*it)[1] << " = " << (*it)[5] << endl;
    }
}

int main()
{   
    string s = "what = 1\n: MODULES\nFileOutput = \"on\" :bool\nPitchDetector = TRuE :bool\nTuner = on:bool\nHarmSplitter = off:bool\nPhaseLocker = on\n\nyes\n junk output = \"yes\"\n\n: FILE MANAGER\nRenameFile  AvgFreq  = 1 \nRenameFile_NoteName = 0 :bool\nRenameFile_Prefix   = \"The Strin:g Is Good\" :string\nRenameFile_Suffix   = \"\":string\nOutputFolder        = \"..\\Folder\\String\\\" :relative path\n\n: PITCH DETECTOR\nAnalysisChannel       = 1  :integer starting from 1\nBlockSize             = 8  :power of 2\nOverlap               = 16 :power of 2\nNormalizeForDetection = 0  :bool\n\n: TUNER\nSmoothing = 0.68 :float\nEnvelope  = 0.45 :float\n\n: PHASE LOCKER\nFFTSize    = 1024  :power of 2\nFFTOverlap = 54687 :power of 2\nWindowType = 0     :always set to 0\nMaxFreq    = 5000  :float";
    CreateConfig(s);

    return 0;
}

Let's break this down. The regex expression I created uses a ^regexy stuff goes here$ format so that each line of text is considered individually: ^=start of line, $=end of line. The regex looks for: variable_name = decimal OR number OR string OR (true OR false). Because each type is stored in its own group, we know what type every match is going to be.

To explain the for loop, I will write the code a few different ways

//You can declare more than one variable of the same type:
for (sregex_iterator var1(str.begin(), str.end(), regexExpr), var2); var1 != var2; var1++)

//Or you can delcare it outside the for loop:
sregex_iterator var1(str.begin(), str.end(), regexExpr);
sregex_iterator var2;
for (; var1 != var2; var1++)

//Or the more classic way:
sregex_iterator var1(str.begin(), str.end(), regexExpr);
for (sregex_iterator var2; var1 != var2; var1++)

Now for the body of the for loop. It says "If group2 is not blank, print group 2 which is a float. If gorup3 is not blank, print group3 which is an int. If group4 is not blank, print group 4 which is a string. If group5 is not blank, print group5 which is a bool. When inside a loop, the syntax is:

//group0 is some kind of "currently evaluating" string plus group matches.
//group1 is my key group
//group2/3/4/5 are my values groups float/int/string/bool.
theString = (*iteratorVariableName)[groupNumber]

Collectives™ on Stack Overflow

how do I parse text file into variables using regex c++?

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related