1

Problem: In my system there are a lot of statistic calculators wrapped as classes. My job is to combine the values from these calculators into one single value called 'total'.

The 'model' that explains how these calculators should be combined is given as a table --- either a csv file or a table in MySQL

instance_name,accessor,argument,post_processing_function_name
fleet_statistics,getCash,2017,func1
city_statistics,getPopulation,2016,func2
....
....

My current solution: I wrote a python script to read the csv file and generate c++ code like this:

double computeTotal()
{
    double total = 0;
    total += func1(fleet_statistics->getCash(2017));
    total += func2(city_statistics->getPopulation(2016));
    ....
    ....
}

Using python to generate the code is very convenient, but the problem is that, it is no longer a 'pure' c++ program. I need to hack the makefile to make sure that the generated c++ file is up to date.

A more elegant solution is to use meta-programming to generate the code in c++ using c++ rather than python. Can anyone show me how to do this?

Jut to mention a few points:

  1. the CSV file is generated for by some business-logics-related-scripts. It has a couple thousands of rows. It is impossible to write the c++ code manually.

  2. The CSV file can change every time when we make a new release.

  3. I use CSV format as an example. It could be a table in MySQL if that will make things easier

  4. objects in the instance_name column are not necessarily from the same BaseClass. The accessor's function signature is variant as well. So it is not convenient to use a function pointer

Thanks!

Improve this question
8
  • 1
    Can you change the format of the input csv, or is that given? Commented Nov 18, 2017 at 9:07
  • 4
    This feels like an XY problem. Do you really need metaprogramming? Why not define a C++ data structure with some function pointers in it? Or why not just write the code by hand? You're asking questions about your proposed solution instead of really saying what the problem is. Commented Nov 18, 2017 at 9:12
  • 2
    More importantly why do you want to change the solution you have? What is wrong with code generation included in makefile? Why do you call it "hacking"? It might be hard to debug, that's for sure, but it works right? "Pure C++" is a meaningless requirement. Art for art's sake. Commented Nov 18, 2017 at 9:15
  • 1
    Why does this CSV file exist in the first place? If someone edits it on a weekly basis, couldn't they edit the C++ code instead? Commented Nov 18, 2017 at 9:23
  • 2
    To repeat Davids question, why do you need metaprogramming? As far as I see it given your example, you need a database. Hard-coding a thousand lines of data sounds just wrong. Especially it it regularly changes. Recompiling should not be required to happen every time that the program is to be used. Commented Nov 18, 2017 at 9:23

4 Answers 4

Reset to default
3

There should be no need to generate and compile C++ code when the CSV file changes.

All you should do is write C++ code once that parses the CSV file performs the calculations specified in the CSV file by invoking C++ functions on C++ objects.

So, roughly speaking, your fleet_statistics and city_statistics classes should implement some common interface that has a double getYearValueByName( String name, int year ) function, which checks the given name and invokes on itself a corresponding function of the same name to get a value for that year.

I understand that this approach is simplistic, and in reality things are more complicated than that; you will need to just keep extending that model, making it more complicated as necessary, until you have all the functionality needed to parse your CSV file and do as it requires.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Generally true, however in some cases generating C++ code at runtime does make sense. See my answer.
2

You mentioned in comments that the format of the input file is under your control. One way to use actual metaprogramming (more specifically preprocessor metaprogramming) would be to change the file's format into something suitable for using the X macro pattern. Something like this:

DATA_POINT(fleet_statistics,getCash,2017,func1)
DATA_POINT(city_statistics,getPopulation,2016,func2)

Then, in your C++ code, you'd do this:

double computeTotal()
{
    double total = 0;
    #define DATA_POINT(instance_name,accessor,argument,post_processing_function_name) \
      total += post_processing_function_name(instance_name->accessor(argument));
    #include "datafile"
    #undef DATA_POINT
}

This way, the C++ code being compiled is generated by the preprocessor from the data file.

However, you should definitely consider the opinions given in other comments and answers (which I share), that a parser could be more appropriate for your case. Normally, it's a good idea to keep code separated from data.

Improve this answer

4 Comments

This is pretty much what I am looking for. I remotely recall that most preprocessor metaprogramming can be converted to template metaprogramming ( template meta-programming was first designed to improve macros ) Do you think it is possible to implement this using template?
@user152503 Why? What you have here is essentially text manipulation, which is what the preprocessor (or other macro processors) is good for. Templates are for type & value manipulation.
@user152503 However, I'm still not convinced that metaprogramming is more appropriate here than an ordinary runtime parser. You mention that "objects in the instance_name column are not necessarily from the same BaseClass". What prevents you from introducing an additional interface to them?
there might be tens of independent classes (not derived), each has a dozen accessors with different signatures. If I build a common interface from which all these classes are derived from, this interface might be very big --- not sure if this is a good design
2

but the problem is that, it is no longer 'pure' c++

So? How is that a real problem? Obviously your case is more complicated than simple "read csv and take total sum". This requires more complicated solutions. But dont make them more complicated than needed.

There is nothing wrong with the technique (i.e. C++ code generation via other script) you are using. While not common you can still see it elsewhere with the notable example of Google's protobuf.

So instead of looking for other complicated solutions (Jit compilation? Really? Even templates are often hard to follow) ask yourself: does my Python code generation work? Does it perform well enough? Is it easy to maintain? How much would the refactoring cost? These are real problems.

Also note that while makefiles are mostly used with C/C++ they are not strictly restricted to them. Actually any build process with one or many languages can be done with them. You can mix them. It's not hacking. It's simply a build process.

Improve this answer

1 Comment

I would add: make sure to document all of this. If your build process is well documented and uses preferably non-obscure tools and languages, there shouldn't be a problem.
0

At first, you want to write some interpreter for the formulas in your CSV files. You probably could find a library doing exactly that. Otherwise, you need to implement such an interpreter (or embed some existing one, like Lua or Guile). Read absolutely some good book like the Dragon Book, SICP then Lisp In Small Pieces. You'll need to represent ASTs and environments (for variable bindings). If you have never been taught about compilers and interpreters you do need to spend several weeks reading about these (the techniques are mature but difficult, so you won't be able to reinvent all of them).

(I guess you have a good operating system, like Linux. Adapt my answer if not)

If performance really matters because the formulas in your CSV files are complex and takes time to be computed, consider translating your ASTs (from the formulas) into some form of code using some JIT compilation library (e.g. libgccjit).

In some rare occasions (but probably not in your case) you might instead generate some C++ at runtime into some temporary file, then compile that as a plugin and dynamically load that temporary plugin (e.g. using dlopen & dlsym on Linux). Generally this is not worth the effort, because C++ compilers are really slow (at compile-time!). Quite often generating C code is more appropriate (but YMMV). But compiling to C or to C++ is never easy and takes months of development time.

I need to hack the makefile to make sure that the generated c++ file is up to date.

Not necessarily (needed to hack the Makefile, which is not a big deal). You could run g++ (or your C++ compiler, if it is not GCC) from your Python script or your C++ program. I don't know if (in your specific case) it is sensible to do that.

NB. Notice that -in the context of C++ specifically- meta-programming means (most often) not generating C++ code or some other code, but refers to template metaprogramming.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.