4

I am working on a C++ server project that has been plagued with a growing main() function, and the code base has grown to a point where compilation time is about 6 minutes (in debug mode) even after I make the slightest change to the main() function. (The main() function is about 5000 lines long!)

I'm using Visual Studio 2017, and (as I understand) the compiler has some pre-compiled header capabilities, as well as capability to not recompile unmodified functions. But those stuff are currently of little use because most of the logic is in the main() function.

Here's a (very simplified) version of my code:

struct GrandServer{
    std::map<std::string,std::function<void(std::string)>> request;
    /* some other functions of this server */
};

int main()
{
    SQLClient sql_client;
    auto query_database=[&sql_client](auto&& callback){/*stuff*/};
    GrandServer server;
    server.request["/my_page.html"] = [](std::string&& data){
        // Do stuff
    };
    server.request["/my_page_2.html"] = [](std::string&& data){
        // Do more stuff
    };
    server.request["/my_page_3.html"] = [](std::string&& data){
        // Do even more stuff
    };
    server.request["/my_page_4.html"] = [&query_database](std::string&& data){
        // Do many many callbacks
        query_database([](std::vector<std::string>&& results){
            std::string id = std::move(results.front());
            do_something_else([id = std::move(id)](auto&& param) mutable {
                /* do more stuff, call more functions that will call back, then answer the client */
            });
        });
    };
    /* Many many more lambda functions */
}

In essence, the core logic of the whole application is contained within the main() function, by defining lambdas stored in a std::map. The lambda functions contain a few levels of lambda functions within them, mostly defining (async) callbacks from the database. The other standalone functions consist mainly of the functions in GrandServer as well as various utility functions (e.g. time conversion, async callback utilities, unicode utilities, etc.) but none of them form the core logic of the application. This feels like really bad code sprawl :(

I'm thinking of converting all the top-level lambdas (i.e. those stored directly in server.request) into normal standalone member functions into a few separate compilation units like such:

// split definitions of methods in this class over several compilation units
// header file left out for brevity
struct MyServer{
    SQLClient sql_client;
    void query_database=[this](auto&& callback){/*stuff*/};

    void do_my_page(std::string&& data){
        // Do stuff
    }

    void do_my_page_2(std::string&& data){
        // Do stuff
    }

    void do_my_page_3(std::string&& data){
        // Do stuff
    }

    void do_my_page_4(std::string&& data){
        // Do many many callbacks
        query_database([](std::vector<int>&& results){
            do_something_else([](auto&& param){
                /* do more stuff, call more functions that will call back, then answer the client */
            });
        });
    }
};

// main.cpp
struct GrandServer{
    std::map<std::string,std::function<void(std::string)>> request;
    /* some other functions of this server */
};

int main()
{
    GrandServer server;
    MyServer my_server;
    server.request["/my_page.html"] = [&my_server](auto&&... params){my_server.do_my_page(std::forward<decltype(params)>(params)...);};
    server.request["/my_page_2.html"] = [&my_server](auto&&... params){my_server.do_my_page_2(std::forward<decltype(params)>(params)...);};
    server.request["/my_page_3.html"] = [&my_server](auto&&... params){my_server.do_my_page_3(std::forward<decltype(params)>(params)...);};
    server.request["/my_page_4.html"] = [&my_server](auto&&... params){my_server.do_my_page_4(std::forward<decltype(params)>(params)...);};
    /* Lots more lambda functions */
}

While this will reduce the size of main() to something much more tolerable, should I expect this to reduce compilation time significantly? There will not be any reduction in the number of template instantiations (in fact, I introduced some new template lambda forwarders in main() that should get inlined).

Also note that due to the callbacks and capturing of variables using move semantics, it is not easy to change the inner lambda functions within each logic flow to normal standalone member or non-member functions. (Template functions and classes cannot be in separate compilation units.) However, each logic flow is usually no more than 100 lines long so I do not believe that this is necessary.

6 minutes to compile a code base of about 6000-7000 lines seems way too slow, and I would think that this is due to my horrendously long main() function. Should I expect breaking this function up as I described above to significantly improve the compile time of this project?

4
  • 1
    Looks like instead of std::function<void(std::string)> what you really want would be void MyServer::*(std::string)> Or at least use pointer-to-member binding to create the std::function instances instead of yet another bunch of lambdas. That is, server.request["/my_page.html"] = std::bind(&MyServer::do_my_page, &my_server, _1); Commented Jul 5, 2017 at 16:17
  • @BenVoigt I would prefer not to do pointer-to-member binding, because the GrandServer definition is in an external header file. Keeping it intact would allow easier updating of the external header when there are fixes. But yes, I think std::bind is probably what I want. Commented Jul 5, 2017 at 16:27
  • 1
    And then you might consider struct { const char* path; void MyServer::*handler(std::string); } const pages[] = { { "/my_page.html", &MyServer::do_my_page }, { ... }, { .. } }; followed by for( auto& page : pages) server.request[page.path] = std::bind(page.handler, &my_server, _1); That way the messy bind call isn't repeated, only the portions that truly differ for every single resource. The pages definition then doesn't even need to be inside main(). Commented Jul 5, 2017 at 16:54
  • Have you copied every single #include <> to `stdfafx.h Commented Jul 5, 2017 at 18:08

1 Answer 1

1

Have you copied every single #include <> to stdfafx.h ?? This goes a long way towards reducing compilation times. The compiler may complain about the precompiled header being too large, but the default size is ridiculously small.

The -Zm option controls the amount allocated for the precompiled header, in megabytes.

I've seen over 10 x improvements in compilation speed for some projects.

If you map a network drive to one of your local disks, there is an easy way to see a further 3x improvement in compilation time.

Here's a solution for that case, it involves replacing the network map to a DOS drive in the registry. Here's what this gives for my own setup:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\DOS Devices] 
"R:"="\DosDevices\D:\devel\build"
"S:"="\DosDevices\D:\devel\src"
Sign up to request clarification or add additional context in comments.

1 Comment

Putting all my includes - Windows.h, standard library, external header-only libraries, my own project headers, everything included - into stdafx.h only reduces the compilation time slightly - to about 4-5 minutes. It seems that the VS compiler doesn't complain that the precompiled header is too large. My code is all stored on my local machine, so I think the part about network drives would be irrelevant for me.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.