0

A fairly simple question. However, I'm getting baffling results in VS2019. So, I devised a few tests to determine what was going on but am not sure what to make of the results.

Problem: Allocated memory of vector is about 1.5x the expected size of the vector. The memory profile indicates unresolved allocations that is approximately equal to the extra value that I'm seeing.

My trials-

  1. Simple struct containing a char, a float and an int. The allocated value is close to calculated value.
  2. Add a vector of 40 ints to the struct and check allocation.
  3. Use 2 such vectors instead of 1
  4. Use 6 now!
  5. Use 2 vectors but do not reserve the entire size of the vector
#include <vector>
struct wow
{
    char a;
    int glint;
    int b;
    std::vector<int> length = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};

    wow(char k, int g, long c)
    {
        a = k;
        glint = g;
        b = c;
    }
};
int main()
{
    std::vector<wow> tr;
    tr.reserve(180000);
    for (int i = 0; i < 180000; ++i)
        tr.emplace_back('g', 23, 6345345);
}

In trials 2, 3 and 4, the allocated/used memory was about 1.5x the calculated value! This was in spite of reserving the required number of elements in the vector. The heap memory allocation more or less matched the memory used in task manager. Given that I was able to match memory allocation for the first case, it is reasonable to assume that they would match in the other cases as well.

Initially, I assumed that the inability of the sizeof operator to determine the size of the vector led to repeat reallocations but this is not the case. The memory profile rises linearly without the short-term bump that is typical of reallocation (5th trial). The breakdown of memory allocation is shown in the link. It indicates that of the 42MB (total), 35MB is spent on the assignment operator and 7MB on allocating the vector. This makes no sense to me. Can someone please clarify?

Memory allocations for reference. The expected (calculated) values for the four trials are 2.05MB, 29.52MB, 57.0MB and 166.8MB. The actual allocation (by VS2019) is 2.58MB, 46.53MB, 82.4MB and 266.25MB.

Edit: - Breakdown of memory allocation and code. Interestingly, the numbers that I get remain the same.

10
  • What does sizeof wow show in your compile(s), and what did you base your expected values on? Commented May 27, 2020 at 2:06
  • sizeof(wow) returns 12B for no vector, 28B for 1, 44B for 2 and 108B for 6 vectors. The expected values are based on the size of the vector that I want to be using, i.e., 40 ints * 4B * #vectors + 12 (for the 2 ints and char). Now that I think about it, you pose a valid question. So, I re-did those tests with an edit to the code. This time, I directly define the vector to be used in the structure and avoid updating it. Editing the memory allocation breakdown as well. Do note that all the numbers remain the same. Commented May 27, 2020 at 2:23
  • Your estimates still look a bit optimistic. For char + 2 ints + 2 vectors sizeof wow is 44 and the theoretical minimum of all allocations would be 180K * (44 + 2 * 40 * 4) or approx. 62.5 MB vs. the 57 MB you quote. And that's assuming no overhead in vector storage for a non-empty vector other than the actual size of elements (which the standard doesn't guarantee, so you'd have to look at the particular implementation you are using). A quick check would be to replace the vectors with malloc'd pointers and compare the memory footprints. Commented May 27, 2020 at 2:57
  • Interesting! I get the impression that sizeof returns the size of the struct without the vectors. Is that why you use 44 + 2 * 40 * 2 instead of 12 + 2 * 40 * 2? If true, could you please guess at the breakdown of 44? Commented May 27, 2020 at 3:14
  • 1
    Puzzled here. sizeof will get you the size of the vector, but can't tell you the size of what the vector points at. vector is typically 3 pointers for a size of 12 or 24 bytes. The char will be padded to align the int that follows and then another int. Let's assume 32 bit int, so either 24 or 40 bytes, because the vector will have to be 64 bit aligned. I wouldn't expect 28 or 44 for wow with a single vector. Commented May 27, 2020 at 3:49

1 Answer 1

1

Thanks to @dxiv, I was able to match the memory usage in the test cases. I'm adding an answer here as reference.

The main issue was that I was working on debug mode instead of release mode. Changing this decreased the memory allocated by almost 20% and resulted in determining the actual allocated memory.

Another portion of the mismatch was from my incorrect estimation of the vector size. STL's are dynamic data structures that allocate memory from the heap for each element added. A vector, at least, is defined by three pointers, one to the start, one to the end and one to capacity of the vector. This 12B of allocation was something I missed as well.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.