2

I am using an external library which provides a function with the following interface:

void foo(const std::vector<int>& data);

I am receiving a very large C-style array from another library which has already been allocated:

int* data = bar();

Is there any way for me to pass on data to foo without allocating and copying each element? data is very large and therefore I want to avoid a copy and allocation if possible.

I could have used allocators, but foo is not templated for an allocator, so I don't believe this is possible.

I understand I may be asking for magic, but if it is possible that would be great. Of course if foo rather took an std::span this would not be a problem.

Improve this question
4
  • foo also wants an non-const reference, so it is able to modify\reallocate it. SOrry, I don't think it's possible. A library that allocates memory for large blobs is also problematic. Is it required to keep that object controlled by library at this point? Commented Jan 16, 2022 at 11:16
  • You say without using an allocator - however, using a custom allocator to create a local std:vector that wraps your int* data (then passing that vector) may be a solution to avoid memory copy/moves. This answer shows a way to do that. Commented Jan 16, 2022 at 11:21
  • @AdrianMole the problem is that std::vector<int, my_allocator<int>> is different type? In general that looks like two incompatible library interfaces. C++ one doesn't have legacy interface (pointer + size), the C one doesn't have an interface that allows using pre-allocated storage. Commented Jan 16, 2022 at 11:40
  • @Swift-FridayPie my bad, it should have been a const reference. Either way I'm also doubting whether this is possible since they are different types. I guess this is the problem that std::span solves. Commented Jan 16, 2022 at 11:43

2 Answers 2

Reset to default
1

Magic

This answer is magic, dependent on the implementation of the compiler.

We can forcibly access the container of a vector.

Take g++ as an example. It uses three protected pointers, _M_start, _M_finish, and _M_end_of_storage to handle storage. So we can create a derived class that sets/resets the pointers to the return of vaule bar() in the constructor and destructor.

Example code for g++:

static_assert(__GNUC__ == 7 && __GNUC_MINOR__ == 5 && __GNUC_PATCHLEVEL__ == 0);

class Dmy: public std::vector<int>
{
    public:
        Dmy(int *b, int *e)
        {
            _M_impl._M_start = b;
            _M_impl._M_finish = e;
            _M_impl._M_end_of_storage = _M_impl._M_finish;
        }

        ~Dmy()
        {
            _M_impl._M_start = 0;
            _M_impl._M_finish = 0;
            _M_impl._M_end_of_storage = 0;
        }
};

foo(Dmy(data, end_of_data));
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

This is great, thank you! I understand it is compiler type and version dependent, but in extreme cases with millions of integers in very tight code, it will be worth it. And if it works, it works (just have to check up on it when changing compiler versions). Appreciate the magic!
0

Is there any way for me to pass on data to foo without allocating and copying each element?

No, there's no way of avoiding it given the premise.

There are two alternative workarounds by changing the premise:

  • Change the other library to return a vector.
  • Change foo to not require a vector. As you point out, std::span would probably be a reasonable choice.
Improve this answer

6 Comments

Thanks for the reply, but unfortunately both library's APIs are out of my control, as implied when I mentioned that foo is from an "external library". I appreciate the help though! I guess this is one of the major downfalls of C/C++ interoperability.
@GaryAllen I guess this is one of the major downfalls of C/C++ interoperability. I wouldn't say so. It's rather a downfall of the design of the API that you're using.
mmm not sure I disagree. Any C API uses raw points like that. Majority of C++ APIs pre C++20 use std::vector instead of std::span. If I require the interoperability of these two libraries, I would surely not blame their API for incompatibility. Anyways
@GaryAllen Majority of C++ APIs pre C++20 use std::vector instead of std::span Spans existed way before they were added to the standard library in C++20. Using std::vector when you don't need it is a bad design (from efficiency perspective).
theoretically I'd agree, but the library in question is OpenCV, probably the most widely used computer vision API. Technically, I'd agree that it's a bad design, but when plenty of popular C++ libraries do it, its hard to justify that its their fault
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.