4

I was reading this question: Cannot return int array because I ran into the same problem.

It seems that data structures (because C can obviously return a locally declared variable) declared locally within a function cannot be returned, in this case an array.

However Python doesn't suffer from the same problem; as far as I can remember, it's possible to declare an array within a function and to return that array without having to pass it as an argument.

What is the difference "under the hood"? Is Python using pointers implicitly (using malloc within the function)?

Improve this question
14
  • 2
    Everything in Python is a pointer. In CPython, they're literally PyObject* and are refcounted. There's no way to allocate an object on the stack (which is the problem in C - things on the stack will disappear after a return, so if you try to return a pointer to your local stack frame, it'll become a dangling pointer). Commented Jan 31, 2017 at 1:37
  • @Kevin, thanks for the explanation. Commented Jan 31, 2017 at 1:39
  • Up voted because this is actually a good question. I don't know who down voted but you should be ashamed of yourself. Commented Jan 31, 2017 at 1:39
  • 1
    "Why can Python functions return locally declared arrays but C can't?" Because Python isn't C? Why should restrictions on C functions have any relevance to the semantics of Python functions? Commented Jan 31, 2017 at 1:51
  • 1
    @jeremyradcliff I'm not sure responding in kind is particularly helpful either, though. YMMV. Commented Jan 31, 2017 at 2:17

2 Answers 2

Reset to default
5

For the record, Python's built-in mutable sequence type is called a list, not an array, but it behaves similarly (it's just dynamically resizable, like C++'s std::vector).

In any event, you're correct that all Python objects are implicitly dynamically allocated; only the references (roughly, pointers) to them are on the "stack" (that said, the Python interpreter stack and the C level stack are not the same thing to start with). Comparable C code would dynamically allocate the array and return a pointer to it (with the caller freeing it when done; different Python interpreters handle this differently, but the list would be garbage collected when no longer referenced in one way or another).

Python has no real concept of "stack arrays" (it always returns a single object, though that object could be a tuple to simulate multiple return values), so returns are always ultimately a single "pointer" value (the reference to the returned object).

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

so in Python, when you index into a data structure, are you implicitly dereferencing a pointer?
@jeremyradcliff: Usually (many) more than one pointer dereference is involved. For example, in CPython (the reference interpreter) if you have a list mylist, and do mylist[2], it has to: 1. Dereference the pointer to mylist 2. Lookup the type of mylist (a pointer in the PyObject structure) and look for implementations of the C equivalent to the __getitem__ special method, and call the one it finds 3. Dereference the pointer to the object representing 2 to pull out the C level value 4. Dereference the pointer to the array of object pointers, at offset 2
I probably missed a few additional dereferences here too. All along the way, there are a ton of implicit pointer traversals involved, to maintain reference counts (the value being looked up has its reference count incremented before it's passed back, and even if you never even store the result, that increment and decrement have to happen), perform type checking and dynamic dispatch, etc. Python is much higher level; getting hung up on exact operations is usually a bad idea.
Thank you for explaining all of that; even if I don't understand all of it it's interesting to realize how much stuff is hidden in higher-level languages. I had no idea it was to that extent.
1

It seems that data structures (because C can obviously return a locally declared variable) declared locally within a function cannot be returned, in this case an array.

You already have a good Python answer; I wanted to look at the C side a little more closely.

Yes, a C function returns a value. That value may be primitive C type, or a struct or union type. Or, it may be a pointer type.

The C language syntax makes arrays and pointers seem very similar, which makes arrays special. Because the name of the array is the same as the address of the first element, it can't be something else. In particular, an array name does not refer to the whole array (except in the case of the sizeof operator). Because any other use of an array name refers to the address of the first element, attempting to return an array results in returning only that address.

Because it's a C function, that address is returned by value: namely, a value of a pointer type. So, when we say, 

char *s = strdup("hello");

s is a pointer type whose value is not "hello", but the value of address of the first element of the array that strdup allocates.

Python doesn't suffer from the same problem

When Y is a property of X, Y is a problem only if that property is, in the eyes of the beholder, undesirable. You can be sure the way C treats arrays is not accidental, and is often convenient.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.