31

I'm trying to use map in Python3. Here's some code I'm using:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    map(writer.writerow, data)

However, since map in Python3 returns an iterator, this code doesn't work in Python3 (but works fine in Python2 since that version of map always return a list)

My current solution is to add a list function call over the iterator to force the evaluation. But it seems odd (I don't care about the return value, why should I convert the iterator into a list?)

Any better solutions?

Improve this question
5
  • 3
    Using map for side effects is what's odd. Python 2 map also collects the return values. The new behavior merely highlights it further. Just don't do that, use a for loop. Commented Aug 25, 2013 at 20:45
  • @delnan Thank you for the link, indeed I shouldn't use map for side effect. Commented Aug 25, 2013 at 20:48
  • 4
    for Python 3, list(map(lambda x:2*x, [1,2,3])) Commented Aug 5, 2016 at 21:01
  • 1
    I feel like "map for the side effects" is a common-enough idiom that one could justify including it as a language feature. However, python3 has decided that map shall be a lazily-evaluated function more in line with other languages. In any case, I find myself typing [*map(...)] almost all the time when I'm using Python for scientific statistics, where usually an immediate result is needed (e.g. numpy.array(...) understands lists but not generators). So the non-lazy map is the superior default for some applications. Commented Oct 28, 2022 at 7:54
  • FWIW I've defined the helpers lmap and amap that add the necessary boiler plate to immediately collect the results of map() as a list or np.array, respectively. It makes the code significantly cleaner, easier to read, and easier to maintain. Scattering casts to list or [*map(...)] everywhere, or using a list comprehension when map is much more succinct, just looks sloppy. Commented Oct 28, 2022 at 7:55

6 Answers 6

Reset to default
29

Using map for its side-effects (eg function call) when you're not interested in returned values is undesirable even in Python2.x. If the function returns None, but repeats a million times - you'd be building a list of a million Nones just to discard it. The correct way is to either use a for-loop and call:

for row in data:
    writer.writerow(row)

or as the csv module allows, use:

writer.writerows(data)

If for some reason you really, really wanted to use map, then you can use the consume recipe from itertools and generate a zero length deque, eg:

from collections import deque
deque(map(writer.writerow, data), maxlen=0)
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Seemingly the list and deque variants are faster than the for loop. Here's a follow-up question: stackoverflow.com/q/79693413/430766
7

You can set up a zero length deque to do this:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    collections.deque(map(writer.writerow, data),0)

This is the same way that itertools.consume(iterator, None) recipe works. It functionally will exhaust the iterator without building the list.

You can also just use the consume recipe from itertools.

But a loop is more readable and Pythonic to me, but YMMV.

Improve this answer

Comments

4 
list(map(lambda x: do(x),y))

will trigger an evaluation and stay in english pen-and-paper math semantics. Some like this more than others, but I've had a coder who spoke English as a second language tell me this is harder to understand than:

[ do(x) for x in y ]

¯\(ツ)/¯

Anyhow, if you want to flush the map stack, trigger it with a list type conversion or a list comprehension.

With context:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    list(map(writer.writerow, data))
    # or: [ writer.writerow(x) for x in data ]
Improve this answer

1 Comment

Exactly what I'm doing. Glad to know that I'm not missing out too much lol
3

If you don't care about the return value, then map is not the best tool for the job. A simple for would be better:

for d in data:
    writer.writerow(d)

That'll work fine in Python 2.x and 3.x. Notice that map is useful when you want to create a new list, if you're traversing over an iterable just for the effect, then use a for.

Improve this answer

Comments

2

You can also use list comprehension, as suggested in the official FAQ:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    [writer.writerow(elem) for elem in data]

List comprehension will force the evaluation of each element even if you don't assign the newly created list to any variable.

Note, however, that the list may still be created behind the scenes, creating a potential performance pitfall, so while relatively succinct, it should not be used if the input sequence can get very long.

Improve this answer

Comments

0

I would use a function to extract data from the iterable using something like this:

def rake(what, where=None):
    for i in what: 
        if where: where.append(i)

rake(map(writer.writerow, data))

If you know up front that you won't ever be collecting the output of the mapped function then you could simplify that to just:

for i in what: pass

But neither approach keeps excess data around unless you supply a list to put it in. And this approach should work equally well with map, filter, reduce, generators, range, and anything else that you could pass in to the rake function that for can iterate over.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.