0

I have a very simple query, and I'm trying to convert it into a NumPy array using the fromiter() function. However, I can't figure out why it's not working, or what the error below means. Any ideas?

import numpy as np

c.execute("SELECT video_id FROM video")
results = c.fetchall()
D = np.fromiter(results, dtype=float, count=-1) 

ERROR: ValueError: setting an array element with a sequence.

3 Answers 3

2
import numpy as np
from itertools import chain

c.execute("SELECT video_id FROM video")
results = c.fetchall()
D = np.fromiter(chain.from_iterable(results), dtype=float, count=-1) 

This should extract the values from the tuples in results.

Sign up to request clarification or add additional context in comments.

1 Comment

Thanks afg - your solution is indeed faster!
2

As before, results returns tuples, so you will have to pull the value out of the tuple using itertools.imap, or your own iterable adapter.

results = itertools.imap(lambda t: t[0], results)

But I'm just guessing the index since I don't know what you are querying.

2 Comments

Keith, really appreciate the help. This did it, and your suggestions have helped me speed up this routine by over 35%.
lambda is slow. If you want to use imap, use operator.itemgetter(0).
0

I'm not expert with MySQL, but it looks like results is ending up as a list where each element is a sequence, like results = [(foo, bar), (baz, bid)], and those items aren't valid numpy array values.

Make sure that each element of results is actually a float, and not, say, a float in a tuple.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.