3

I am new to NumPy and am trying to use it in my code for some tables.

I have a list of coordinates that looks like this: 

coordinates = [["2 0"], ["0 1"], ["3 4"]]

and want to write it like this:

coordinatesNumpy = np.array([[2, 0], [0, 1], [3, 4]])

In regular Python that's easy to do but how do you do it with NumPy? Should I just make the table with regular Python functions for lists and then convert the 2d table to np.array or does NumPy have methods for splitting and stuff?

I tried some things but they all give me an error. The latest thing I tried:

flowers = np.array([np.array([int(coordinate[0]), int(coordinate[2])]) for coordinate in coordinates])

How could I do something like this with NumPy?

Improve this question

3 Answers 3

Reset to default
3

Take a look at numpy.fromstring:

coordinates_numpy = np.array([np.fromstring(i, dtype=int, sep=' ')
                             for j in coordinates for i in j])
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

List comprehension with pure Python

This works:

>>> flowers = np.array([[int(x)  for x in coordinate[0].split()] 
                        for coordinate in coordinates])
>>> flowers
array([[2, 0],
       [0, 1],
       [3, 4]])

I am not aware of any NumPy function that would do this in one step.

Performance

Let's check how fast things are.

For your example data, the pure Python version is the fastest:

%timeit np.array([np.fromstring(i, dtype=int, sep=' ') for j in coordinates for i in j])
100000 loops, best of 3: 18.4 µs per loop

%timeit np.array([np.fromstring(item[0], dtype=int, sep=' ').tolist() for item in coordinates])
10000 loops, best of 3: 19 µs per loop

%timeit np.array([[int(x)  for x in coordinate[0].split()] for coordinate in coordinates])
100000 loops, best of 3: 12.1 µs per loop

Make the data bigger:

long_coords = coordinates * 1000

But still, the pure Python version is the fastest:

%timeit np.array([np.fromstring(i, dtype=int, sep=' ') for j in long_coords for i in j])
100 loops, best of 3: 12.2 ms per loop

%timeit np.array([np.fromstring(item[0], dtype=int, sep=' ').tolist() for item in long_coords])
100 loops, best of 3: 14.2 ms per loop

%timeit np.array([[int(x)  for x in coordinate[0].split()] for coordinate in long_coords])
100 loops, best of 3: 7.54 ms per loop

Consistent results for even larger data:

very_long_coords = coordinates * 10000

%timeit np.array([np.fromstring(i, dtype=int, sep=' ') for j in very_long_coords for i in j])
10 loops, best of 3: 125 ms per loop

%timeit np.array([np.fromstring(item[0], dtype=int, sep=' ').tolist() for item in very_long_coords])
10 loops, best of 3: 140 ms per loop

%timeit np.array([[int(x)  for x in coordinate[0].split()] for coordinate in very_long_coords])
10 loops, best of 3: 73.5 ms per loop
Improve this answer

6 Comments

Just added Approach #2 in my solution. Could you add that too in the benchmarks?
Gives me ValueError: total size of new array must be unchanged for NumPy '1.10.2' on Python 3.5.
Did you get the error with coordinates or long_coords or very_long_coords?
With any. For example: C = [['2 0'], ['0 1'], ['3 4']].
Hmm, works at my end on for NumPy '1.10.1' on Python 2.7. Hope it's not a version issue. Will add updated runtimes in my solution I guess, hope that's okay.
|
2

Assuming C as the input list, two approaches could be suggested to solve it.

Approach #1 : Using one level of list comprehension with np.fromstring -

np.array([np.fromstring(item[0], dtype=int, sep=' ').tolist() for item in C])

Approach #2 : Vectorized approach using padding with np.core.defchararray.add and then getting the separated numerals -

np.fromstring(np.core.defchararray.add(C," "),dtype=int,sep=" ").reshape(len(C),-1)

Sample runs -

In [82]: C = [['2 0'], ['0 1'], ['3 4']]

In [83]: np.array([np.fromstring(item[0], dtype=int, sep=' ').tolist() for item in C])
Out[83]: 
array([[2, 0],
       [0, 1],
       [3, 4]])

In [84]: np.fromstring(np.core.defchararray.add(C, " "),dtype=int,sep=" ").reshape(len(C),-1)
Out[84]: 
array([[2, 0],
       [0, 1],
       [3, 4]])

Benchmarking

Borrowing the benchmarking code from @Mike Müller's solution, here are the runtimes for the long_coords and very_long_coords cases -

In [78]: coordinates = [["2 0"], ["0 1"], ["3 4"]]
    ...: long_coords = coordinates * 1000
    ...: %timeit np.array([np.fromstring(i, dtype=int, sep=' ') for j in long_coords for i in j])
    ...: %timeit np.array([np.fromstring(item[0], dtype=int, sep=' ').tolist() for item in long_coords])
    ...: %timeit np.array([[int(x)  for x in coordinate[0].split()] for coordinate in long_coords])
    ...: %timeit np.fromstring(np.core.defchararray.add(long_coords, " "), dtype=int,sep=" ").reshape(len(long_coords),-1)
    ...: 
100 loops, best of 3: 7.27 ms per loop
100 loops, best of 3: 9.52 ms per loop
100 loops, best of 3: 6.84 ms per loop
100 loops, best of 3: 2.73 ms per loop

In [79]: coordinates = [["2 0"], ["0 1"], ["3 4"]]
    ...: very_long_coords = coordinates * 10000
    ...: %timeit np.array([np.fromstring(i, dtype=int, sep=' ') for j in very_long_coords for i in j])
    ...: %timeit np.array([np.fromstring(item[0], dtype=int, sep=' ').tolist() for item in very_long_coords])
    ...: %timeit np.array([[int(x)  for x in coordinate[0].split()] for coordinate in very_long_coords])
    ...: %timeit np.fromstring(np.core.defchararray.add(very_long_coords, " "), dtype=int,sep=" ").reshape(len(very_long_coords),-1)
    ...: 
10 loops, best of 3: 80.7 ms per loop
10 loops, best of 3: 103 ms per loop
10 loops, best of 3: 71 ms per loop
10 loops, best of 3: 27.2 ms per loop
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.