Keep getting ValueError with numpy while trying to create array

Question

I am trying to create a new array with an existing array's elements. I keep getting:ValueError: Setting void-array with object members using buffer.

import numpy as np
import datetime

date = datetime.date(2013, 4, 5)
results = [(date,0,1,2,3), (date,5,1,5,6), (date,3,4,4,7)] 
stock_dt = np.dtype([('date', object),
                     ('open', np.int8),
                     ('high', np.int8),
                     ('low', np.int8),
                     ('close', np.int8)])

d = np.array(results, dtype=stock_dt)
matches = []
for item in d:
    if item['high'] == 1:
        matches.append(item)

rec = np.array(matches, dtype=stock_dt)

print rec

Brendan Dolan-Gavitt · Accepted Answer · 2013-03-28 02:40:17Z

4

The problem is that matches is not a list of tuples, so you cant make a structured array out of it. Instead it's a list of structured arrays, which need to be merged back into a single structured array. You can use numpy.lib.recfunctions.stack_arrays for this:

In [21]: import numpy.lib.recfunctions as rfn

In [22]: rfn.stack_arrays(matches,usemask=False)
Out[22]: 
array([(datetime.date(2013, 4, 5), 0, 1, 2, 3),
       (datetime.date(2013, 4, 5), 5, 1, 5, 6)], 
      dtype=[('date', 'O'), ('open', 'i1'), ('high', 'i1'), ('low', 'i1'), ('close', 'i1')])

You could also consider doing away with the loop entirely:

In [23]: d[d['high'] == 1]
Out[23]: 
array([(datetime.date(2013, 4, 5), 0, 1, 2, 3),
       (datetime.date(2013, 4, 5), 5, 1, 5, 6)], 
      dtype=[('date', 'O'), ('open', 'i1'), ('high', 'i1'), ('low', 'i1'), ('close', 'i1')])

Which should be faster, to boot.

answered Mar 28, 2013 at 2:40

Brendan Dolan-Gavitt

6771 gold badge6 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Wallace Over a year ago

How could I use multiple arguments using your second answer?

Brendan Dolan-Gavitt Over a year ago

What d[d['high'] == 1] is doing is indexing the array d using a list of booleans created by d['high'] == 1. To use multiple criteria you can use the & and | operators, like d[(d['high'] == 1) & (d['low'] == 2)]. Beware of making those too complex though, since it can get unreadable quickly.

Jared · Accepted Answer · 2013-03-28 02:36:25Z

1

Change

rec = np.array(matches, dtype=stock_dt)

to

rec = np.array(matches)

When you're iterating over matches you aren't dealing with tuples anymore so you shouldn't pass dtype=stock_dt to np.array again.

answered Mar 28, 2013 at 2:36

Jared

26.5k7 gold badges61 silver badges62 bronze badges

1 Comment

Wallace Over a year ago

I'm still getting same Error

Collectives™ on Stack Overflow

Keep getting ValueError with numpy while trying to create array

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related