Getting numpy error (ValueError: setting an array element with a sequence)

Question

The following code written using python 3.6. It should create a final matrix containing binary vectors. During the loop, each batch is taken from consecutive vectors to be used by a method simulate_output() but eventually the batch is flatten to be a single vector:

tmp_list = []
output = []
num_components = 4
dim = 16
total_vectors = 100000

X = np.random.choice(np.array([0, 1], dtype=np.uint8), size=(total_vectors, dim))
num_vectors = np.unique(X, axis=0)

for i in range(0, len(num_vectors), num_components):
    batch = num_vectors[i:(i + num_components)]
    # output.append(simulate_output(batch)) # Comment this line will not solve the error.
    batch = np.hstack(batch)  # to flatten the list into a single vector
    tmp_list.append(batch)

final_matrix = np.array(tmp_list, dtype=np.int8)
print(final_matrix)

For some runs I get this error:

Traceback (most recent call last):
  File "test.py", line 65, in <module>
    final_matrix = np.array(tmp_list, dtype=np.int8)
ValueError: setting an array element with a sequence.

I believe the error is on last line final_matrix = np.array(tmp_list, dtype=np.int8) but I have no idea why and how to fix it since in some runs it works while in other runs it does not.

Thank you

What version of numpy are you running? Running this example (sans the output.append(...) line) works fine for me on '1.14.5'? I typically have seen this error come up when an element of a list is a list itself. If it breaks for you again I'd inspect the contents of tmp_list and make sure it's what you actually expect it to be. — Jorden
– Jorden, Commented Jun 28, 2018 at 23:49
@Jorden I upgrade to 1.14.5 but still getting the same error. If you try running multiple times, you should get this error. — Ahmad
– Ahmad, Commented Jun 28, 2018 at 23:59
I feel stuck. The code seems fine but cannot figure out the error — Ahmad
– Ahmad, Commented Jun 29, 2018 at 0:02

ykaner · Accepted Answer · 2018-06-29 00:22:53Z

3

I found your problem. In this line:

final_matrix = np.array(tmp_list, dtype=np.int8)

you expect the final_matrix to be a 2-dimensional numpy array. this can be if all the lines have the same length, but this is not exactly your case. your last flatten batch vector is shorter because len(num_vectors) is not divided by num_components(4).

If you simply put:

tmp_list = tmp_list[:-1]

after the for loop, everything will be fine. I think one element out of thousands is negligible. If you still don't want to delete it, try to pad it with zeroes to the desired size - num_components * dim.

answered Jun 29, 2018 at 0:22

ykaner

1,8701 gold badge11 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ahmad Over a year ago

Great catch! Thank you so much. MAKES SENSE

Collectives™ on Stack Overflow

Getting numpy error (ValueError: setting an array element with a sequence)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related