So I have a numpy array of strings that contain numeric values separated by spaces, for example:
np.array(['1 2', '3 4'])
array(['1 2', '3 4'], dtype='<U3')
and I want to convert it to a numerical matrix like:
np.array([[1,2],[3,4]])
array([[1, 2],[3, 4]])
I'm looking for an operation that can leverage numpy vecotrized operations, as speed is important here. The rows have length 2 in this example, but I need a general approach with an arbitrary row length.
Thanks!
Here is an approach assuming nonnegative ints coming in pairs of two separated by a single space:
def to_num(x):
y = (x[:,None].view(np.int32)-48)*10**np.arange(x.itemsize//4-1,-1,-1)
splt = y.argmin(1)
z = np.take_along_axis(y.cumsum(1),np.column_stack([splt-1,np.full(*y.shape-np.arange(2))]),1)
z[:,1]+=10**(y.shape[1]-splt-1)*16-z[:,0]
z[:,0]//=10**(y.shape[1]-splt)
end = (y[:,::-1]>=0).argmax(1)
z[:,1]+=np.concatenate([[0],48*np.cumsum(10**np.arange(end.max()))])[end]
z[:,1]//=10**end
return z
For example, 10^6 pairs take roughly 3 secs on my machine:
from timeit import timeit
x = np.random.randint(0,1000000,(1000000,2))
x = np.array([" ".join(map(str, y)) for y in x.tolist()])
(to_num(x) == [[int(z) for z in y.split()] for y in x.tolist()]).all()
# True
timeit(lambda:to_num(x), number=10)
# 2.9360161621589214
If it dont have to be that fast you could iterate element-wise over it and then apply:
def seperate_sting(s):
split_numbers = s.split(' ')
output = np.asarray(split_numbers).astype(int)
return output
seperate_sting('1 1')
>>> array([1, 1])
First, try to split your string with the white space, and when it's done check for the function numpy.asmatrix()
np.matrix(';'.join(a)) uses the ''1 2; 3 4" syntax that np.matrix accepts. But this is slower than the list comprehensions. np.matrix still has to use string operations to split the lines and numbers, just reversing our join. It doesn't use fast compiled code to do that.
np.char.splitandpd.Series.str.splitbut both of them are slower than the native Python loops in the accepted answer of the duplicate target.