Concatenate sparse matrices in Python using SciPy/Numpy

Question

What would be the most efficient way to concatenate sparse matrices in Python using SciPy/Numpy?

Here I used the following:

>>> np.hstack((X, X2))
array([ <49998x70000 sparse matrix of type '<class 'numpy.float64'>'
        with 1135520 stored elements in Compressed Sparse Row format>,
        <49998x70000 sparse matrix of type '<class 'numpy.int64'>'
        with 1135520 stored elements in Compressed Sparse Row format>], 
       dtype=object)

I would like to use both predictors in a regression, but the current format is obviously not what I'm looking for. Would it be possible to get the following:

    <49998x1400000 sparse matrix of type '<class 'numpy.float64'>'
     with 2271040 stored elements in Compressed Sparse Row format>

It is too large to be converted to a deep format.

Saullo G. P. Castro · Accepted Answer · 2020-08-12 15:02:45Z

92

You can use the scipy.sparse.hstack to concatenate sparse matrices with the same number of rows (horizontal concatenation):

from scipy.sparse import hstack
hstack((X, X2))

Similarly, you can use scipy.sparse.vstack to concatenate sparse matrices with the same number of columns (vertical concatenation).

Using numpy.hstack or numpy.vstack will create an array with two sparse matrix objects.

edited Aug 12, 2020 at 15:02

answered Oct 31, 2013 at 15:30

Saullo G. P. Castro

59.4k28 gold badges191 silver badges244 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

simeon Over a year ago

Seems hstack is quite slow, check this post out on a similar question link

Saullo G. P. Castro Over a year ago

@simeon interesting that Scipy's dev team hasn't adopted such efficient solution

mgokhanbakal Over a year ago

For the horizontal concatenation hstack() and for the vertical concatenation vstack() can be used.

Collectives™ on Stack Overflow

Concatenate sparse matrices in Python using SciPy/Numpy

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related