All Questions
Tagged with sparse-array or sparse-matrix
4,441 questions
2
votes
1
answer
125
views
PCA with arpack returns different values when the order of observations change, but why?
I have recently noticed that when I change the order of the observations in a sparse array, scikit-learn PCA with svd_solver="arpack" returns different floating point numbers. Is this an ...
- 23
0
votes
3
answers
128
views
Database Rows to static columns [closed]
I have a single column DB file that is rigidly formatted. It doesn't have a header or field names in the export that I've imported into a Goggle Sheet. I need help with how to approach this task of ...
- 379
1
vote
2
answers
97
views
Confused about how tf.keras.Sequential works in TensorFlow – especially activation and input_shape
I'm learning TensorFlow to build machine learning models in Python. I tried following the official documentation on creating a simple classification model, but I couldn't clearly understand the tf....
- 45
6
votes
1
answer
113
views
Efficient filtering of sparse matrix from full
I have a large, full, dense, symmetric matrix (eg 30k² or 60k² positive elements, ~100% density). I want to extract either (i) the top (eg) 1% highest values; or (2) the values above a user-specified ...
- 639
0
votes
0
answers
52
views
MICCG(0) for a fluid simulation fails at Neumann boundaries
I am trying to follow Robert Bridson's Fluid Simulation Notes (https://www.cs.ubc.ca/~rbridson/fluidsimulation/fluids_notes.pdf) to implement my own eulerian fluid simulator for the first time.
I was ...
- 11
0
votes
1
answer
58
views
how to use mkl_dcsrgemv or other functions in OneAPI to cal. scalar prodoct between mass dim sparse matrix and vector?
I program in fortran with Intel OneAPI compiler ifx and MKL packages.
I want to cal. the scalar product between a mass dim sparse matrix and a vector.
When the dim of the sparse matrix could be ...
0
votes
1
answer
126
views
It is possible to compute both the size and values for a std array in a constexpr function?
I am trying to express the structure of a sparse matrix (and the structure resulting from sparse matrix operations) at compile time with template parameters and constexpr functions.
I've defined this ...
- 41
1
vote
2
answers
91
views
How to exploit sparsity with lists in a Cython function
I have a function I need to implement which does not vectorize well, and so I am implementing it in Cython to make the nested for-loops workable. My problem is of a similar complexity as naive matrix-...
2
votes
0
answers
191
views
How to reuse cuDSS factors when solving a system of linear equations Ax=b
I am using cuDSS to solve a set of Ax=b equations as follows
cudssMatrixType_t mtype = CUDSS_MTYPE_SPD;
cudssMatrixViewType_t mview = CUDSS_MVIEW_UPPER;
cudssIndexBase_t base = CUDSS_BASE_ZERO;
...
- 83
1
vote
1
answer
75
views
scipy.sparse: one subarray at many locations in a larger array
Say I have a sparse subarray whose contents and shape are known:
import scipy.sparse as sp
sub = sp.coo_array([[a, b], [c, d]])
I'd like to place this subarray at many locations, according to some ...
2
votes
1
answer
112
views
How to convert Eigen Sparse Matrix to Cuda CuSparse Matrix
I am trying to convert an Eigen Sparse Matrix of size NxN to CuSparse matrix so that I can use CUDA to solver the matrix Ax=B. Came across this thread which is pretty old (Convert Eigen::SparseMatrix ...
- 83
1
vote
1
answer
105
views
How to efficiently multiply a vector by a transposed sparse matrix in Eigen?
I have with a sparse matrix A and a vector x in Eigen, and I need to perform the following transposed matrix × matrix × vector operation: ATAx. This can be decomposed into two matrix-vector ...
- 24.2k
1
vote
1
answer
54
views
How to produce tensor of first occurrencies of another tensor using PyTorch
Let's say we have an ordered 1D tensor of ints/longs t.
I want to produce a new tensor a with size max(t) where each term a[i] contains the first occurrence of the value i in the tensor t.
We could ...
- 146
4
votes
2
answers
129
views
Identifying and removing duplicate columns/rows in sparse binary matrix in PyTorch
Let's suppose we have a binary matrix A with shape n x m,
I want to identify rows that have duplicates in the matrix, i.e. there is another index on the same dimension with the same elements in the ...
- 146
0
votes
1
answer
67
views
Matrix free arnoldi
I want to do a "matrix-free eigenvalue evaluation" of a unitary matrix using ARPACK package. Unitary matrix comes from a sparse hermitian matrix, something that happens in many-body problem.
...
- 103
1
vote
0
answers
85
views
dgCMatrix is not an extension of class matrix?
I am working on a program which works with Volesti package in R. I have a large matrix (About 40k x 120k) and I want to use a function from Volesti which only accepts either class matrix or classes ...
- 63
0
votes
1
answer
39
views
Is it possible to subdivide the surface of a 3D Sphere (Earth) using 256 sided polygons?
Background for purpose sparse/compressible data structure/compressible honeycumbs, vague description to give you an idea: This would allow an array of [0..255] connection points hopefully. If all of ...
- 301
4
votes
1
answer
113
views
need to vectorize efficiently calculating only certain values in the matrix multiplication A * B, using a logical array L the size of A * B
I have matrices A (m by v) and B (v by n). I also have a logical matrix L (m by n).
I am interested in calculating only the values in A * B that correspond to logical values in L (values of 1s). ...
- 41
3
votes
2
answers
105
views
Fastest way to find first ZERO element of a sparse matrix
When using sparse matrices, it's easy to find non-zero entries quickly (as these are the only elements stored). However, what's the best way to find the first ZERO entry? Using approaches like find(X==...
- 639
4
votes
4
answers
363
views
How to improve efficiency of my python function involving sparse matrices?
I want to implement a particular function in python involving sparse matrices and want to make it as fast as possible. I describe in detail what the problem is and how I implemented it so far.
Problem:...
- 167
1
vote
0
answers
134
views
Optimizing the running time of solving an SDP in CVXPY
I am trying to optimize the following part of my code that involves solving a semi-definite programming (SDP) in CVXPY.
# optimization variables X and V
V = cp.Variable((p,p))
X = cp.Variable((num,p))...
- 33
2
votes
1
answer
138
views
Optimizing the construction of a large matrix in NumPy
I am working on a part of my code where I need to construct a matrix S1 of size num x num, where num = d^2 is typically very large i.e., > 6000. Below is the part of the code.
S1 = np.zeros((num, ...
- 33
1
vote
0
answers
35
views
Calculating CrossEntropyLoss over sparse tensors
While the following works as expected:
import torch
from torch.nn import CrossEntropyLoss
loss = CrossEntropyLoss()
T1 = torch.randn(10, 20, 4, 4) # The BatchSize is a 10 times repeated tensor (1,20,...
- 1,076
6
votes
5
answers
222
views
First index/value of sparse array
I have a sparse array in bash where I don't know the value of the lowest numbered index, and I want to pull that value out with its index. How do I find it?
#/bin/bash
a=( [12]="blue" [31]=&...
- 1,759
2
votes
1
answer
142
views
Generalized Nonsymmetric Eigensolver Python
How do I solve a nonsymmetric eigenproblem. In terms of scipy.sparse.linalg.eigsh the matrix needs to be "real symmetric square matrix or complex Hermitian matrix A" (https://docs.scipy.org/...
- 23
2
votes
1
answer
71
views
Very similar functions have noticeably different running times
I have these 2 functions to find the left null space of a matrix over GF(2) for the Quadratic Sieve Factoring Algorithm, given a list of integers(each of which represent a bitarray):
import random
...
- 1,429
1
vote
1
answer
58
views
superlu causing problems with mlpack in Rcpp
I have the following Rcpp code
#include <RcppArmadillo.h>
#define _USE_MATH_DEFINES
#include<cmath>
#include <boost/math/special_functions/bessel.hpp>
#include <mlpack.h>
#...
- 115
4
votes
0
answers
176
views
Convergence of Coordinate Descent
I am trying to implement the algorithms described in the paper Learning Fast Approximations of Sparse Coding and completed the ISTA and FISTA algorithms and the dictionary update rule using projected ...
1
vote
0
answers
48
views
Can't generate a simple CuSparseMatrixCOO using CUDA from Julia
At the moment I'm trying to build a COO using the CUSPARSE library. Looking at the documentation, and the function arguments, the way to create this is using the row and column indices followed by the ...
- 83
4
votes
0
answers
61
views
Efficiently combining sparse and dense sub matrices with R Matrix package
The following code will create a sparse matrix w and a dense matrix v and combines them horizontally. The result is efficiently stored, preserving the sparsity in w. Is this the best way to do this? ...
- 2,495
0
votes
1
answer
66
views
Sparse matrix in pandas/scipy with row and column indices
I have a dataframe in pandas that looks like this:
>>> df[['BranchNumber', 'ModelArticleNumber', 'ActualSellingPrice']].info()
<class 'pandas.core.frame.DataFrame'>
Index: 447970 ...
- 4,041
0
votes
1
answer
131
views
Problem in finding the determinant of a 21by21 matrix using a C++ program
I have been using a C++ program to calculate the deteminant of a nbyn
square matrix. It works well for lower orders ( I have got exact results for n=6). I need to calculate the determinant of a 21by21 ...
- 11
0
votes
0
answers
45
views
SPMV functions in MKL Library cannot work properly on my computer. How can I fix it?
This is my first time working with the MKL library. After referring to the official documentation, I obtained the latest version of the MKL library via the apt package manager in the WSL2 Ubuntu ...
1
vote
1
answer
477
views
PyTorch Error: Calling linear solver with sparse tensors requires compiling PyTorch with CUDA cuDSS and is not supported in ROCm build
I'm trying to use torch.sparse.spsolve to solve a linear system of equations as follows:
A_sparse = torch.sparse_coo_tensor(indices, values, size=(eq_counter, self.num_regions))
A_sparse_csr = ...
- 105
0
votes
1
answer
202
views
SciPy sparse matrix csr_matrix and csc_matrix functions - how much intermediate memory do they use?
For the scipy functions csr_matrix, and csc_matrix, and specifically the forms that take the row and column indices:
csr_matrix((data, (row_ind, col_ind)), [shape=(M, N)])
csc_matrix((data, (row_ind, ...
- 27.3k
0
votes
0
answers
51
views
Optimising log-linear learning implementation for multiple players in Python
I am working on testing log-linear learning for the class of potential games, and I want to test it with more than two players. I am having issues optimising the code.
Currently, I am utilising the ...
- 41
1
vote
1
answer
59
views
scipy coo_matrix consider some small numbers as 0.0 but still put in sparse matrix
I use some python and numpy/scipy code and produce a sparse array:
<COOrdinate sparse array of dtype 'float64'
with 21 stored elements and shape (403, 196)>
Coords Values
(206, 138) ...
- 177
0
votes
1
answer
122
views
HackerRank Bill division: Time limit exceeded in Bash
I'm playing on HackerRank, and I'd like to solve the Bill division exercise in BASH. It is a really simple problem, I've already solved it in multiple languages.
My initial Bash solution was this:
#!/...
- 3,847
0
votes
0
answers
140
views
Efficiently compute the diagonal of the inverse of a sparse matrix
In Python have a large NxN sparse matrix representing the precision matrix Q (inverse covariance matrix) of a multivariate Gaussian distribution. I am interested in obtaining the marginal variances of ...
4
votes
2
answers
166
views
How to make Sparce Matirx operations faster?
Sparse matrix are matrices that the vast majority of elements are 0s, with only a few non-zero elements.Now I have to fill my Sparsematrix class such that matrices can do add,subtract,and multiply.
I ...
- 41
3
votes
0
answers
282
views
Solving sparse linear system on GPU is much slower than CPU
Below is the code which solves a sparse linear system:
import cupyx
import cupyx.scipy.sparse.linalg
import time
import scipy
import scipy.sparse.linalg
import pathlib
file_dir = str(pathlib.Path(...
- 31
1
vote
1
answer
269
views
Error ParallelMapDatasetV2 when trying to predict a model using Keras
I'm trying to train a simple classification model using Keras that works with text and has two output classification: 0 or 1. I transformed my text data using TfidVectorizer() from SKlearn and ...
1
vote
1
answer
76
views
python and sparse matrix made out of 2X2 matrices
I can straight forwardly create the matrix C as (where the elements 0, A, B are 2X2 matrices)
[ 0 A 0 0 0 0 ... 0 0
B 0 A 0 0 0 ... 0 0
0 B 0 A 0 0 ... 0 0
0 0 B 0 A 0 .... ...
- 165
0
votes
1
answer
85
views
Memory Layout for Sparse matrix
I have a very specific sparse matrix layout and I am looking for storage recommendations.
The matrices I consider
are symmetric and positive definite
made up of block matrices (all blocks have the ...
- 1,383
0
votes
2
answers
174
views
Efficient Hadamard product of matrix-matrix multiplication
I am trying to compute in Python the following operation: (AB).C, where A,B,C are sparse matrices, and with the lower dot " ." I indicated the Hadamard (entry-wise) product of matrices, ...
3
votes
1
answer
129
views
Large matrices for the quantum Ising Model
I made a Python code that computes the density matrix of states for the Ising model with a transverse field and then calculates the von Neumann entropy for the system. The problem is that the density ...
0
votes
0
answers
34
views
Problematic destructor in a circularly linked sparse matrix program [duplicate]
This is not a homework problem but for self-study.
I am writing a program that reads in user input and transforms it into a sparse matrix (operator>>), adds two sparse matrices (operator+), and ...
- 45
1
vote
1
answer
128
views
How to convert a graph into a sparse adjacency matrix having a given type (other then "dgCMatrix")
How to convert a graph to a sparse ‘dgTmatrix’ in one step?
library(igraph)
library(Matrix)
g <- make_ring(5)
adj <- as_adjacency_matrix(g, sparse = TRUE)
class(adj)
# [1] "dgCMatrix"
#...
- 1,602
0
votes
1
answer
121
views
Plot a path in a very large 2D matrix using Matplotlib
I want to plot a path in a very large (e.g. 500K x 500K) 2D matrix. My current approach is creating an uint8 np array (for space-saving reasons, as I only have 4 possible values) and plotting it with ...
- 21
0
votes
0
answers
53
views
Efficiently multiply a special sparse 3D tensor with a vector in python
I have a 3D tensor T of size (nrows, ncols, ncols), which I would like to multiply with a vector x of size nrows.
However, T will be a huge sparse tensor with 6×6 nonzero entries in each row;
thus, it ...
- 203