3,933 questions
4
votes
1
answer
115
views
Numpy vectorising n-variable function by index
Given a closed-form function f(x,y,z), what is the most efficient way to fill up a 3D ndarray arr[z][y][x] with the values of f(x,y,z)?
For example, if I wanted a 3D ndarray arr[z][y][x] with each ...
- 51
4
votes
2
answers
194
views
Python multiprocessing shared memory seems to hang or crash when interacting with small numpy arrays
I've been trying to parallelize some code that I wrote in python. The actual work is embarrassingly parallel, but I don't have much experience with multiprocessing in Python.
The actual code I'm ...
- 116
4
votes
1
answer
169
views
Weird behavior in large complex128 NumPy arrays, imaginary part only [closed]
I'm working on numerical simulations. I ran into an issue with large NumPy arrays (~ 26 GB) on Linux with 128 GB of RAM. The arrays are of type complex128.
Arrays are instantiated without errors (if ...
2
votes
2
answers
65
views
List of points [ (x,y), …] to np.array()
I have a list of points such as [(x, y), …]
I want to convert it in a numpy.array() such as [[0,0,1,…], …], 1 representing a point of the list.
According that np.array().shape is : ( max(x)-min(x)+1, ...
- 111
2
votes
2
answers
142
views
Making a random grid that doesn't vary 'too much'
I'm making a meshgrid to represent background radiation levels, but I'd like it to be less noisy. My implementation is trivial, I've plotted
import numpy as np
x = np.linspace(-2, 2, 100)
y = np....
1
vote
1
answer
81
views
Arbitrary Stencil Slicing in Numpy
Is there a simple syntax for creating references to an arbitrary number of neighbouring array elements in numpy?
The syntax is relatively straightforward when the number of neighbours is hard-coded. A ...
- 440
1
vote
1
answer
99
views
Numpy __array__ not working when return array with single item
Because plain dict is not adequate for inheritance, I design the following MyDict with UserDict in python standard libraries:
import numpy as np
from collections import UserDict
class MyUserDict(...
1
vote
0
answers
16
views
How do I expand an "named" n-dimensional matrix in n dimensions at once?
I start with a 0 by 0 matrix that I want to expand to a (1,1) matrix, then (2,2) etc. I also want to have the columns named, so the (0,0) matrix would increase to:
First
First 0
then
...
1
vote
1
answer
59
views
numpy.ndarray of seconds of a day to datetime format
I have a Python numpy.ndarray of seconds of the day with a bunch of decimal seconds:
import numpy as np
sec = [40389.66574375, 40390.12063928, 40391.32714992, 40392.64457077, 40393.48519607, 40394....
- 273
0
votes
0
answers
43
views
How to convert holoscan Tensor to ndarray
I have a holoscan Application, a MiddleOperator where I take a Tensor, i want to convert this Tensor to ndarray, do something with it, and emit to "out".
import os
from holoscan.core import ...
- 172
2
votes
2
answers
146
views
issue with sympy lambdify
I just want numerically integrate the Lorenz' famous 3 equations
I create this code that seems perfectly work:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import ...
- 37
0
votes
0
answers
58
views
subclass numpy arrays: when using resize, how to re-attach the view to sliced children arrays
I am trying to create a buffer array that creates extra indexes of the numpy so I can assign rather than using functions like np.append and where "children" sliced array can still be a view ...
3
votes
0
answers
217
views
Numpy IO seems to have an 2GB overhead in StorNex (cvfs) File System
TL;DR:
Initally I thought the numpy load functions doubles memory usage at peak
after some additional tests it seems like the underlying file system (StorNex [cfvs]) leads to a size-independent 2GB ...
- 496
0
votes
1
answer
81
views
Apparently weird condition on inclusion of endpoint in np.arange() [duplicate]
The numpy.arange function takes the three parameters: start, stop, step (positional args.)
The default step is 1.
Throughout my entire Numpy experience, the last element of the resultant array is not ...
1
vote
0
answers
110
views
PySide6 QImage conversion to PyQtGraph ndarray
I cannot generate a QImage with some text on it, keep it in memory, and then successfully display it on a pyqtgraph.ImageItem which needs it as an np.ndarray
import sys
import numpy as np
import ...
- 413
1
vote
1
answer
61
views
Pandas dataframe assign nested list not working
I'm trying to assign a dataframe cell with a nested list:
df.loc['y','A'] = [[2]]
However, the actual assigned value is [2].
It works expected for [2], [[[2]]], [[[[2]]]], but just not for [[2]]
See ...
- 13
0
votes
2
answers
61
views
Identify identical vectors as part of a multidimensional dot product
I am wanting to identify identical vectors after a dot product calc. The below works for a single dimension but not multi-dimensionally.
Single Dimension
a = np.array([0.8,0.5])
b = np.array([0.8,0.5])...
- 1,637
1
vote
1
answer
57
views
unexpected results when using normal and advanced indexing with numpy
Consider the following:
import numpy as np
a = np.random.rand(123, 45, 3)
print(a[:, :, [0, 1]].shape) # (123, 45, 2)
print(a[:, 0, [0, 1]].shape) # (123, 2)
print(a[0, :, [0, 1]].shape) # (2, 45)...
- 101
5
votes
1
answer
143
views
Is there a Numpy method or function to split an array of uint64 into two arrays of uint32
Say I have an array as follows:
arr = np.asarray([1, 2, 3, 4294967296, 100], dtype=np.uint64)
I now want two arrays, one array with the lower 32 bits of every element, and one with the upper 32 bits ...
- 360
1
vote
2
answers
206
views
'numpy.ndarray' object has no attribute 'groupby'
I am trying to apply target encoding to categorical features using the category_encoders.TargetEncoder in Python. However, I keep getting the following error:
AttributeError: 'numpy.ndarray' object ...
- 13
10
votes
1
answer
659
views
Arrays of size 0 in NumPy
I need to work with arrays that can have zeros in their shapes. However, I am encountering an issue. Here's an example:
import numpy as np
arr = np.array([[]])
assert arr.shape == (1,0)
arr.reshape((...
- 2,105
2
votes
1
answer
146
views
0-dimensional array problems with `numpy.vectorize`
numpy.vectorize conveniently converts a scalar function to vectorized functions that can be applied directly to arrays. However, when inputting a single value into the vectorized function, the output ...
- 145
1
vote
0
answers
80
views
Python Numpy Crash without error or warning with exit code -1073740791 due to _multiarray_umath.cp312-win_amd64.pyd
Numpy library crash without console error or warning constantly
In Python Console see notifications either Process finished with exit code -1073741819 (0xC0000005) or Process finished with exit code -...
- 11
3
votes
2
answers
70
views
How to reduce verbosity of self-documenting expressions in Python f-strings?
This script:
import numpy as np
a = np.array([2, 3, 1, 9], dtype='i4')
print(a)
print(f'{a=}')
produces:
[2 3 1 9]
a=array([2, 3, 1, 9], dtype=int32)
Is there a way to get just a=[2 3 1 9] from {a=}...
- 8,635
0
votes
1
answer
122
views
NumPy Stride Tricks: Is it possible to add the windows back into the original array size at the same location without for loops?
I'm currently trying to implement my own version of 2D Pooling (with channels included) in Python using only NumPy, but I've run into a bit of a roadblock with vectorizing the backpropagation process. ...
1
vote
1
answer
62
views
removing Nans from a 3D array without reshaping my data
I have a 3D array (121, 512, 1024) made up of frames of 512x1024 images.
The bottom several rows of the images have Nans which mess up my processing. I want to remove these and end up with something ...
-1
votes
1
answer
49
views
Sort columns of numpy unitary matrix such that highest element of each column is on the diagonal
Take a unitary matrix U. I want to swap the columns such that the largest element of each column (in absolute value) is on the diagonal (modulo ties). What is the best way to do this in numpy?
- 539
0
votes
1
answer
60
views
How to get row numbers of maximum elements in a 2D Numpy array? [duplicate]
I have a 2D array a given by:
a = np.array([[2, 3, 1, 9], [0, 5, 4, 7], [2, 4, 6, 8]])
[[2 3 1 9]
[0 5 4 7]
[2 4 6 8]]
I would like to get row numbers of maximum elements column-wise, i.e. given ...
- 8,635
0
votes
1
answer
74
views
Removing rows from numpy 3D array based on last element
What I'm trying to do is essentially removing all rows h,s in a 3D numpy array a if a[h,s,v] = some value for all v
More specifically, I have a loaded image from cv2 which contains some transparent ...
- 435
1
vote
2
answers
54
views
Counting the hashtags in a collection of tweets: two methods with inconsistent results
I'm playing around with a numpy dataframe containing two columns: 'tweet_text' and 'cyberbullying_type'. It was created through this dataset as follows:
df = pd.read_csv('data/cyberbullying_tweets.csv'...
- 494
3
votes
2
answers
67
views
Project one index from another array
I have an array a of size (M, N, K).
And I have an array b of size (M, N) with integer values of [0, K-1].
How do I get the array... c of size (M, N), where
c[i, j] == a[i, j, b[i, j]]
in the simplest ...
- 403
1
vote
3
answers
72
views
Have numpy.concatenate return proper subclass rather than plain ndarray
I have a numpy array subclass, and I'd like to be able to concatenate them.
import numpy as np
class BreakfastArray(np.ndarray):
def __new__(cls, n=1):
dtypes=[("waffles", int), ...
- 995
1
vote
3
answers
68
views
Can I create a multidimensional array containing a unit matrix without nested loops? [closed]
Suppose I have a Numpy array n indices, where the first n-2 represents some counting indices and the last 2 indices represent a square MxM matrix. I want to initialize this structure so it will ...
- 883
5
votes
4
answers
107
views
Slice a numpy 2d array using another 2d array
I have a 2D array of (4,5) and another 2D array of (4,2) shape. The second array contains the start and end indices that I need to filter out from first array i.e., I want to slice the first array ...
- 620
0
votes
0
answers
38
views
np.zeros becomes faster with a very large array size [duplicate]
%timeit -n 1000 -r 7 np.zeros(100000000, dtype=bool)
gives 4.22 ms
%timeit -n 1000 -r 7 np.zeros(1000000000, dtype=bool) # 1 more zero = 10 times longer array
gives 3.13 μs (not even milliseconds, ...
- 41
2
votes
1
answer
365
views
Efficiently custom array creation routines in JAX
I'm still getting a handle of best practices in jax. My broad question is the following:
What are best practices for the implementation of custom array creation routines in jax?
For instance, I want ...
- 539
1
vote
3
answers
80
views
Does Numpy return view or copy when combining slicing and advanced indexing?
The following snippet:
import numpy as np
x = np.arange(25).reshape(5, 5)
print(x.base)
y = x[:2, [0, 2]]
print(y.base)
outputs
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 ...
- 180
1
vote
2
answers
43
views
Form element-wise list from scalar and matrix
I have a zero-dimensional numpy scalar s and a two-dimensional numpy matrix m. I want to form a matrix of vectors in which all the elements of m are paired with s as in the following example:
import ...
- 539
2
votes
1
answer
149
views
How can I convert the datatype of a numpy array sourced from an awkward array
I have a numpy array I converted from awkward array by to_numpy() function, and the resulting array has the datatype: dtype=[('phi', '<f8'), ('eta', '<f8')]). I want to make it a regular tuple ...
- 33
0
votes
2
answers
111
views
Vectorization with if/else conditions Python
I am somehow new to python and would like to get rid of use of for loop as it makes the program very slow. So I would like vectorize it. however, I am not sure how would I use it with two conditions
...
- 11
4
votes
2
answers
168
views
How to Mark Repeated Entries as True Starting from the Second Occurrence Using NumPy?
Problem
I have a NumPy array and need to identify repeated elements, marking the second occurrence and beyond as True, while keeping the first occurrence as False.
For example, given the following ...
- 43
0
votes
0
answers
26
views
Is this expect behavior for NumPy indexing? If so, why? [duplicate]
I am using NumPy version 2.1.3 and Python 3.12.2. Say I define
ones_arr = np.ones((1, 2, 3))
Now I slice
ones_arr[0, :, [0, 1, 2]]
The result has shape (3, 2), but I would expect it to have shape (2,...
- 43
0
votes
1
answer
102
views
I'm trying to define NN model with Pytorch but there's a type error when I wanted to to convert y-train to y-train-tensor What should I do to fix it?
Here is my code. I import Wisconsin breast cancer database from kaggle to vscode. All dataset variables all numeric and float 64.
There is this type error:
(Exception has occurred: TypeError
can't ...
5
votes
2
answers
178
views
MODIS LST tiles concatenation or merging issue with xarray
I am having an issue while concatenating two xarray data files. The two files (MODIS LST, accessed through the STAC API) are from the same date/time but correspond to adjacent geographical tiles (side-...
0
votes
1
answer
40
views
Index a batch of numpy vectors with a batch of numpy index matrices [duplicate]
If I have a vector vec, I can index it with a matrix as follows:
import numpy as np
vec = np.asarray([1,2,3,4]) # Shape (4,)
mat = np.asarray([[0,2],
[3,1]]) # Shape (2,2)
result =...
- 539
1
vote
2
answers
56
views
Remove specific indices in each row of a numpy ndarray
I have integer arrays of the type:
import numpy as np
seed_idx = np.asarray([[0, 1],
[1, 2],
[2, 3],
[3, 4]], dtype=np.int_)
...
- 539
1
vote
2
answers
52
views
How to write a function for numpy array input whose action depends on the input's numerical value
The function has this mathematical form
f(x) = 1, if x<1
= g(x), for 1<=x<10
= 0, for x >=10
where g(x) is a simple function.
It is straightforward to write such a function if ...
- 21.1k
3
votes
2
answers
91
views
Adding 2D numpy arrays with differing axes arrays: how to properly replace the deprecated interp2d with RectBivariateSpline?
I need to add two 2D numpy arrays of possibly different shapes and different corresponding axes arrays.
What I mean is this: Lets define two different sets of x- and y-axes, and calculate the z-values ...
- 376
3
votes
1
answer
63
views
Modify numpy array of arrays
I have a numpy array of numpy arrays and I need to add a leading zero to each of the inner arrays:
a = [[1 2] [3 4] [5 6]] --> b = [[0 1 2] [0 3 4] [0 5 6]]
Looping through like this:
for item in a:...
- 33
2
votes
2
answers
77
views
Saving numpy array after indexing is much slower
I am running into an issue that saving a numpy array after indexing results in much slower saving. A minimal reproducible example can be seen below:
import time
import numpy as np
def mre(save_path):
...
- 23