I have multiple numpy arrays (.npy) in a folder. Is there a way to read all of them automatically in Python? Or do I need to enter their names manually? I couldn't find information related to reading multiple numpy arrays from a folder for Python.
Add a comment
|
2 Answers
You can use glob to grab all the .npy files matching a specified *.npy pattern. glob.glob returns a list of pathnames and glob.iglob returns an iterator instead of storing all the pathnames simultaneously (will be useful if you have a huge amount of files). Here is a small example:
Code:
import os
import glob
import numpy as np
# Let's create folder
folder = './np_arrays'
try:
os.mkdir(folder)
except OSError:
print('Folder exists!')
# Some dummy arrays
a = np.zeros((1, 5))
b = np.ones((1, 5))
# Save them as .npy to the created folder
np.save(os.path.join(folder, 'a'), a)
np.save(os.path.join(folder, 'b'), b)
# Getting all the numpy arrays .npy files based on matching pattern (*.npy)
file_paths = glob.glob(os.path.join(folder, '*.npy'))
print(file_paths)
# Import arrays from folder and store them as a dict
array_dict = {os.path.basename(f)[0]: np.load(f) for f in file_paths}
print(array_dict)
Output:
['./np_arrays/a.npy', './np_arrays/b.npy']
{'a': array([[0., 0., 0., 0., 0.]]), 'b': array([[1., 1., 1., 1., 1.]])}
Comments
You can always loop over all files that have the extension you are looking for and store them in a data structure that makes sense for your use case
For example to store them all in a dictionary where the keys are the filenames:
arrays = {}
for filename in os.listdir(dir):
if filename.endswith('.npy'):
arrays[filename] = load_array(filename)
1 Comment
botloggy
Thanks for the solution. I think in the current version,
np.load would work instead of load_array.