saving Keras augmented data as a numpy array

Question

using keras ImageDataGenerator, we can save augmented images as png or jpg :

    for X_batch, y_batch in datagen.flow(train_data, train_labels, batch_size=batch_size,\
                save_to_dir='images', save_prefix='aug', save_format='png'):

I have a dataset of the shape (1600, 4, 100,100), which means 1600 images with 4 channels of 100x100 pixels. How can I save the augmented data as numpy array of shape (N,4,100,100) instead of individual images?

You want to save each batch in a file? Like np.save('batch.npy', X_batch) ? — niklascp
– niklascp, Commented Aug 10, 2017 at 13:52
You can't. Read the documentation: flow(x, y): Takes numpy data & label arrays, and generates batches of augmented/normalized data. Yields batches **indefinitely, in an infinite loop**.. Although, you could probably exact only the first M batches and join them together. — Imanol Luengo
– Imanol Luengo, Commented Aug 10, 2017 at 14:33

Yu-Yang · Accepted Answer · 2017-08-10 15:40:23Z

3

Since you know the number of samples = 1600, you can stop datagen.flow() as long as this number is reached.

augmented_data = []
num_augmented = 0
for X_batch, y_batch in datagen.flow(train_data, train_labels, batch_size=batch_size, shuffle=False):
    augmented_data.append(X_batch)
    num_augmented += batch_size
    if num_augmented == train_data.shape[0]:
        break
augmented_data = np.concatenate(augmented_data)
np.save(...)

Note that you should set batch_size properly (e.g. batch_size=10) so that no extra augmented images are generated.

answered Aug 10, 2017 at 15:40

Yu-Yang

14.6k2 gold badges57 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

FJ_Abbasi Over a year ago

So I can keep 'num_augmented' as large as I want? Will that be a right approach to create a large training dataset from less data?

Yu-Yang Over a year ago

Umm...from my own experience, I feel like there's not too much information you could "squeeze out of" your data by image augmentation. Image augmentation makes your model more robust by applying small random transformations on the data. If you distort the image too much, your model may learn some undesirable patterns from it. The effect of a large num_augmented is more like running through the same dataset for several epochs, rather than a several-times-larger dataset.

Collectives™ on Stack Overflow

saving Keras augmented data as a numpy array

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related