Accessing tensor numpy array using `dataset.map()` in tensorflow

Question

I am trying to access the numpy array from a tensor object that is processed with https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map.

I get the error: AttributeError: 'Tensor' object has no attribute 'numpy'

When I try to access the tensor as: np_array = tensor.numpy()

While if I use: dataset.take(n), i am able to access the numpy array.

For more clarity on the situation I am facing, here is a short reproducible example of the error in a google colab:

https://colab.research.google.com/drive/13ectGEMDSygcyuW4ip9zrWaHO3pSxc3p?usp=sharing

Tensorflow version: 2.4.1

Update: Adding code in addition to the colab above:

import os
import numpy as np
import tensorflow as tf

# This works
def get_spectrogram_and_label_id(file_path):
    spectrogram, label = get_spectrogram(audio) # not showing the function here since it is irrelevant
    return spectrogram, label

# This doesn't!
def get_spec_and_label_time(spectrogram, label):
    time_step_spectrogram = generate_time_step_samples(spectrogram)
    return time_step_spectrogram, label

# I want to manipulate the Tensor by extracting the numpy array as part of the map function
def generate_time_step_samples(tensor):
    np_array = tensor.numpy() # ERROR: AttributeError: 'Tensor' object has no attribute 'numpy'
    # Do something with the numpy array
    return np_array

filenames = ['file1.wav', 'file2.wav', ...]
files_ds = tf.data.Dataset.from_tensor_slices(filenames)
spectrogram_ds = files_ds.map(get_spectrogram_and_label_id) # this works
spectrogram_time_ds = spectrogram_ds.map(get_spec_and_label_time) # this doesn't

More details in the google colab.

@NicolasGervais Added code excerpt, with more details in the google colab: colab.research.google.com/drive/… — vbfh
– vbfh, Commented Apr 28, 2021 at 17:35

Timbus Calin · Accepted Answer · 2021-04-28 19:02:39Z

4

You cannot access .numpy() inside a .map() function.

This is not a bug, it is how TensorFlow works with static graphs behind the scenes.

Read my answer here for a more comprehensive explanation.

AttributeError: 'Tensor' object has no attribute 'numpy' in Tensorflow 2.1

answered Apr 28, 2021 at 19:02

Timbus Calin

15.2k6 gold badges49 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

vbfh Over a year ago

Thanks for the reference @TimbusCalin , this helps. However, I am struggling to implement what I am trying to do. I tried replacing: spectrogram_time_step_ds = spectrogram_ds.map(get_time_step_spectrogram_and_label_id) by

spectrogram_time_step_ds = tf.py_function(func=get_time_step_spectrogram_and_label_id, inp=[spectrogram_ds], Tout=[tf.int64, np.ndarray])

but get TypeError: Expected DataType for argument 'Tout' not <class 'numpy.ndarray'>. I still need to access the numpy array but not sure how

Timbus Calin Over a year ago

What happens if you write data type exactly, like np.int32/np.float32?

Collectives™ on Stack Overflow

Accessing tensor numpy array using `dataset.map()` in tensorflow

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related