Adding an LSTM layer to my autoencoder in keras

Question

I have this auto-encoder model built in keras with tensorflow backend

encoded = Dense(units=600, activation='relu')(input_img)
encoded = Dense(units=500, activation='relu')(encoded)
encoded = Dense(units=bottleneck, activation='relu')(encoded)
decoded = Dense(units=500, activation='relu')(encoded)
decoded = Dense(units=600, activation='relu')(decoded)
decoded = Dense(units=img_size, activation='sigmoid')(decoded)

The input to the encoder is a one-dimensional array i.e [1,2,3,4,5] or in other words, a vector of an image

I'd like to add an LSTM layer in to improve my results, however, my understanding it that an LSTM requires 3 dimensional data, and I want to keep my data as a vector. Could someone give me an example of how I could integrate a layer like this using reshaping perhaps? All of my attempts thus far have failed.

I don't think an LSTM is suitable for an image because you don't predict pixel N from the previous k pixels. LSTMs are more suitable for NLP tasks because meaning is derived from sequences of words. But if you really want to do this, your vector should be [[1], [2], ..., [5]] — BallpointBen
– BallpointBen, Commented Mar 12, 2020 at 20:49
My image is a spectrogram, and I hoped to feed it to the lstm sequentially some how??? and then have the lstm return a 1d vector for the rest of the model — William Dwinnell
– William Dwinnell, Commented Mar 12, 2020 at 21:03

Ronakrit W. · Accepted Answer · 2020-03-12 20:45:16Z

1

keras LSTM require dimension (number of batch, time step, features) so you need to change from your 1D array which I assume it represents, 1 example, 1 feature and 5 time steps, [1,2,3,4,5] to (1,5,1) from (your number of example, number of time step, number of feature) np.array([1,2,3,4,5]).reshape((1,5,1)). And you need to reshape the output also, to (number of batch, time step, predicted value), for example, [2,3,4,5,6] to np.array([2,3,4,5,6]).reshape((1,5,1)). So, now you can add LSTM.

Example

number_of_steps = 5
number_of_feature = 1
latent_dimension = 10
bottleneck = 2
input_of_model= Input(shape=(number_of_steps,number_of_feature))

# just in case you want state_h, state_c for something else
lstm_output, state_h, state_c = LSTM(latent_dimension, return_sequences=True, return_state=True)(input_of_model)

encoded = Dense(units=600, activation='relu')(lstm_output)
encoded = Dense(units=500, activation='relu')(encoded)
encoded = Dense(units=bottleneck, activation='relu')(encoded)
decoded = Dense(units=500, activation='relu')(encoded)
decoded = Dense(units=600, activation='relu')(decoded)
decoded = Dense(units=number_of_feature, activation='sigmoid')(decoded)

M = Model(inputs=[input_of_model], outputs=[decoded])
M.compile(keras.optimizers.Adam(),loss='mse')
M.summary()
M.fit(x=np.array([1,2,3,4,5]).reshape((1,5,1)), y=np.array([2,3,4,5,6]).reshape((1,5,1)), epochs=200)

answered Mar 12, 2020 at 20:45

Ronakrit W.

7034 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

William Dwinnell Over a year ago

Would this work if I have a list of vectors to train off of? or only with one vector?

Ronakrit W. Over a year ago

yes, you need to convert all of your input data to 3D tensor and specify your batch size

William Dwinnell Over a year ago

I will attempt to figure this out myself, although I keep getting errors when trying to reshape my data

William Dwinnell Over a year ago

Also why would the last layer of the decoded layer be the number of features? I'd expect it to be the number of timesteps

Ronakrit W. Over a year ago

because in this example, it try to predict the value of the next timestep which happen to have the same length as input because you try to build autoencoder.

|

Collectives™ on Stack Overflow

Adding an LSTM layer to my autoencoder in keras

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related