Python autoencoder returns mixed up image

Question

I'm trying to create Python autoencoder image compression but all I'm getting is the mixed up images.

Here's my code:

path1 = 'C:\\Users\\klaud\\Desktop\\images\\'
all_images = []
subjects = os.listdir(path1)
numberOfSubject = len(subjects)
print('Number of Subjects: ', numberOfSubject)
for number1 in range(0, numberOfSubject):  # numberOfSubject
  path2 = (path1 + subjects[number1] + '/')
  sequences = os.listdir(path2)
  numberOfsequences = len(sequences)
  for number2 in range(0, numberOfsequences):
    path3 = path2 + sequences[number2]
    img = cv2.imread(path3, 0)
    img = img.reshape(512, 512, 1)
    all_images.append(img)

x_train = np.array([all_images[0], all_images[1]])
x_test = np.array(all_images[2:])


print("X TRAIN \n")
print(x_train)
print("X TEST \n")
print(x_test)


x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.

print (x_train.shape)
print (x_test.shape)

latent_dim = 4

class Autoencoder(Model):
  def __init__(self, latent_dim):
    super(Autoencoder, self).__init__()
    self.latent_dim = latent_dim
    self.encoder = tf.keras.Sequential([
      layers.Flatten(),
      layers.Dense(latent_dim, activation='relu'),
    ])
    self.decoder = tf.keras.Sequential([
      layers.Dense(262144, activation='sigmoid'),
      layers.Reshape((512, 512))
    ])

  def call(self, x):
    encoded = self.encoder(x)
    decoded = self.decoder(encoded)
    return decoded

autoencoder = Autoencoder(latent_dim)

autoencoder.compile(optimizer='adam', loss=losses.MeanSquaredError())

autoencoder.fit(x_train, x_train,
                epochs=10,
                shuffle=True,
                validation_data=(x_test, x_test))

encoded_imgs = autoencoder.encoder(x_test).numpy()
decoded_imgs = autoencoder.decoder(encoded_imgs).numpy()



n = 6
plt.figure(figsize=(20, 6))
for i in range(n):
  # display original
  ax = plt.subplot(2, n, i + 1)
  plt.imshow(x_test[i])
  plt.title("original")
  plt.gray()
  ax.get_xaxis().set_visible(False)
  ax.get_yaxis().set_visible(False)

  # display reconstruction
  ax = plt.subplot(2, n, i + 1 + n)
  plt.imshow(decoded_imgs[i])
  plt.title("reconstructed")
  plt.gray()
  ax.get_xaxis().set_visible(False)
  ax.get_yaxis().set_visible(False)
plt.show()

This is what I'm getting:

I have no idea whether it's a autodecoder problem or is there a problem with the way matplotlib is showing up plots? I tried to change almost everything and if I'm not getting the errors while compiling the program then it's mixed up images. I'd apprecatie any helpful advice!

Tim · Accepted Answer · 2023-02-02 15:06:25Z

1

This looks like you only train on two images:

x_train = np.array([all_images[0], all_images[1]])

In that case, the network can reach a (local) minimum by learning to reproduce the tiny input space - and just spits out a mixture of both images regardless of input.

To fix this, you absolutely need a larger training data set! It does not need to be ImageNet, but maybe something like the Stanford dogs would be a good fit.

answered Feb 2, 2023 at 15:06

Tim

3461 silver badge7 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

clautsick Over a year ago

Hi! I tried with larger dataset and it keeps mixing up the images :O I'm at loss here and don't know what to do...

Tim Over a year ago

How many images have you put into the dataset? What's the output of print(x_train) ? You could look into ready-made tensorflow datasets: tensorflow.org/datasets Also, your latent dimension is really small, probably too small to correctly reconstruct anything.

clautsick Over a year ago

So I tried to do my own dataset instead of minst dataset right. I inserted like 100 pictures, split them into test and training and changed latent dimension to even 512 but the images are still mixed up. Ironically it's the task that I should perform on like 8 pictures (uni stuff lmao)

Tim Over a year ago

You should start with smaller images. Does it work with 64x64 images? 512 hidden unity is still really small for fully connected hidden layers - you try to reconstruct 262144 pixels per image... Also, what do you mean with mixed up, do you still see two overlapped images?

clautsick Over a year ago

Tried this on 64x64 images - still not working. I mean the images are overlapping :(

|

Collectives™ on Stack Overflow

Python autoencoder returns mixed up image

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related