Use in Python trained tensorflow keras model in Java for image recognition

Question

I use keras in Python to do my first steps in neuronal networks. I trained a model to recognize small 30px x 30px images.

import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.python.keras import Sequential

train_ds = keras.preprocessing.image_dataset_from_directory('images', validation_split=0.2,
  subset="training", seed=123, batch_size=32, image_size=(30, 30))
val_ds  = keras.preprocessing.image_dataset_from_directory('images', validation_split=0.2,
  subset="validation", seed=123, batch_size=32, image_size=(30, 30))

class_names = train_ds.class_names
print(class_names)

AUTOTUNE = tf.data.experimental.AUTOTUNE

train_ds = train_ds.cache().prefetch(buffer_size=AUTOTUNE)
val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)
num_classes = 13

model = Sequential([
  layers.experimental.preprocessing.Rescaling(1./255, input_shape=(30, 30, 3)),
  layers.Conv2D(10, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(15, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(20, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Dropout(0.2),
  layers.Flatten(),
  layers.Dense(64, activation='relu'),
  layers.Dense(num_classes)
])

model.compile(
  optimizer='adam',
  loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
  metrics=['accuracy'])

model.summary()

epochs = 50

history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)

model.save("model.h5")

I'm pretty happy with that because I have a val_accuracy of 0.9912.

But now I want to use the model in Java to categorize buffered images:

public class Test
{
    public static void main(String[] args) throws Exception
    {
        MultiLayerNetwork model = KerasModelImport.importKerasSequentialModelAndWeights("ki.h5");
        BufferedImage img = ImageIO.read(new File("testImage.png"));
        ImageLoader loader = new ImageLoader(30, 30, 3);
        INDArray input = loader.asMatrix(img);
        INDArray output = model.output(input);
        System.out.println("Test " + output);
    }
}

Here I receive this error:

Exception in thread "main" org.deeplearning4j.nn.modelimport.keras.exceptions.UnsupportedKerasConfigurationException: Unsupported keras layer type Rescaling. Please file an issue at https://github.com/eclipse/deeplearning4j/issues.

I got it working in Python by removing the Rescaling Layer. But then I got the exception

Unknown Keras loss function sparsecategoricalcrossentropy. Please file an issue at https://github.com/eclipse/deeplearning4j/issues.

I'm open to any suggestions. Is there a better way to usethe model from Java?

Or should I change something in my Python file or how I save the model?

Nimantha · Accepted Answer · 2023-11-14 05:57:29Z

1

I'm from the deeplearning4j project. From the looks of it, @Andreas Radauer is correct. This one slipped through the cracks when we were implementing missing import functions.

For users willing to do it, it is also possible to register custom layers with model import to work around this. You can use our tensorflow like api (samediff) and define the function that implements this yourself and register it as a layer. If you think that is suitable for your use case then you can see an example here: https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-modelimport/src/test/java/org/deeplearning4j/nn/modelimport/keras/e2e/KerasCustomLossTest.java

Otherwise, we'll look in to adding the missing functions. Most of the keras work has focused on the convolution layers and ensuring we get those right.

edited Nov 14, 2023 at 5:57

Nimantha

6,5276 gold badges32 silver badges78 bronze badges

answered Oct 2, 2021 at 22:33

Adam Gibson

3,2151 gold badge12 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Andreas Radauer Over a year ago

Thank you. I think for my first problem there is already an open issue github.com/eclipse/deeplearning4j/issues/9447

Andreas Radauer Over a year ago

I opened a new issue for the 2nd problem of the missing sparsecategoricalcrossentropy loss function github.com/eclipse/deeplearning4j/issues/9468

Andrew-Harelson · Accepted Answer · 2021-10-02 21:32:35Z

0

deeplearning4j looks like it does not support defining your loss function as an object. You have to specify it with a string. https://github.com/eclipse/deeplearning4j/issues/8990

Try changing

model.compile(
  optimizer='adam',
  loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
  metrics=['accuracy'])

to

model.compile(
  optimizer='adam',
  loss='sparse_categorical_crossentropy',
  metrics=['accuracy'])

Note that using this method you cannot set the kwarg from_logits. I think this will probably break your model. If that's the case, you'll need to file an issue with them because it's a bug in their library.

answered Oct 2, 2021 at 21:32

Andrew-Harelson

1,0985 silver badges11 bronze badges

1 Comment

Andreas Radauer Over a year ago

thank you for your answer. you are right, now I cant't set from_logits accuracy is down to 7% and i got a new error in java: Got rank 3 array as input to ConvolutionLayer (layer name = conv2d, layer index = 0) with shape [3, 30, 30]. Expected rank 4 array with shape [minibatchSize, layerInputDepth, inputHeight, inputWidth]. (layer name: conv2d, layer index: 0, layer type: ConvolutionLayer) I thought reusing the model is much easier

Collectives™ on Stack Overflow

Use in Python trained tensorflow keras model in Java for image recognition

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related