Can't use Tensorflow.js predict() function

Question

I've trained my own graph model. And I want to use it on browser. Here is my code:

async function predict() {
        const model = await tf.loadGraphModel('./model/model.json');
        let img = document.getElementById('test');
        var example = tf.browser.fromPixels(img);
        example = example.expandDims(0);
        const output = await model.predict(example).data();
        console.log(output);
    }

When I run this, it gives this error on console:

Uncaught (in promise) Error: This execution contains the node 'SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/Exit_4', which has the dynamic op 'Exit'. Please use model.executeAsync() instead. Alternatively, to avoid the dynamic ops, specify the inputs [SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack_2/TensorArrayGatherV3]
    at t.compile (tfjs:2)
    at t.execute (tfjs:2)
    at t.execute (tfjs:2)
    at predict ((index):85)
    at /websites/optik2/async http://localhost/websites/optik2/:96

I need predict() function, executeAsync() is not good as it is.

EDIT

Ok, I'm using as executeAsync now as @Jason Mayes says. But it's returning some values like that:

t {kept: false, isDisposedInternal: false, shape: Array(3), dtype: "float32", size: 1200, …}
rank: 3
isDisposed: false
kept: false
isDisposedInternal: false
shape: (3) [1, 300, 4]
dtype: "float32"
size: 1200
strides: (2) [1200, 4]
dataId: {}
id: 2198
rankType: "3"
scopeId: 3545
__proto__: Object

How can I get the bounding box of this?

Ok, I'm using it now. And do you know how can I get bounding box from executeAsync. It's my own model — sundowatch
– sundowatch, Commented Mar 4, 2020 at 6:10
What is returned currently? As it is async you need to wait for it to complete before you will populate output. Updated answer below. — Jason Mayes
– Jason Mayes, Commented Mar 4, 2020 at 6:15

NorahKSakal · Accepted Answer · 2020-04-14 10:48:57Z

What is the length of the output from const output = await model.executeAsync(data)?

You should look for these shapes in the output;

output[X] = detection_boxes   // shape: [1, x, 4]  x: number of bounding boxes
output[Y] = detection_scores  // shape: [1, x]     x: number of scores
output[Z] = detection_classes // shape: [1, x]     x: number of classes

Then you can fetch the predictions through the following;

const boxes = output[0].dataSync()
const scores = output[1].arraySync()
const classes = output[2].dataSync()

You can then construct a prediction object with all the predicted bounding boxes by doing this;

buildDetectedObjects(scores, threshold, imageWidth, imageHeight, boxes, classes, classesDir) {
    const detectionObjects = []
    scores.forEach((score, i) => {
      if (score > threshold) {
        const bbox = [];
        const minY = boxes[i * 4] * imageHeight;
        const minX = boxes[i * 4 + 1] * imageWidth;
        const maxY = boxes[i * 4 + 2] * imageHeight;
        const maxX = boxes[i * 4 + 3] * imageWidth;
        bbox[0] = minX;
        bbox[1] = minY;
        bbox[2] = maxX - minX;
        bbox[3] = maxY - minY;

        detectionObjects.push({
          class: classes[i],
          label: classesDir[classes[i]].name,
          score: score.toFixed(4),
          bbox: bbox
        })
      }
    })

    return detectionObjects
  }

classesDir being a dictionary with the classes from the training;

let classesDir = {
    1: {
        name: 'Class name 1',
        id: 1,
    },
    2: {
        name: 'Class name 2',
        id: 2,
    }
}

The prediction object will then be an array with objects;

[{
  bbox:[x,y,width,height],
  class: X,
  label: class name,
  score: 0.XYZ
},
{
  bbox:[x,y,width,height],
  class: X,
  label: class name,
  score: 0.XYZ
}]

Jason Mayes · Accepted Answer · 2020-03-04 06:19:36Z

2

Try model.executeAsync() instead of predict.

As it is async you should use:

const output = await model.executeAsync(data);

edited Mar 4, 2020 at 6:19

answered Mar 4, 2020 at 6:07

Jason Mayes

3012 silver badges8 bronze badges

Comments

edkeveked · Accepted Answer · 2020-03-04 09:04:17Z

1

You are using console.log to display a tensor.

console.log(tensor)

Instead you would need to use the print method of the tensor to see its output.

tensor.print()

After getting the tensor from the backend, then console.log can be used to display the output as a plain js array

data = await tensor.data()
console.log(data) // plain js array

answered Mar 4, 2020 at 9:04

edkeveked

18.4k10 gold badges60 silver badges95 bronze badges

8 Comments

sundowatch Over a year ago

Thanks for you your reply. tensor.data() or tensor.dataSync(), only returns Float32Array. How can I get the bounding boxes?

edkeveked Over a year ago

If you use data or dataSync you will have an array that contains all the data. But if you want the array to keep the same shape as the tensor, you can rather use array or arraySync

sundowatch Over a year ago

What about bounding box? How can I interpret the result?

edkeveked Over a year ago

Looking at the shape [1, 300, 4] you have 300 bounding boxes or detections in the image.

sundowatch Over a year ago

How can I access those?

|

Collectives™ on Stack Overflow

Can't use Tensorflow.js predict() function

3 Answers 3

Comments

Comments

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related