How to call tensorflow_model_server from Nodejs, with an image

Question

This is the metadata of my serving model

"metadata": {"signature_def": {
 "signature_def": {
  "serving_default": {
   "inputs": {
    "vgg16_input": {
     "dtype": "DT_FLOAT",
     "tensor_shape": {
      "dim": [
       {
        "size": "-1",
        "name": ""
       },
       {
        "size": "224",
        "name": ""
       },
       {
        "size": "224",
        "name": ""
       },
       {
        "size": "3",
        "name": ""
       }
      ],
      "unknown_rank": false
     },
     "name": "serving_default_vgg16_input:0"
    }
   }...

Sadly I don't know how to talk to it from NodeJs. How to transorm a local image to a valid 224,224,3 DT_FLOAT Tensor ...

In python, i can do it with this code, but I would like the nodejs version

import numpy as np
import requests
from keras.preprocessing import image

image_path = './data/black/fhb2l97vdi8qc0rt5ow3.jpg'
img = image.img_to_array(image.load_img(image_path, target_size=(224, 224))) / 255.
img = img.astype('float16')

payload = {
    "instances": [{'vgg16_input': img.tolist()}]
}

r = requests.post('http://ip:port/v1/models/color:predict', json=payload)
print(r.content)

So far my code

var request = require('request');
var fs = require('fs');
var myImg = __dirname + '/../tensorflow2/data/black/0a13y2gtunswi8ox4bjf.jpg';



var options = {
    method: 'POST',
    url: `http://ip:port/v1/models/color:predict`,
    json:{ 
          instances: [{'vgg16_input': ??????}]
        }

};

request(options, function (err, resp, body) {
    if (err)
      cb(err);

      console.log(body);
  });

Maybe i could use some function from tensorflowjs ...

Michał Słapek · Accepted Answer · 2019-11-04 18:28:57Z

2

+200

The image must be passed in JSON as list of lists of floats (pixel is list of 3 RGB values, row is list of pixels and image is list of rows).

We need to decode and resize the JPEG image. Install sharp package with npm install sharp.

Image preparation is as follows:

const fs = require('fs');
const sharp = require('sharp');

function imgToJson(buffer) {
  var decoded = [];

  var h;
  var w;
  var line;
  var pixel;

  var b = 0;
  for (h = 0; h < 224; h++) {
      var line = [];
      for (w = 0; w < 224; w++) {
          var pixel = [];

          pixel.push(buffer[b++] / 255.0); /* r */
          pixel.push(buffer[b++] / 255.0); /* g */
          pixel.push(buffer[b++] / 255.0); /* b */

          line.push(pixel);
      }
      decoded.push(line);
  }

  return decoded;
}

async function prepare_image(imagePath) {
  var jpegData = fs.readFileSync(imagePath); /* for example sake synchronous */
  const buffer = await sharp(jpegData).resize(224, 224).raw().toBuffer();
  return imgToJson(buffer);
}

The result of prepare_image is future returning the list of lists of floats representing image. The last step is to perform request:

var request = require('request');

async function perform_request(imagePath) {
  var decoded = await prepare_image(imagePath);
  var options = {
      method: 'POST',
      url: 'http://ip:port/v1/models/color:predict',
      json: { 
            instances: [{'vgg16_input': decoded}]
      }
  };

  request(options, function (err, resp, body) {
      if (err)
        cb(err);

        console.log(body);
    });
}

perform_request("image.jpeg");

edited Nov 4, 2019 at 18:28

answered Nov 2, 2019 at 19:46

Michał Słapek

1,6121 gold badge12 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Laurent Debricon Over a year ago

Thank you. js answer : { predictions: [ [ 0.869870663, 0.109581947, 1.78813934e-7, 0.00818741322, 4.76837158e-7, 0.00215744972, 3.57627869e-7, 0.00000619888306, 8.14288654e-8, 0.00000102930289 ] ] } python answer "predictions": [[0.979003787, 2.9951334e-05, 8.94069672e-08, 0.000181376934, 7.18832e-05, 0.00502485037, 6.68466091e-05, 0.00051087141, 3.17406625e-06, 4.25011876e-07] weird that it's not the same prediction ...

Laurent Debricon Over a year ago

Second alternative : { error: 'Input to reshape is a tensor with 153600 values, but the requested shape requires a multiple of 25088\n\t [[{{node sequential/flatten/Reshape}}]]' }

Michał Słapek Over a year ago

As mentioned in answer, the second alternative does not perform resize - therefore it assumes, that the image is already (224, 224).

Michał Słapek Over a year ago

For clarity I've decided to remove the alternative solution (which does not support resizing, using jpeg-js). It is available in answer edit history.

Michał Słapek Over a year ago

I've fixed path passing (remove unused variable myImg). Please, retry this answer. Did you used output from import request ... Python script or directly from TensorFlow?

|

Collectives™ on Stack Overflow

How to call tensorflow_model_server from Nodejs, with an image

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related