Using a batchsize greater than 1 when using Tensorflows c++ API

Question

I have a Tensorflow model trained in Python and frozen with the freeze_graph script. I've succesfully loaded the model in c++ and made inference for a single image. However, it seems freeze_graph sets the batchsize to only a single image at a time, since I am unable to pass the model a tensor with more than one image.

Does anyone know a way of changing this? I haven't been able to locate where in the script it actually happens.

Thanks!

EDIT:

Okay, so I scrapped Keras just to eliminate any black magic that might be doing, and I set a batch size of 16 when defining the network with Tensorflow.

If I print the graph def, the placeholder has a shape:

node {
  name: "inputs"
  op: "Placeholder"
  attr {
    key: "dtype"
    value {
      type: DT_FLOAT
    }
  }
  attr {
    key: "shape"
    value {
      shape {
        dim {
          size: 16
        }
        dim {
          size: 50
        }
        dim {
          size: 50
        }
        dim {
          size: 3
        }
      }
    }
  }
}

However, when I attempt to load and run the model in c++ with a tensor of shape 16 x 50 x 50 x 3, I get this error:

tensorflow/core/framework/tensor.cc:433] Check failed: 1 == NumElements() (1 vs. 16)Must have a one element tensor

Something must be happening somewhere when I freeze the graph?

freeze_graph doesn't 'set' the batch size of the model. How were you feeding the inputs in your original model? — keveman
– keveman, Commented Jul 6, 2016 at 14:51
I've built the model using Keras, which I believe uses a variable batchsize per default. This made me assume that freeze_graph must do something in the case of a placeholder with a dimension of variable size, but I might be wrong. Is there a way I can check what the shape of the placeholder actually is? — ppries
– ppries, Commented Jul 7, 2016 at 16:24
You can try printing out the graph in text form using graph.as_graph_def() and see if the placeholder has the shape attribute set. — keveman
– keveman, Commented Jul 7, 2016 at 16:35
Thanks for the help so far -- I've updated my OP, if you have the time to take a look! — ppries
– ppries, Commented Jul 8, 2016 at 12:12
Is the graph that you show before or AFTER running freeze_graph? It will be useful to show the one after freeze_graph. — keveman
– keveman, Commented Jul 8, 2016 at 15:24

ppries · Accepted Answer · 2016-07-20 12:52:56Z

1

This turned out to be a stupid mistake on my part. When getting the output of the graph, I called .scalar<float>() on it. This worked perfectly fine when I only had one input image, and therefore only one output, but obviously I can't cast a vector to a scalar. Changing it to .flat<float>() fixed my issue.

answered Jul 20, 2016 at 12:52

ppries

1151 silver badge11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Using a batchsize greater than 1 when using Tensorflows c++ API

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related