Caffe Python API reference?

Question

My main gripe about Caffe is that although there are a few examples in the documentation, there is no definitive reference guide. This applies especially to the Python interface (which would be great if there was a reference guide), and also to prototxt. It seems that in order to use Caffe properly, the user must already be an expert in Google Protobuf and CUDA. Shame to admit I have no experience in either of those things.

So how do I look up things (like function signature, supported values for various variables, and so on) in the Python interface? From this example we can see that we can create a data layer in Python as follows.

n.data, n.label = L.Data(batch_size=batch_size, backend=P.Data.LMDB, source=lmdb,
                         transform_param=dict(scale=1./255), ntop=2)

This is very nice, because afterwards we can export the network we made in Python to prototxt. The problem however, is that I have no idea how to make a slightly different data layer (with different parameters or different backends for example), and there seems to be nowhere to look these things up. How, for example, do I construct the following layer in prototxt using PyCaffe code?

layer {
  name: "image"
  type: "HDF5Data"
  top: "image"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "./training_data_paths.txt"
    batch_size: 64
  }
}

Where do I look up what (Python) functions to call and what arguments they take and what are the valid values for the arguments? I've asked essentially the same question on Caffe Users' Group but no one answered.

Shai · Accepted Answer · 2016-05-31 09:02:19Z

3

The protobuffer definition of caffe can be found at $CAFFE_ROOT/src/caffe/proto/caffe.proto there you can see the different parameters and their accepted value.

To create "HDF5Data" layer instead of a "Data" layer using caffe.NetSpec() interface, you can

n.image = L.HDF5Data(hdf5_data_param={'source': './training_data_paths.txt',  
                                      'batch_size': 64},
                     include={'phase': caffe.TRAIN})

edited May 31, 2016 at 9:02

answered May 29, 2016 at 6:29

Shai

115k39 gold badges259 silver badges398 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Ray Over a year ago

I guess this is the closest to an API reference as we can get. It still doesn't answer a lot of questions, for example, in Python we use the class HDF5Data as if it's a subclass of Data, but where is that apparent in the caffe.proto file? In practice, HDF5Data is not subclassing Data, but the interface is as though it is, due to the way parameters are read from and to the proto files. But I only found that out through arduous deciphering of the source code, and life would be so much easier if there was an actual API reference.

Shai Over a year ago

@Ray I suppose that from python interface point of view, HDF5Data is not a subclass of Data...

John Over a year ago

@Shai: Good solition. However, I found that you may be missing a parameter ntop=2. Is it right? In addition, If I want to make train and val prototxt in a single prototxt. Could I use the code

n.data, n.label = L.HDF5Data(batch_size=batch_size, source=source_path, ntop=2, include={'phase': caffe.TRAIN})         n.data, n.label = L.HDF5Data(batch_size=batch_size, source=source_path,ntop=2, include={'phase': caffe.TEST})

? Do I need change the variable ndata,nlabel in testing and training? Thanks

Shai Over a year ago

@user8264 if you want both data and label outputs then you need ntop=2. as for train and test phase: there's no simple solution

John Over a year ago

Thanks. It means we must be write it in individual file. This is my reference question stackoverflow.com/questions/42289880/…

Collectives™ on Stack Overflow

Caffe Python API reference?

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related