Dynamically creating python class from a protobuf file at run time?

Question

In Python, is it possible at run time to convert a Google Protocol Buffers .proto file into a python class that reads that data? Python is a very dynamic language. When you use protoc to convert a .proto file to python source code, the generated code makes a lot of use of python metaclasses, so it's already very dynamic.

Ideally, I'm thinking of something like this:

import whatever
module = whatever.load_from_file("myfile.proto")

Is this possible?

(I am new to protocol buffers, please let me know if my question makes no sense)

@AndriyIvaneyko Perhaps my question was unclear, I have updated it. I'm talking about Google's Protocol Buffers format — Amandasaurus
– Amandasaurus, Commented Feb 11, 2016 at 15:02
If the .proto file itself is a python source code that you want to include as a module, you are probably looking for this: import imp; module = imp.load_source('any_name', 'myfile.proto') — Berci
– Berci, Commented Feb 11, 2016 at 15:05
Anyway, just import the converted python code (perhaps use sys.path.append(its_dir) before). — Berci
– Berci, Commented Feb 11, 2016 at 15:06

Kenton Varda · Accepted Answer · 2016-02-13 19:26:58Z

16

In theory, all the pieces exist to make this work. The Python protobuf implementation could call the C++ .proto parser library (libprotoc) as a C extension to get Descriptors, and then could feed those into the metaclasses.

However, as far as I know, no one has quite tied it altogether. (Disclaimer: My knowledge is a few years old, I may have missed a new development, but I don't see anything in the docs.)

Incidentally, Cap'n Proto's Python implementation does do what you describe, proving it is possible. But that doesn't help you if you need to work with Protobuf format.

(Disclosure: I was the author of most of Google's open source Protobuf code, and I am also the author of Cap'n Proto.)

answered Feb 13, 2016 at 19:26

Kenton Varda

45.9k10 gold badges134 silver badges116 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Amandasaurus Over a year ago

Alas, I need to work with protobuf files. I'm not in a position to change that. However "No, I don't think anyone has done this" is an acceptable answer.

Roland Puntaier Over a year ago

github.com/capnproto/pycapnp

Wuzhou Zhang · Accepted Answer · 2017-08-30 18:24:41Z

2

The most elegant solution to this problem is to use init.py. See: What is __init__.py for?

You can invoke your script to generate Python protobuf classes in init.py. Upon importing, that script will be automatically invoked to generate Python protobuf classes.

answered Aug 30, 2017 at 18:24

Wuzhou Zhang

1572 silver badges3 bronze badges

Collectives™ on Stack Overflow

Dynamically creating python class from a protobuf file at run time?

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related