Hey I know there is a solution for this in Java, I'm curious to know if anyone knows of a Python 3 solution for converting a JSON object or file into protobuf format. I would accept either or as converting to an object is trivial. Searching the stackoverflow site, I only found examples of protobuf->json, but not the other way around. There is one extremely old repo that may do this but it is in Python 2 and our pipeline is Python 3. Any help is as always, appreciated.
2 Answers
The library you're looking for is google.protobuf.json_format. You can install it with the directions in the README here. The library is compatible with Python >= 2.7.
Example usage:
Given a protobuf message like this:
message Thing {
string first = 1;
bool second = 2;
int32 third = 3;
}
You can go from Python dict or JSON string to protobuf like:
import json
from google.protobuf.json_format import Parse, ParseDict
d = {
"first": "a string",
"second": True,
"third": 123456789
}
message = ParseDict(d, Thing())
# or
message = Parse(json.dumps(d), Thing())
print(message.first) # "a string"
print(message.second) # True
print(message.third) # 123456789
or from protobuf to Python dict or JSON string:
from google.protobuf.json_format import MessageToDict, MessageToJson
message_as_dict = MessageToDict(message)
message_as_dict['first'] # == 'a string'
message_as_dict['second'] # == True
message_as_dict['third'] # == 123456789
# or
message_as_json_str = MessageToJson(message)
3 Comments
jimh
if you see my comment above I do suggest
google.protobuf.json_format. why would you use Parse instead of ParseDict and which is more practical for a protobuf being piped to an API? basically trying to get a protobuf from an external source like FHIR, then pipe to a RESTful API in python
kingkupps
I just used
Parse since I consume raw JSON strings more than dicts. Are you trying to go the other way? I.e. from message to JSON? In that case you can use MessageToJson or MessageToDict. Not sure if I'm understanding your question.jimh
I am trying to parse a message to JSON but I need a test protobuf stream with relevant data for us to parse so basically, JSON to message piped out of python, then piped through the API, then parsed back into JSON because we will be receiving either JSONs or Protobufs
Here is a much simpler way by using xia-easy-proto module. No need to pre-define anything.
pip install xia-easy-proto
And then
from xia_easy_proto import EasyProto
if __name__ == '__main__':
songs = {"composer": {'given_name': 'Johann', 'family_name': 'Pachelbel'},
"title": 'Canon in D',
"year": [1680, 1681]}
song_class, song_payload = EasyProto.serialize(songs)
print(song_class) # It is the message class
print(song_payload) # It is the serialized message
4 Comments
Liu Charles
This is really helpful. Looking for a long time. Could you just open source it so anyone can help improve it. It is really useful. thanks
Liu Charles
and how is the performance? will the serialize cost too much for each record than protobuf with already generated class?
Liu Charles
How do you think of MakeSimpleProtoClass() in google/protobuf/proto_builder.py; see google/protobuf/internal/proto_builder_test.py?
Soral
All documentation could be found in the related pip help pages. There is no performance issue if you could reuse the message class. I will think about open sourcing it.
google.protobuf.json_format.ParseDictseems like it may be a viable solution, but am having trouble interpreting if this message format is the format we are most likely to receive for protobuf input if it is via a RESTful API.