Java : Proto parser library

Question

I want to parse a proto file. Wanted to check is there any java library available which can parse proto files. Based on my requirement I cannot use descriptor parseFrom method or protoc command. Please suggest thanks in advance.

$ protoc --include_imports --descriptor_set_out temp *.proto // I don't want to do this manual step 
or 
DescriptorProtos.FileDescriptorProto descriptorProto = DescriptorProtos.FileDescriptorProto.parseFrom(proto.getBytes());

Appreciate suggestion thanks

Could you clarify why you don't want to run the tool that is designed to give you exactly what you're looking for? (I can think of various potential reasons, but your specific reason would affect possible solutions.) — Jon Skeet
– Jon Skeet, Commented Jan 26, 2022 at 15:28
@Jon Skeet- It is because I want to run this step in jenkins and we are not allowed to make any configuration changes there. — Maana
– Maana, Commented Jan 31, 2022 at 15:39
When you say "parse a proto file", what do you mean - parse it into what exactly? Parsers typically produce an AST representation of the input - is that what you want? — jon hanson
– jon hanson, Commented Feb 1, 2022 at 20:39
@jon-hanson So, I writing a sonarqube custome plugin in java which will scan any application(java, go etc) and look for .proto file and parse it. I want that parser to give back details about all messages, fieldTypes etc.. — Maana
– Maana, Commented Feb 1, 2022 at 23:32
@Sergey Vyacheslavovich Brunov- I am writing sonarqube custom proto parser so I should be able to scan any application java or golang and read the proto file. The thing with generated code is that it is not consistent across different languages. In java we get .class file where in go we get pb.go — Maana
– Maana, Commented Feb 1, 2022 at 23:37

Sergey Vyacheslavovich Brunov · Accepted Answer · 2022-02-02 17:17:04Z

Possible solution: `io.protostuff:protostuff-parser` library

Let's consider the 3.1.38 version of the io.protostuff:protostuff-parser library as the current version.

Maven dependency: Maven Repository: io.protostuff » protostuff-parser » 3.1.38.
protostuff-parser Maven module directory in GitHub repository: protostuff-compiler/protostuff-parser at v3.1.38 · protostuff/protostuff-compiler.
GitHub repository: protostuff/protostuff-compiler at v3.1.38.

Example program

Please, consider the below example program as a draft to get started with the library.

Input file

Let's assume the /some/directory/data/test.proto file exist with the following content:

message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
  enum ContentType {
    WEB = 1;
    IMAGES = 2;
    VIDEO = 3;
  }
  ContentType content_type = 4;
}

`pom.xml`: Dependencies

<project>
    <dependencies>
        <dependency>
            <groupId>io.protostuff</groupId>
            <artifactId>protostuff-parser</artifactId>
            <version>3.1.38</version>
        </dependency>
        <dependency>
            <groupId>com.google.inject</groupId>
            <artifactId>guice</artifactId>
            <version>5.1.0</version>
        </dependency>
    </dependencies>
</project>

Program

import com.google.inject.Guice;
import com.google.inject.Injector;
import io.protostuff.compiler.ParserModule;
import io.protostuff.compiler.model.Message;
import io.protostuff.compiler.model.Proto;
import io.protostuff.compiler.parser.Importer;
import io.protostuff.compiler.parser.LocalFileReader;
import io.protostuff.compiler.parser.ProtoContext;
import java.nio.file.Path;
import java.util.List;

public final class Program {
    public static void main(final String[] args) {
        final Injector injector = Guice.createInjector(new ParserModule());
        final Importer importer = injector.getInstance(Importer.class);
        final ProtoContext protoContext = importer.importFile(
            new LocalFileReader(Path.of("/some/directory/data")),
            "test.proto"
        );

        final Proto proto = protoContext.getProto();

        final List<Message> messages = proto.getMessages();
        System.out.println(String.format("Messages: %s", messages));

        final Message searchRequestMessage = proto.getMessage("SearchRequest");
        System.out.println(String.format("SearchRequest message: %s", searchRequestMessage));

        final List<Enum> searchRequestMessageEnums = searchRequestMessage.getEnums();
        System.out.println(String.format("SearchRequest message enums: %s", searchRequestMessageEnums));
    }
}

The program output:

Messages: [Message{name=SearchRequest, fullyQualifiedName=..SearchRequest, fields=[Field{name=query, typeName=string, tag=1, options=DynamicMessage{fields={}}}, Field{name=page_number, typeName=int32, tag=2, options=DynamicMessage{fields={}}}, Field{name=result_per_page, typeName=int32, tag=3, options=DynamicMessage{fields={}}}, Field{name=content_type, typeName=ContentType, tag=4, options=DynamicMessage{fields={}}}], enums=[Enum{name=ContentType, fullyQualifiedName=..SearchRequest.ContentType, constants=[EnumConstant{name=WEB, value=1, options=DynamicMessage{fields={}}}, EnumConstant{name=IMAGES, value=2, options=DynamicMessage{fields={}}}, EnumConstant{name=VIDEO, value=3, options=DynamicMessage{fields={}}}], options=DynamicMessage{fields={}}}], options=DynamicMessage{fields={}}}]
SearchRequest message: Message{name=SearchRequest, fullyQualifiedName=..SearchRequest, fields=[Field{name=query, typeName=string, tag=1, options=DynamicMessage{fields={}}}, Field{name=page_number, typeName=int32, tag=2, options=DynamicMessage{fields={}}}, Field{name=result_per_page, typeName=int32, tag=3, options=DynamicMessage{fields={}}}, Field{name=content_type, typeName=ContentType, tag=4, options=DynamicMessage{fields={}}}], enums=[Enum{name=ContentType, fullyQualifiedName=..SearchRequest.ContentType, constants=[EnumConstant{name=WEB, value=1, options=DynamicMessage{fields={}}}, EnumConstant{name=IMAGES, value=2, options=DynamicMessage{fields={}}}, EnumConstant{name=VIDEO, value=3, options=DynamicMessage{fields={}}}], options=DynamicMessage{fields={}}}], options=DynamicMessage{fields={}}}
SearchRequest message enums: [Enum{name=ContentType, fullyQualifiedName=..SearchRequest.ContentType, constants=[EnumConstant{name=WEB, value=1, options=DynamicMessage{fields={}}}, EnumConstant{name=IMAGES, value=2, options=DynamicMessage{fields={}}}, EnumConstant{name=VIDEO, value=3, options=DynamicMessage{fields={}}}], options=DynamicMessage{fields={}}}]

Thank you Sergey Vyacheslavovich Brunov this is what I was looking for :)
Sergey Vyacheslavovich Brunov : Any idea on how to resolve import. If import is not found I am getting error like - Can not load proto: google.protobuf.descriptor.proto not found.
@Maana, that is another question. Please, consider asking it as a separate question (not a comment) with the corresponding minimal reproducible example.

Collectives™ on Stack Overflow

Java : Proto parser library

1 Answer 1

Possible solution: `io.protostuff:protostuff-parser` library

Example program

Input file

`pom.xml`: Dependencies

Program

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Possible solution: io.protostuff:protostuff-parser library

Example program

Input file

pom.xml: Dependencies

Program

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related

Possible solution: `io.protostuff:protostuff-parser` library

`pom.xml`: Dependencies