Protocol Puffers

A little, highly templated, and protobuf-compatible serialization/deserialization library written in C++20

📕 Documentation

Requirements

• a compiler and a standard library implementation with C++20 support
  • GCC 11 or above, or
  • Clang 12 or above, or
  • MSVC 14.29 (Visual Studio 2019 Version 16.9) or above
• CMake 3
• GoogleTest (optional, for unit tests)
• vcpkg (optional, vcpkg install protopuf to install)

Features

• Data structures are described using type in C++ instead of DSLs like the Protocol Buffer Language (.proto)
• Fully compatible with encoding of the Protocol Buffers, capable of mutual serialization/deserialization
• Extensive compile-time operations aimed to improving run-time performance

An Example

For the following data structure described using .proto:

message Student {
    uint32 id = 1;
    string name = 3;
}

message Class {
    string name = 8;
    repeated Student students = 3;
}

We can use protopuf to describe it as C++ types:

using namespace pp;

using Student = message<
    uint32_field<"id", 1>, 
    string_field<"name", 3>
>;

using Class = message<
    string_field<"name", 8>, 
    message_field<"students", 3, Student, repeated>
>;

Subsequently, both serialization and deserialization become so easy to do:

// serialization
Student twice {123, "twice"}, tom{456, "tom"}, jerry{123456, "jerry"};
Class myClass {"class 101", {tom, jerry}};
myClass["students"_f].push_back(twice);

array<byte, 64> buffer{};
auto bufferEnd = message_coder<Class>::encode(myClass, buffer);
assert(begin_diff(bufferEnd, buffer) == 45);

// deserialization
auto [yourClass, bufferEnd2] = message_coder<Class>::decode(buffer);
assert(yourClass["name"_f] == "class 101");
assert(yourClass["students"_f][2]["name"_f] == "twice");
assert(yourClass["students"_f][2]["id"_f] == 123);
assert(yourClass["students"_f][1] == (Student{123456, "jerry"}));
assert(yourClass == myClass);
assert(begin_diff(bufferEnd2, bufferEnd) == 0);

More examples can be found in our test cases (test/message.cpp).

Supported Field Types

Category	Supported Types
Varint	int32, int64, uint32, uint64, sint32, sint64, bool, enum
64-bit	fixed64, sfixed64, double
Length-delimited	string, bytes, embedded messages, packed repeated fields
32-bit	fixed32, sfixed32, float

Worklist

• named field via NTTP (#6)
• dynamic reflection on field names and numbers (#9)
• compatibility test between protopuf and protobuf (#10)
• benchmark test between protopuf and protobuf (#11)

Known issues

• There is a known bug related to template parameter lists of lambda expressions in Visual Studio 2019 Version 16.8, which can produce a wrong compilation error while compiling protopuf
• Although class type in NTTP (P0732R2) is implemented in GCC 10, there is a CTAD bug (PR96331, exists until GCC 10.2) to reject valid NTTP usage, which prevent protopuf to compile successfully

Special Thanks

to JetBrains for its Open Source License of All Products Pack