Before you go down this rabbit hole I'd recommend quickly doing a POC microbenchmark, with a sample payload, against formats with schema to get an idea of why this isn't more popular:
- JSON (without schema) vs Protobuf (with schema)
- Msgpack (without schema, compressed) vs Protobuf (with schema)
- JSON + gzip or brotli encoding vs Protobuf + gzip or brotli
- JSON + gzip or brotli encoding vs Msgpack + gzip or brotli
Compression will erase a lot of the gains even with a schema-defined protocol, but if your goal is low CPU (just serialization) then msgpack is usually good enough.
- JSON (without schema) vs Protobuf (with schema)
- Msgpack (without schema, compressed) vs Protobuf (with schema)
- JSON + gzip or brotli encoding vs Protobuf + gzip or brotli
- JSON + gzip or brotli encoding vs Msgpack + gzip or brotli
Compression will erase a lot of the gains even with a schema-defined protocol, but if your goal is low CPU (just serialization) then msgpack is usually good enough.