Spring webflux Netty: How to expose proto as json endpoints without duplication of code? - json

Question

Use-case:
Developers/I, want to only implement a Protobuf implementation (binary protocol). However, I need a way to add config, so, the same implementation is exposed as rest/json api as well -- without code duplication.
I have proto endpoints exposed. I also want consumers to post json equivalent of those proto objects and return/receive json equivalent of the results with type info (Pojo?). The type info helps with OpenAPI / Swagger documentation too!
What are the most elegant/simple ways to achieve that without code duplication?
Any example github code that achieves that would be helpful.
Note: This is for webflux & netty - no tomcat.
ProtobufJsonFormatHttpMessageConverter - works for tomcat, does not work for netty. A working example code would be great.

Answer 1

I was messing around with this and ended up with this. Nothing else worked for me.
Using protov3 and setting a protobuf like this
syntax = "proto3";
option java_package = "com.company";
option java_multiple_files = true;
message CreateThingRequest {
...
message CreateThingResponse {
....
I can scan for the protobuf files by setting app.protoPath in my application.properties
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.google.common.reflect.ClassPath;
import com.google.protobuf.Message;
import com.google.protobuf.util.JsonFormat;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.http.codec.json.Jackson2JsonDecoder;
import org.springframework.http.codec.json.Jackson2JsonEncoder;
import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder;
import org.springframework.web.reactive.config.WebFluxConfigurer;
#Configuration
public class WebConfig implements WebFluxConfigurer {
#Value("${app.protoPath:com.}")
private String protoPath;
#Override
public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
configurer.defaultCodecs().jackson2JsonEncoder(
new Jackson2JsonEncoder(Jackson2ObjectMapperBuilder.json().serializerByType(
Message.class, new JsonSerializer<Message>() {
#Override
public void serialize(Message value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
String str = JsonFormat.printer().omittingInsignificantWhitespace().print(value);
gen.writeRawValue(str);
}
}
).build())
);
final ClassLoader loader = Thread.currentThread().getContextClassLoader();
Map<Class<?>, JsonDeserializer<?>> deserializers = new HashMap<>();
try {
for (final ClassPath.ClassInfo info : ClassPath.from(loader).getTopLevelClasses()) {
if (info.getName().startsWith(protoPath)) {
final Class<?> clazz = info.load();
if (!Message.class.isAssignableFrom(clazz)) {
continue;
}
#SuppressWarnings("unchecked") final Class<Message> proto = (Class<Message>) clazz;
final JsonDeserializer<Message> deserializer = new CustomJsonDeserializer() {
#Override
public Class<Message> getDeserializeClass() {
return proto;
}
};
deserializers.put(proto, deserializer);
}
}
} catch (IOException e) {
throw new RuntimeException(e);
}
configurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(Jackson2ObjectMapperBuilder.json().deserializersByType(deserializers).build()));
}
private abstract static class CustomJsonDeserializer extends JsonDeserializer<Message> {
abstract Class<? extends Message> getDeserializeClass();
#Override
public Message deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException {
Message.Builder builder = null;
try {
builder = (Message.Builder) getDeserializeClass()
.getDeclaredMethod("newBuilder")
.invoke(null);
} catch (Exception e) {
throw new RuntimeException(e);
}
JsonFormat.parser().merge(jp.getCodec().readTree(jp).toString(), builder);
return builder.build();
}
}
}
Then I just use the object types in the returns;
#PostMapping(
path = "/things",
consumes = {MediaType.APPLICATION_JSON_VALUE, "application/x-protobuf"},
produces = {MediaType.APPLICATION_JSON_VALUE, "application/x-protobuf"})
Mono<CreateThingResponse> createThing(#RequestBody CreateThingRequest request);
With https://github.com/innogames/springfox-protobuf you can get the responses to show in swagger but the requests still aren't showing for me.
You'll have to excuse the messy Java I'm a little rusty.

Answer 2

I needed to support json and the following code helped
#Bean
public WebFluxConfigurer webFluxConfigurer() {
return new WebFluxConfigurer() {
#Override
public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
ObjectMapper mapper = new ObjectMapper()
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
.registerModule(new ProtobufModule());
configurer.customCodecs().register(new Jackson2JsonEncoder(mapper));
configurer.customCodecs().register(new Jackson2JsonDecoder(mapper));
}
};
}

Answer 3

Try adding ProtoEncoder in your WebFlux config:
#EnableWebFlux
public class MyConfig implements WebFluxConfigurer {
#Override
public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
configurer.customCodecs().register(new ProtobufEncoder());
}
}
Then in your request mapping return the proto object:
#GetMapping (produces = "application/x-protobuf")
public MyProtoObject lookup() {
return new MyProtoObject();
}
Furthermore, if you want to serialize the proto object into JSON and return String, then have a look at com.googlecode.protobuf-java-format:protobuf-java-format library and JsonFormat::printToString capability (https://code.google.com/archive/p/protobuf-java-format/):
#GetMapping
public String lookup() {
return new JsonFormat().printToString(new MyProtoObj());
}

Answer 4

Since version 4.1 spring provides org.springframework.http.converter.protobuf.ProtobufHttpMessageConverter for reading and writing protos as Json.
However, If you are using Spring 5.x and Protobuf 3.x there is org.springframework.http.converter.protobuf.ProtobufJsonFormatHttpMessageConverter for more explicit conversion of Json.
This documentation should help you:
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/http/converter/protobuf/ProtobufHttpMessageConverter.html
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/http/converter/protobuf/ProtobufJsonFormatHttpMessageConverter.html