I´m building an GUI with Javafx and i was able to create a DataOutput to xml.
But now, i like to parse everything to JSON.
To save data to a File I used a Marshaller:
try {
JAXBContext context = JAXBContext
.newInstance(FirmenListWrapper.class);
Marshaller m = context.createMarshaller();
m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
FirmenListWrapper wrapper = new FirmenListWrapper();
wrapper.setPersons(firmenData);
m.setProperty(MarshallerProperties.MEDIA_TYPE, "/json");
m.marshal(wrapper, file);
setFirmaFilePath(file);
} catch (Exception e) {
but it´s not working like this. Any Idea why?
my FirmenWrapper looks like this:
#XmlRootElement(name = "firmen")
public class FirmenListWrapper {
private List<Firmen> firmen;
#XmlElement(name = "firmen")
public List<Firmen> getFirmen() {
return firmen;
}
public void setFirmen(List<Firmen> firmen) {
this.firmen = firmen;
}
}
Thanks!
Related
I'm exploring reactive programming with Spring Webflux and therefore, I'm trying to make my code completely nonblocking to get all the benefits of a reactive application.
Currently my code for the method to parse a Json String to a JsonNode to get specific values (in this case the elementId) looks like this:
public Mono<String> readElementIdFromJsonString(String jsonString){
final JsonNode jsonNode;
try {
jsonNode = MAPPER.readTree(jsonString);
} catch (IOException e) {
return Mono.error(e);
}
final String elementId = jsonNode.get("elementId").asText();
return Mono.just(elementId);
}
However, IntelliJ notifies me that I'm using an inappropriate blocking method call with this code:
MAPPER.readTree(jsonString);
How can I implement this code in a nonblocking way? I have seen that since Jackson 2.9+, it is possible to parse a Json String in a nonblocking async way, but I don't know how to use that API and I couldn't find an example how to do it correctly.
I am not sure why it is saying it is a blocking call since Jackson is non blocking as far as I know. Anyway one way to resolve this issue is to use schedulers if you do not want to use any other library. Like this.
public Mono<String> readElementIdFromJsonString(String input) {
return Mono.just(Mapper.readTree(input))
.map(it -> it.get("elementId").asText())
.onErrorResume( it -> Mono.error(it))
.subscribeOn(Schedulers.boundedElastic());
}
Something along that line.
import reactor.core.publisher.Mono;
import java.nio.charset.StandardCharsets;
import org.springframework.core.ResolvableType;
import org.springframework.core.io.buffer.DataBufferUtils;
import org.springframework.core.io.buffer.DefaultDataBuffer;
import org.springframework.core.io.buffer.DefaultDataBufferFactory;
import org.springframework.http.codec.json.AbstractJackson2Decoder;
import org.springframework.util.MimeType;
import org.springframework.util.MimeTypeUtils;
import com.fasterxml.jackson.databind.ObjectMapper;
#FunctionalInterface
public interface MessageParser<T> {
Mono<T> parse(String message);
}
public class JsonNodeParser extends AbstractJackson2Decoder implements MessageParser<JsonNode> {
private static final MimeType MIME_TYPE = MimeTypeUtils.APPLICATION_JSON;
private static final ObjectMapper OBJECT_MAPPER = allocateDefaultObjectMapper();
private final DefaultDataBufferFactory factory;
private final ResolvableType resolvableType;
public JsonNodeParser(final Environment env) {
super(OBJECT_MAPPER, MIME_TYPE);
this.factory = new DefaultDataBufferFactory();
this.resolvableType = ResolvableType.forClass(JsonNode.class);
this.setMaxInMemorySize(100000); // 1MB
canDecodeJsonNode();
}
#Override
public Mono<JsonNode> parse(final String message) {
final byte[] bytes = message.getBytes(StandardCharsets.UTF_8);
return decode(bytes);
}
private Mono<JsonNode> decode(final byte[] bytes) {
final DefaultDataBuffer defaultDataBuffer = this.factory.wrap(bytes);
return this.decodeToMono(Mono.just(defaultDataBuffer), this.resolvableType, MIME_TYPE, Map.of())
.ofType(JsonNode.class)
.subscribeOn(Schedulers.boundedElastic())
.doFinally((t) -> DataBufferUtils.release(defaultDataBuffer));
}
private void canDecodeJsonNode() {
if (!canDecode(this.resolvableType, MIME_TYPE)) {
throw new IllegalStateException(String.format("JsonNodeParser doesn't supports the given tar`enter code here`get " +
"element type [%s] and the MIME type [%s]", this.resolvableType, MIME_TYPE));
}
}
}
I'm currently working on a project where I need to fetch a large amount of data from the Database and parse it into a specific Json format, I already have built my custom Serializers and Its working properly when i pass a List to Gson. But as I was already working with Streams from my JPA Layer, I thought I could pass the Stream down to the Gson parser so that it could transform it directly to my Json data. But I'm getting an empty Json object instead of a correctly populated one.
So, if anyone could point to me a way to make Gson work with Java 8 Streams or if this isn't possible currently.. i could not find anything on Google, so i came to Stackoverflow.
You could use JsonWriter to streaming your data to output stream:
public void writeJsonStream(OutputStream out, Stream<DataObject> data) throws IOException {
try(JsonWriter writer = new JsonWriter(new OutputStreamWriter(out, "UTF-8"))) {
writer.setIndent(" ");
writer.beginArray();
data.forEach(d -> {
d.beginObject();
d.name("yourField").value(d.getYourField());
....
d.endObject();
});
writer.endArray();
}
}
Note that you're in charge of controling the json structure.
That is, if your DataObject contains nested Object, you have to write beginObject()/endObject() respectively. The same goes for nested array.
It is not as trivial as one would expect, but it can be done in a generic way.
When you look into the Javadoc to TypeAdapterFactory, they provide a very simplistic way of writing a TypeAdapterFactory for a custom type. Alas, it does not work as expected because of problems with element type detection. The proper way to do this can be found in Gson-internal CollectionTypeAdapterFactory. It is quite complex, but taking what's necessary one can come up with something like that:
final class StreamTypeAdapterFactory implements TypeAdapterFactory {
#SuppressWarnings("unchecked")
#Override
public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> typeToken) {
Type type = typeToken.getType();
Class<? super T> rawType = typeToken.getRawType();
if (!Stream.class.isAssignableFrom(rawType)) {
return null;
}
Type elementType = ExtraGsonTypes.getStreamElementType(type, rawType);
TypeAdapter<?> elementAdapter = gson.getAdapter(TypeToken.get(elementType));
return (TypeAdapter<T>) new StreamTypeAdapter<>(elementAdapter);
}
private static class StreamTypeAdapter<E> extends TypeAdapter<Stream<E>> {
private final TypeAdapter<E> elementAdapter;
StreamTypeAdapter(TypeAdapter<E> elementAdapter) {
this.elementAdapter = elementAdapter;
}
public void write(JsonWriter out, Stream<E> value) throws IOException {
out.beginArray();
for (E element : iterable(value)) {
elementAdapter.write(out, element);
}
out.endArray();
}
public Stream<E> read(JsonReader in) throws IOException {
Stream.Builder<E> builder = Stream.builder();
in.beginArray();
while (in.hasNext()) {
builder.add(elementAdapter.read(in));
}
in.endArray();
return builder.build();
}
}
private static <T> Iterable<T> iterable(Stream<T> stream) {
return stream::iterator;
}
}
The ExtraGsonTypes is a special class that I used to circumvent package-private access to $Gson$Types.getSupertype method. It's a hack that works if you're not using JDK 9's modules - you simply place this class in the same package as $Gson$Types:
package com.google.gson.internal;
import java.lang.reflect.*;
import java.util.stream.Stream;
public final class ExtraGsonTypes {
public static Type getStreamElementType(Type context, Class<?> contextRawType) {
return getContainerElementType(context, contextRawType, Stream.class);
}
private static Type getContainerElementType(Type context, Class<?> contextRawType, Class<?> containerSupertype) {
Type containerType = $Gson$Types.getSupertype(context, contextRawType, containerSupertype);
if (containerType instanceof WildcardType) {
containerType = ((WildcardType)containerType).getUpperBounds()[0];
}
if (containerType instanceof ParameterizedType) {
return ((ParameterizedType) containerType).getActualTypeArguments()[0];
}
return Object.class;
}
}
(I filed an issue about that in GitHub)
You use it in the following way:
Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new StreamTypeAdapterFactory())
.create();
System.out.println(gson.toJson(Stream.of(1, 2, 3)));
I am currently building an API gateway for a new microservices system, using the Spring Netflix Zuul library.
So far my gateway contains PRE and POST filters that intercept the requests and perform the required logic, etc.
One thing that I see is that REST calls to specific microservices require invoking an API endpoint (either GET or POST) containing JSON payload data that is very complex.
For an end-user sending a request to a microservice containing this JSON would not be user friendly.
I had an idea such that the API gateway act as a mediator, where the user can submit a more "simplified/ user-friendly" JSON to the API gateway, which will transform the JSON payload with the correct "complex" JSON structure that the target microservice can understand in order to handle the request efficiently.
My understanding of how Netflix Zuul is that this can be done by creating a RouteFilter and then including this logic here.
Can anyone explain if (or how) this transformation could be done using Netflix Zuul?
Any advice is appreciated.
Thanks.
No doubt you can do it with Zuul, i am currently trying to do almost the same. i'd suggest you take a look at this repo :
sample-zuul-filters
and the official doc on github.
Filters have to extend ZuulFilter and implement the following methods :
/**
*return a string defining when your filter must execute during zuul's
*lyfecyle ('pre'/'post' routing
**/
#Override
public String filterType(){
return 'pre'; // run this filter before sending the final request
}
/**
* return an int describing the order that the filter should run on,
* (relative to the other filters and the current 'pre' or 'post' context)
**/
#Override
public int filterOrder {
return 1; //this filter runs first in a pre-request context
}
/**
* return a boolean indicating if the filter should run or not
**/
#Override
public boolean shouldFilter() {
RequestContext ctx = RequestContext.getCurrentContext();
if(ctx.getRequest().getRequestURI().equals("/theRouteIWantToFilter"))
{
return true;
}
else {
return false;
}
}
/**
* After all the config stuffs you can set what your filter actually does
* here. This is where your json logic goes.
*/
#Override
public Object run() {
try {
RequestContext ctx = RequestContext.getCurrentContext();
HttpServletRequest request = ctx.getRequest();
InputStream stream = ctx.getResponseDataStream();
String body = StreamUtils.copyToString(stream, Charset.forName("UTF-8"));
// transform your json and send it to the api.
ctx.setResponseBody(" Modified body : " + body);
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
I am not sure my answer is 100% accurate since i am working on it but it's a start.
I've done payload conversion in pre filter but this should work in route filter as well. Use com.netflix.zuul.http.HttpServletRequestWrapper to capture and modify the original request payload before forwarding the request to target microservice.
Sample code:
package com.sample.zuul.filters.pre;
import com.google.common.io.CharStreams;
import com.netflix.zuul.ZuulFilter;
import com.netflix.zuul.context.RequestContext;
import com.netflix.zuul.http.HttpServletRequestWrapper;
import com.netflix.zuul.http.ServletInputStreamWrapper;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.json.simple.parser.ParseException;
import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
public class JsonConverterFilter extends ZuulFilter {
#Override
public String filterType() {
return "pre";
}
#Override
public int filterOrder() {
return 0; // Set it to whatever the order of your filter is
}
#Override
public boolean shouldFilter() {
return true;
}
#Override
public Object run() {
RequestContext context = RequestContext.getCurrentContext();
HttpServletRequest request = new HttpServletRequestWrapper(context.getRequest());
String requestData = null;
JSONParser jsonParser = new JSONParser();
JSONObject requestJson = null;
try {
if (request.getContentLength() > 0) {
requestData = CharStreams.toString(request.getReader());
}
if (requestData == null) {
return null;
}
requestJson = (JSONObject) jsonParser.parse(requestData);
} catch (Exception e) {
//Add your exception handling code here
}
JSONObject modifiedRequest = modifyJSONRequest(requestJson);
final byte[] newRequestDataBytes = modifiedRequest.toJSONString().getBytes();
request = getUpdatedHttpServletRequest(request, newRequestDataBytes);
context.setRequest(request);
return null;
}
private JSONObject modifyJSONRequest(JSONObject requestJSON) {
JSONObject jsonObjectDecryptedPayload = null;
try {
jsonObjectDecryptedPayload = (JSONObject) new JSONParser()
.parse("Your new complex json");
} catch (ParseException e) {
e.printStackTrace();
}
return jsonObjectDecryptedPayload;
}
private HttpServletRequest getUpdatedHttpServletRequest(HttpServletRequest request, final byte[] newRequestDataBytes) {
request = new javax.servlet.http.HttpServletRequestWrapper(request) {
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(
new InputStreamReader(new ByteArrayInputStream(newRequestDataBytes)));
}
#Override
public ServletInputStream getInputStream() throws IOException {
return new ServletInputStreamWrapper(newRequestDataBytes);
}
/*
* Forcing any calls to HttpServletRequest.getContentLength to return the accurate length of bytes
* from a modified request
*/
#Override
public int getContentLength() {
return newRequestDataBytes.length;
}
};
return request;
}
}
Is there a way to serialize an object so that it could then be rehydrated by .Net Core Configuration Binder?
Basically, I'd like to get this Test to pass:
[Test]
public void Can_Serialize_And_Rehydrate()
{
var foo = new Foo{ Prop1 = 42; Prop2 = "Test" }
Dictionary<string, string> serialized = Serialize(Foo);
var deserializedFoo = new Foo();
new ConfigurationBuilder()
.AddInMemoryCollection(serialized)
.Build()
.Bind(deserializedFoo);
Assert.AreEqual(deserializedFoo.Prop1, 42);
Assert.AreEqual(deserializedFoo.Prop2, "Test");
}
Is there a Serializer out-of-the-box, or am I'm going to need to write my own Serialize() method?
AddInMemoryCollection's signature is like below, so why are you trying to serialize your dictionary here? You could just use it as it is.
public static IConfigurationBuilder AddInMemoryCollection(
this IConfigurationBuilder configurationBuilder,
IEnumerable<KeyValuePair<string, string>> initialData)
If you like to know more about how to test your custom configurations, I would suggest to look here:
https://github.com/aspnet/Configuration/blob/1.0.0/test/Microsoft.Extensions.Configuration.Binder.Test/ConfigurationBinderTests.cs
I was able to get this working by "hijacking" a JsonConfigurationProvider and plugging serialized Json directly into it. Not sure if this is the best way, but it does work:
public class ConfigurationSerializer
{
private class CustomJsonProvider : JsonConfigurationProvider
{
public CustomJsonProvider() : base(new JsonConfigurationSource())
{
}
public IDictionary<string, string> GetData(Stream s)
{
Load(s);
// Return the Configuration Dictionary
return Data;
}
}
public Dictionary<string, string> Serialize(object o)
{
var serialized =
JsonConvert.SerializeObject(
o,
new JsonSerializerSettings {NullValueHandling = NullValueHandling.Ignore});
using (var ms = new MemoryStream(Encoding.UTF8.GetBytes(serialized)))
{
var jsonProvider = new CustomJsonProvider();
return jsonProvider
.GetData(ms)
.ToDictionary(key => key.Key, value => value.Value);
}
}
}
I have JAXB objects created from a schema. While marshalling, the xml elements are getting annotated with ns2. I have tried all the options that exist over the net for this problem, but none of them works. I cannot modify my schema or change package-info.java. Please help
After much research and tinkering I have finally managed to achieve a solution to this problem. Please accept my apologies for not posting links to the original references - there are many and I wasn't taking notes - but this one was certainly useful.
My solution uses a filtering XMLStreamWriter which applies an empty namespace context.
public class NoNamesWriter extends DelegatingXMLStreamWriter {
private static final NamespaceContext emptyNamespaceContext = new NamespaceContext() {
#Override
public String getNamespaceURI(String prefix) {
return "";
}
#Override
public String getPrefix(String namespaceURI) {
return "";
}
#Override
public Iterator getPrefixes(String namespaceURI) {
return null;
}
};
public static XMLStreamWriter filter(Writer writer) throws XMLStreamException {
return new NoNamesWriter(XMLOutputFactory.newInstance().createXMLStreamWriter(writer));
}
public NoNamesWriter(XMLStreamWriter writer) {
super(writer);
}
#Override
public NamespaceContext getNamespaceContext() {
return emptyNamespaceContext;
}
}
You can find a DelegatingXMLStreamWriter here.
You can then filter the marshalling xml with:
// Filter the output to remove namespaces.
m.marshal(it, NoNamesWriter.filter(writer));
I am sure there are more efficient mechanisms but I know this one works.
For me, only changing the package-info.java class worked like a charm, exactly as zatziky stated :
package-info.java
#javax.xml.bind.annotation.XmlSchema
(namespace = "http://example.com",
xmlns = {#XmlNs(prefix = "", namespaceURI = "http://example.com")},
elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED)
package my.package;
import javax.xml.bind.annotation.XmlNs;
You can let the namespaces be written only once. You will need a proxy class of the XMLStreamWriter and a package-info.java. Then you will do in your code:
StringWriter stringWriter = new StringWriter();
XMLStreamWriter writer = new Wrapper((XMLStreamWriter) XMLOutputFactory
.newInstance().createXMLStreamWriter(stringWriter));
JAXBContext jaxbContext = JAXBContext.newInstance(Collection.class);
Marshaller jaxbMarshaller = jaxbContext.createMarshaller();
jaxbMarshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
jaxbMarshaller.marshal(books, writer);
System.out.println(stringWriter.toString());
Proxy class (the important method is "writeNamespace"):
class WrapperXMLStreamWriter implements XMLStreamWriter {
private final XMLStreamWriter writer;
public WrapperXMLStreamWriter(XMLStreamWriter writer) {
this.writer = writer;
}
//keeps track of what namespaces were used so that not to
//write them more than once
private List<String> namespaces = new ArrayList<String>();
public void init(){
namespaces.clear();
}
public void writeStartElement(String localName) throws XMLStreamException {
init();
writer.writeStartElement(localName);
}
public void writeStartElement(String namespaceURI, String localName) throws XMLStreamException {
init();
writer.writeStartElement(namespaceURI, localName);
}
public void writeStartElement(String prefix, String localName, String namespaceURI) throws XMLStreamException {
init();
writer.writeStartElement(prefix, localName, namespaceURI);
}
public void writeNamespace(String prefix, String namespaceURI) throws XMLStreamException {
if(namespaces.contains(namespaceURI)){
return;
}
namespaces.add(namespaceURI);
writer.writeNamespace(prefix, namespaceURI);
}
// .. other delegation method, always the same pattern: writer.method() ...
}
package-info.java:
#XmlSchema(elementFormDefault=XmlNsForm.QUALIFIED, attributeFormDefault=XmlNsForm.UNQUALIFIED ,
xmlns = {
#XmlNs(namespaceURI = "http://www.w3.org/2001/XMLSchema-instance", prefix = "xsi")})
package your.package;
import javax.xml.bind.annotation.XmlNs;
import javax.xml.bind.annotation.XmlNsForm;
import javax.xml.bind.annotation.XmlSchema;
You can use the NamespacePrefixMapper extension to control the namespace prefixes for your use case. The same extension is supported by both the JAXB reference implementation and EclipseLink JAXB (MOXy).
http://wiki.eclipse.org/EclipseLink/Release/2.4.0/JAXB_RI_Extensions/Namespace_Prefix_Mapper
Every solution requires complex overwriting or annotations which seems not to work with recent version. I use a simpler approach, just by replacing the annoying namespaces. I wish Google & Co would use JSON and get rid of XML.
kml.marshal(file);
String kmlContent = FileUtils.readFileToString(file, "UTF-8");
kmlContent = kmlContent.replaceAll("ns2:","").replace("<kml xmlns:ns2=\"http://www.opengis.net/kml/2.2\" xmlns:ns3=\"http://www.w3.org/2005/Atom\" xmlns:ns4=\"urn:oasis:names:tc:ciq:xsdschema:xAL:2.0\" xmlns:ns5=\"http://www.google.com/kml/ext/2.2\">", "<kml>");
FileUtils.write(file, kmlContent, "UTF-8");