I'm using the-sett/elm-aws-core to get information from the AWS API, which unfortunately is very very inconsistent. Most of the endpoints return JSON and that works fine with that lib, which takes a JSON Decoder to make a request, but the EC2 endpoint returns XML (because why not).
The lib doesn't have any options not to decode JSON as far as I can tell, which does not work at all :
let ec2 region = Service.defineRegional "ec2" "2016-11-15" Service.QUERY Service.SignV4 (Service.setXmlNamespace "https://ec2.amazonaws.com/doc/2016-11-15/") region in
let params = [("Action", "DescribeImages"), ("Version", "2016-11-15"), ("Owner.1", "self")] in
Http.request "DescribeImages" GET "/" Http.emptyBody JSONDECODERHERE |> Http.addQuery params |> Http.send (ec2 region) creds |> Task.attempt msg
Failed : Problem with the given value:
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<DescribeImagesResponse .......
As you can see in there you need to pass a JSON Decoder to Http.request, but that, of course, fails when receiving XML. Is there a way to build a "fake" JSON decoder that would actually do nothing and just pass on the raw string? I tried using Decode. string but that's still actually decoding it, which fails. If there is a way I could then run an XML decoder manually on it in my update function, which would be fine.
Thank you
It's not possible to make a "fake" decoder that does what you want, because the problem isn't with the decoding. The problem is with the parsing, which is done before decoding. Parsing is the process of converting the string into a data structure typically called an Abstract Syntax Tree (AST), but since Elm compiles to JavaScript and JSON is also a subset of JavaScript the parse result of is really just a JavaScript object. Decoding is the process of turning that untyped data structure into a properly typed data structure.
It is therefore not possible to accomplish what you want with this API. Most likely you'll need to build the http request yourself and use elm/http directly.
Related
I have never encountered this sort of collection or object before until now (its the response from a request to Google-Cloud-Vision API).
I wrote a class that uses the API and does what I want correctly. However the only way that I can extract/manipulate data in the response is by using this module:
from google.protobuf.json_format import MessageToJson
I basically serialized the protobuff into a string and then used regex to get the data that I want.
There MUST be a better way than this. Any suggestions? I was hoping to have the API response give me a json dict or json dict of dicts etc... All I could come up with was turning the response into a string though.
Here is the file from the github repository:
image_analyzer.py
Thank you all in advance.
The built in json module will parse the string into a dictionary, like json.loads(MessageToJson(response1)).
You can just access the fields in the message object directly, e.g.:
response1 = vision_client.face_detection(image=image)
print(response1)
print(response1.face_annotations[0].detection_confidence)
I am writing a REST Client for one of the Vendor REST Service. I use jersey 2.x and JSON-P, below are dependencies I add.
<dependencies>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-processing</artifactId>
<version>2.26</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>2.26</version>
</dependency>
</dependencies>
I successfully write code for GET request and received JSON output. I saved it to a file and used JSON-P to interpret and do my own logic without any issues.
But now I need to write a POST request. When I use CURL as below I am able do it. and want to implement the same using Jeresey 2.x and JSON-P.
curl -v -H "Accept:application/json" -H "Content-Type:application/json" -u user:password -X POST --databinary #./InputParameters.json https://<IP>:<PORT>/Configuration/server
InputParameters.json contents
{
"ip": "10.197.70.16",
"partNumber": 202067,
"model": "IBM P7"
}
When I tried to pass response body as String in JSON format ({"ip": "10.197.70.16", "partNumber": 202067, "model": "IBM P7"}), but didn't work. So tried as JsonObject as below still didn't work.
JsonObject jsonObj = Json.createObjectBuilder()
.add("ip", "10.197.70.16")
.add("partNumber", 202067)
.add("model", "IBM P7")
.build();
response = invocationBuilder.post(Entity.json(jsonObj));
I know core java, based on that experience I jumped into writing this program and got success with GET but not POST. I doubt I am doing something fundamentally wrong with POST.
Let's unpack what you're doing for a bit. First there's this part:
JsonObject jsonObj = Json.createObjectBuilder()
.add("ip", "10.197.70.16")
.add("partNumber", 202067)
.add("model", "IBM P7")
.build();
This creates a javax.json.JsonObject instance. JsonObject, which is part of the JSON-P Java API, is pretty much what it says: a class to represent a JSON object. A JsonObject instance contains a hierarchy of javax.json.JsonValue instances, which conform to more specific types like JsonArray, JsonString, other JsonObjects and so on. In this regard it's not unlike the classes of the DOM API for representing XML documents (let's hope Oracle keeps the API docs at that URL for a while). But JSON is fortunately a lot more straightforward than XML and DOM.
Your jsonObj instance would contain a JsonString instance with value "10.197.70.16" mapped to name "ip", a JsonNumber with value 202067 (probably represented as BigDecimal) mapped to name "partNumber" and so on.
Next your code executes this:
Entity.json(jsonObj)
javax.ws.rs.client.Entity.json(something) basically states that you want to create an entity that will provide the payload for a JAX-RS client invocation with as Content-Type application/json. In other words, the something you create it for must be transformed to a JSON representation when it's sent to the API, which should expect a JSON payload and know how to handle it. Note that Entity.json(...) has a generic type parameter. The method signature is static <T> Entity<t> json(T entity). So you're creating an instance of Entity<JsonObject> with the payload entity jsonObj.
When this is handed over to the post method of a javax.ws.rs.client.Invocation.Builder instance (the post method is actually defined in its parent interface SyncInvoker) the client implementation goes to work.
response = invocationBuilder.post(Entity.json(jsonObj));
It takes the provided Entity instance and its content (our jsonObj), checks what the desired output is of the Entity (this is application/json) and seeks a provider that can turn objects of the given type into that output. In other words, some component must be located that can be given a javax.json.JsonObject and write a representation of it as JSON to an OutputStream. The component handling this could be a javax.ws.rs.ext.MessageBodyWriter that claims it can perform this transformation and was registered to the JAX-RS runtime. Various libraries supply such providers and you can also write your own. This makes JAX-RS extensible to deal with various scenarios, handle non-standard input and output or lets you tune its behaviour. When multiple providers are capable of handling the given entity type and producing the desired output, there are rules to determine which one takes on the job. Note that this can depend on what is on your classpath. There are ways of forcing this explicitly.
The client puts together the invocation through its configuration, using the proper URL, query parameters, HTTP method, headers and so on. The payload is created by writing the entity to an OutputStream in the required format. In your example this results in a POST to the server. When the invocation has been completed you receive a javax.ws.rs.core.Response that you can use to determine the HTTP result code and retrieve a response payload, if any. The readEntity(Class<T> entityType) method of Response works like the reverse of turning an Entity into a payload. It searches for a MessageBodyReader that can interpret the response stream according to the value returned from response.getMediaType() and can create an instance of Class entityType from it.
So with all of that explained, what exactly is going wrong in your approach? Well, the issue is that the default implementations available to your JAX-RS runtime probably don't have a writer specifically for an input of type JsonObject and with expected output application/json. It may seem very logical if the server expects JSON, that you should be able to supply a JsonObject as payload. But if the JAX-RS implementation can't find something to handle that class, then at best it can just use some default approach. In that case it may try to interpret the object as a POJO and serialize it to JSON in a default manner, which could lead to weirdness like this:
{
"valueMap": {
"ip": {
"value": "10.197.70.16"
},
"partNumber": {
"num": 202067,
"integral": TRUE
},
...
}
}
That's what a literal interpretation of the JsonObject instance could look like, depending on which implementation it uses and what is used by JAX-RS to turn it into JSON output. Of course it's possible that the object can't be serialized to JSON at all, either because no suitable MessageBodyWriter can be found or it runs into an error when creating the output.
A first solution would be a very simple one. Just turn the JsonObject into a String and simply provide that as the entity:
StringWriter stringWriter = new StringWriter();
try (JsonWriter jsonWriter = Json.createWriter(stringWriter);) {
jsonWriter.writeObject(jsonObject);
} // some error handling would be needed here
String jsonPayload = stringWriter.toString();
response = invocationBuilder.post(Entity.json(jsonPayload));
It seems you had already tried that. A possible problem with this is that the MessageBodyWriter that gets used needs to just output the String's bytes in a suitable encoding (probably UTF-8) when presented with a String as output and application/json as the required content type. Some may not do that. You could try Entity.entity(jsonPayload, MediaType.TEXT_PLAIN_TYPE.withCharset("UTF-8")) but then the server might reject the call.
Other possible solutions are
Writing your own MessageBodyWriter for String objects with an #javax.ws.rs.Produces(MediaType.APPLICATION_JSON) annotation on it.
Better yet, writing such a class that accepts JsonObject instances.
Creating POJO classes for your JSON structure and letting those get used for generating JSON from instances or deserializing JSON responses to instances.
Finding an extension library that contains suitable providers for dealing with javax.json classes.
The addition of the com.owlike:genson dependency to your project is exactly the application of that last suggestion. The Genson library provides conversions between Java objects and JSON in both directions, as well as data binding. Part of its code base is dedicated to JAX-RS providers, among which a class suitable for accepting JsonObject as input.
Issue is resolved after adding below dependency. At this point I am not sure on what does it do.
Thanks to Swamy (TCS) for his support to resolve this.
<dependency>
<groupId>com.owlike</groupId>
<artifactId>genson</artifactId>
<version>1.4</version>
</dependency>
Example using genson
String serialize = new Genson().serialize(
Json.createObjectBuilder()
.add("ip", "10.197.70.16")
.add("partNumber", 202067)
.add("model", "IBM P7")
.build()
);
response = invocationBuilder.post(Entity.json(serialize));
I would like to release a cocoa touch framework including some json files which contain logic I don't want the framwork's user to see; unfortunately inside the .framework file there is still the json visible.
I thought about including it in a swift file:
struct JsonProvider {
static let json = "..."
}
but my json is that large that the file isn't usable any more. I didn't find a solution to command line compile and then include it.
Is there a solution to one of the two problems, i.e.
hide the json inside the framework or
precompile a swift file and then add it to the framework?
The answer to this really depends on how secure you need the file to be. Nothing is going to be 100% secure but you can make it more or less difficult for an attacker to gain access.
Option A: Hide the file
Add a '.' at the beginning of the file name, which will hide it from those browsing the directory that don't know about hidden files.
Caveats:
Anyone with knowledge of hidden files can figure this out.
Option B: Obfuscate
Encode your file, using Base64 or other encoding methods.
Caveats:
Only deters the lazy/mildly curious. Encodes are easy to defeat.
Option C: Encryption or storing in code
Encrypt the file using a symmetrical algorithm such as AES and store the cipher in code.
Alternatively, remove the json file and create a variable in code with a string that holds the json.
var myJson = """
{
"jsonData": "value"
}
"""
Caveats:
Code can be decompiled to reveal hardcoded strings, but it's difficult. Someone would have to gain access your .ipa file which is protected by Apple's DRM. You could also opt to encode the string, but if someone is already decompiling your code then they can figure out how to defeat obfuscation.
Option D: Don't include the file at all
This is a pretty broad topic outside the scope of your question, but essentially you host your file somewhere. Where and how you do this again depends on how secure you need the data to be. Ideally serving the data over HTTPS and blocking self-signed certificates from being used in your app so that it can't be proxied (ie man in the middle).
URLSession already does a pretty good job of this out of the box, but you could take it even further by using certificate pinning: https://developer.apple.com/news/?id=g9ejcf8y
Essentially you create certificate configurations on your server and bundle the public keys in your app, the connection will be refused unless the pinning requirements are met. Caveat is that you have to update your app whenever your certificates change.
There are many ways:
You can encrypt that file end decrypt again before using.
Change json file type to picture or something else to make sure no one know this is JSON file.
You can add json file content to sources as base64 encoded string.
struct JSONFiles {
internal static let fileOneBase64String = "iVBORw0KGgoAAAANSUhEUgAAAMEAAADLCAYAAADX..."
static var fileOneJSON: [String: Any] {
let data = Data(base64Encoded: fileOneBase64String)!
return try! JSONSerialization.jsonObject(with: data, options: []) as! [String: Any]
}
}
You can use it later anywhere in you framework by
print(JSONFiles.fileOneJSON)
Hope it help you
UPDATE:
Why as base64 encoded string?
The main benefit base64 encoded string - you don't need to escape special symbols contained in JSON string. Don't need to remove new line symbols/intendation symbols from json string before add to swift source file.
See example
let string: String = "{ \"name\":\"John\", \"email\": \"hello#stackoverflow\" \"age\":30, \"car\":null }"
But if json is more complex? If it contains 10 nested levels, arrays, dictionaries?
You can cover by UnitTests that value contained in fileOneBase64String can be decoded to JSON object
import XCTest
#testable import MyAwesomeFramework
class FrameworkTests: XCTestCase {
func testThatBase64EncodedJSONStringCanBeDecoded() {
let data = Data(base64Encoded: JSONFiles.fileOneBase64String)
XCTAssertNotNil(data, "Unable to convert base64 string to Data")
do {
_ = try JSONSerialization.jsonObject(with: data, options: [])
} catch {
XCTFail("Expected decoded JSON ")
}
}
}
I have a situation some methods use one link to request *http.Request. In it body it contains json. And I need to get different data from that json. But I don't want to decode json every time in every method again and again. Can I decode once and attach the link of struct or json (in string) to request. I try to find the solution but I didn't found anything. I know that I can create my own functional for this, but I think that such packages like "*http.Request" or "github.com/gorilla/mux" should have such functionality from the box. Anyone known?
I have an inbound payload in JSON format. I'm converting it using the "JSON to Object" converter, and then passing on the data to a component (as a JsonData object.) My component then returns the same JsonData object with modifications. I'm trying to use the Amazon S3 component as the next step in my flow, and trying to tie the bucket name and other values to elements accessible in the JsonData object.
Here is the expression for the bucket name for instance:
#[json: TopKey/BucketName]
From experience this has worked with JSON.
However when I run this, here is what I get:
Message : Failed to invoke getObjectContent. Message payload is of type: JsonData
Code : MULE_ERROR-29999
Failed to invoke getObjectContent. Message payload is of type: JsonData (org.mule.api.MessagingException)
org.mule.module.s3.processors.GetObjectContentMessageProcessor:177 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html)
Is there a way I can use my JsonData object and pull information from it, or do I have to convert it back to something else before passing it on to the Amazon S3 component?
Thanks,
After trying a little more to play with my expression, I figured out I can just access elements the way I do it in my Java component already:
#[payload.get("TopKey").get("BucketName").getTextValue()]
and I have my BucketName!
Remove the empty space from your expression: #[json:TopKey/BucketName]
You can set the "Return Class" to java.util.Map in the "JSON to Object" processor, you can then access the value via #[payload.TopKey.BucketName]