Convert string representation of an array of json objects to map Elixir - json

Here I have a payload coming to my controller action endpoint:
%{
"mandrill_events" => "[{\"event\":\"send\",\"msg\":{\"ts\":1365109999,\"subject\":\"This an example webhook message\",\"email\":\"example.webhook#mandrillapp.com\",\"sender\":\"example.sender#mandrillapp.com\",\"tags\":[\"webhook-example\"],\"opens\":[],\"clicks\":[],\"state\":\"sent\",\"metadata\":{\"user_id\":111},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa\",\"_version\":\"exampleaaaaaaaaaaaaaaa\"},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa\",\"ts\":1518203456},{\"event\":\"send\",\"msg\":{\"ts\":1365109999,\"subject\":\"This an example webhook message\",\"email\":\"example.webhook#mandrillapp.com\",\"sender\":\"example.sender#mandrillapp.com\",\"tags\":[\"webhook-example\"],\"opens\":[],\"clicks\":[],\"state\":\"sent\",\"metadata\":{\"user_id\":111},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa1\",\"_version\":\"exampleaaaaaaaaaaaaaaa\"},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa1\",\"ts\":1518203456}]"
}
I am trying to decode the content of mandrill_events, so that I can then access some values, but I think the bracket is throwing it off.
get_in(payload, ["mandrill_events"]) |> Base.url_decode64 |> Poison.decode!
But that didn't work either.
** (ArgumentError) argument error
:erlang.iolist_to_binary(:error)
(poison) lib/poison/parser.ex:35: Poison.Parser.parse/2
(poison) lib/poison/parser.ex:51: Poison.Parser.parse!/2
(poison) lib/poison.ex:83: Poison.decode!/2
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
(iex) lib/iex/evaluator.ex:250: IEx.Evaluator.handle_eval/5
(iex) lib/iex/evaluator.ex:230: IEx.Evaluator.do_eval/3
(iex) lib/iex/evaluator.ex:208: IEx.Evaluator.eval/3
(iex) lib/iex/evaluator.ex:94: IEx.Evaluator.loop/1
(iex) lib/iex/evaluator.ex:24: IEx.Evaluator.init/4

Short answer: get_in(a, ["mandrill_events"]) |> Poison.decode! should give you what you want.
The reason as to why the answer provided here includes the operation |> Base.url_decode64 is because that question was dealing with base64 encoded payloads. Base64 encoding is simply a way of mapping a payload to a subset of the ASCII characters that are guaranteed to be recognized by every router, so that the payload value does not get corrupted when in transit.
For example, you could do:
get_in(a, ["mandrill_events"]) |> Base.url_encode64
which will render something like this:
"W3siZXZlbnQiOiJzZW5kIiwibXNnIjp7InRzIjoxMzY1MTA5OTk5LCJzdWJqZWN0IjoiVGhpcyBhbiBleGFtcGxlIHdlYmhvb2sgbWVzc2FnZSIsImVtYWlsIjoiZXhhbXBsZS53ZWJob29rQG1hbmRyaWxsYXBwLmNvbSIsInNlbmRlciI6ImV4YW1wbGUuc2VuZGVyQG1hbmRyaWxsYXBwLmNvbSIsInRhZ3MiOlsid2ViaG9vay1leGFtcGxlIl0sIm9wZW5zIjpbXSwiY2xpY2tzIjpbXSwic3RhdGUiOiJzZW50IiwibWV0YWRhdGEiOnsidXNlcl9pZCI6MTExfSwiX2lkIjoiZXhhbXBsZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJfdmVyc2lvbiI6ImV4YW1wbGVhYWFhYWFhYWFhYWFhYWEifSwiX2lkIjoiZXhhbXBsZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJ0cyI6MTUxODIwMzQ1Nn0seyJldmVudCI6InNlbmQiLCJtc2ciOnsidHMiOjEzNjUxMDk5OTksInN1YmplY3QiOiJUaGlzIGFuIGV4YW1wbGUgd2ViaG9vayBtZXNzYWdlIiwiZW1haWwiOiJleGFtcGxlLndlYmhvb2tAbWFuZHJpbGxhcHAuY29tIiwic2VuZGVyIjoiZXhhbXBsZS5zZW5kZXJAbWFuZHJpbGxhcHAuY29tIiwidGFncyI6WyJ3ZWJob29rLWV4YW1wbGUiXSwib3BlbnMiOltdLCJjbGlja3MiOltdLCJzdGF0ZSI6InNlbnQiLCJtZXRhZGF0YSI6eyJ1c2VyX2lkIjoxMTF9LCJfaWQiOiJleGFtcGxlYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYTEiLCJfdmVyc2lvbiI6ImV4YW1wbGVhYWFhYWFhYWFhYWFhYWEifSwiX2lkIjoiZXhhbXBsZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWExIiwidHMiOjE1MTgyMDM0NTZ9XQ=="
When you are dealing with base64 encoded payload, you will need to first decode it so you get a JSON string which in turn you can deserialize using Poison.
As a full sanity test, the following would work as well:
get_in(a, ["mandrill_events"]) |> Base.url_encode64 |> Base.url_decode64 |> Poison.decode!
Of course, if the string is not base64 encoded, and you try to base64 decode it accordingly as you currently are doing, then it will throw an :error which Poison does not know how to convert to an elixir term as its input is to be a JSON string, not an atom

Related

How to get the value of a JSON property in Elixir

I'm returning this kind os data to an variable:
{"numbers":[0.8832325122263557,0.9905363563950811, ...]}
How can i remove this string "numbers": and stay just with the [] ?
Here's a script that
Uses Mix.install/2 to instal Jason. In a mix project you would add jason to your deps in mix.exs instead.
Uses the ~S sigil to quote the JSON without needing to escape the " characters.
Uses Jason.decode!/2 to parse the JSON.
Uses Map.get/3 to get the value of the numbers key from the resulting map.
Inspects the value with IO.inspect/2, causing it to print out.
Uses |> pipes to pass the data between function calls succinctly.
Mix.install([:jason])
~S({"numbers":[0.8832325122263557,0.9905363563950811]})
|> Jason.decode!()
|> Map.get("numbers")
|> IO.inspect()
Running and output:
$ elixir example.exs
[0.8832325122263557, 0.9905363563950811]
Just to propose an alternative solution without a JSON parser, I would use Regex.
Captures any string inside brackets, e.g. [numbers]
Splits the string by ,
Converts the string numbers to floats
string = ~S({"numbers":[0.8832325122263557,0.9905363563950811]})
~r/(?:.+\[)(?'numbers'.+)(?:\].+)/
|> Regex.scan(string, capture: ["numbers"])
|> List.first()
|> List.first()
|> String.split(",")
|> Enum.map(&String.to_float/1)
This works well if your input is always expected to be like {"numbers":[0.8832325122263557,0.9905363563950811, ...]}.

Proper JSON Escaping in Clojure API

I have a Clojure function that writes out the contents of a vector of URLs to JSON, and returns this from an API endpoint. This endpoint data is read by an Elm JSON Decoder on a frontend app. Running this Clojure function directly gives me the following:
(json/write-str ["http://www.example.com?param=value" "http://www.example2.com?param=value"])
Which returns:
"[\"http:\\/\\/www.example.com?param=value\",\"http:\\/\\/www.example2.com?param=value\"]"
Wonderful! If I input this result directly into an Elm Decoder such as this:
stringArrayDecoder : Json.Decoder.Decoder (List String)
stringArrayDecoder =
Json.Decoder.list Json.Decoder.string
It parses it happily with no error.
However, when I view the JSON response from the endpoint, it is losing some of the escaping and I get this:
["http:\/\/www.example.com?param=value","http:\/\/www.example2.com?param=value"]
Which my Elm decoder cannot read.
How do I avoid this? How can I get the fully escaped JSON values produced by my internal function through the API endpoint and into my Elm frontend Decoder?
JSON allows you to escape forward slashes / and other characters as to prevent stuff like </ from popping up in html.
write-str has an :escape-slash boolean option:
:escape-slash boolean
If true (default) the slash / is escaped as \/
Thus you can instead write
(json/write-str ["http://url.one" "http://url.two"] :escape-slash false)
=> "[\"http://url.one\",\"http://url.two\"]"

Get the JSON from HttpEntity

I am using akka.http.scaladsl.model.HttpResponse, HttpEntity.
After getting the response , it is of type responseEntity of the format (Content-type: 'application/json', {MyJSONHERE}). Is there a way I can extract my json from the entity.
I tried entity.getDataBytes which gives the content of the entity in ByteString format. I want to properly read the JSON and parse it. Can someone guide me on this?
Code below works for me
entity.dataBytes.runWith(Sink.fold(ByteString.empty)(_ ++ _)).map(_.utf8String) map { result =>
JsonMethods.parse(result)
}
dataBytes returns Source[ByteString, Any], Sink.fold combines all parts of the stream into one ByteString and utf8String converts ByteString into usual String.
Here is some useful docs about HttpEntity.
Can you try below code?
entity.getDataBytes.utf8String
That would return String representation of JSON.

Ruby parse string to json

So I have some json that looks like this, which I got after taking it out of some other json by doing response.body.to_json:
{\n \"access_token\": \"<some_access_token>\",\n \"token_type\": \"Bearer\",\n \"expires_in\": 3600,\n \"id_token\": \<some_token>\"\n}\n"
I want to pull out the access_token, so I do
to_return = {token: responseJson[:access_token]}
but this gives me a
TypeError: no implicit conversion of Symbol into Integer
Why? How do I get my access token out? Why are there random backslashes everywhere?
to_json doesn't parse JSON - it does the complete opposite: it turns a ruby object into a string containing the JSON representation of that object is.
It's not clear from your question what response.body is. It could be a string, or depending on your http library it might have already been parsed for you.
If the latter then
response.body["access_token"]
Will be your token, if the former then try
JSON.parse(response.body)["access_token"]
Use with double quotes when calling access_token. Like below:
to_return = {token: responseJson["access_token"]}
Or backslashes are escaped delimiters and make sure you first parse JSON.

Erlang: Parse string to json

I have the following string:
"{\"headers\":[\"CNPJ\",\"PDF\",\"error\"],\"rows\":[[\"17192451000170\",\"FILE:application/pdf;170286;\",null],[\"234566767544\",\"FILE:application/pdf;456378;\",null],[\"233456767544\",\"FILE:application/pdf;456378;\",null]]}"
how do I parse it to a normal Json format?
meaning:
{"rows" :[
{"CNPJ":"17192451000170","PDF":"FILE:application/pdf;170286;","error":null},
{"CNPJ":"17192451000170","PDF":"FILE:application/pdf;170286;","error":null},
{"CNPJ":"17192451000170", "PDF":"FILE:application/pdf;170286;,"error":null"}
]}
or any other json format
This is already a valid JSON format.
If you just want to strip \ then you can simply:
(hbd#crayon2.yoonka.com)31> JsonOrg = <<"{\"headers\":[\"CNPJ\",\"PDF\",\"error\"],\"rows\":[[\"17192451000170\",\"FILE:application/pdf;170286;\",null],[\"234566767544\",\"FILE:application/pdf;456378;\",null],[\"233456767544\",\"FILE:application/pdf;456378;\",null]]}">>.
<<"{\"headers\":[\"CNPJ\",\"PDF\",\"error\"],\"rows\":[[\"17192451000170\",\"FILE:application/pdf;170286;\",null],[\"234566767544\",\"FI"...>>
(hbd#crayon2.yoonka.com)32> io:format("~s~n", [binary_to_list(JsonOrg)]).
{"headers":["CNPJ","PDF","error"],"rows":[["17192451000170","FILE:application/pdf;170286;",null],["234566767544","FILE:application/pdf;456378;",null],["233456767544","FILE:application/pdf;456378;",null]]}
ok
You can also parse back and forth between Json and Erlang. I tested that with the yajler decoder:
(hbd#crayon2.yoonka.com)43> {ok, Parsed} = yajler:decode(<<"{\"headers\":[\"CNPJ\",\"PDF\",\"error\"],\"rows\":[[\"17192451000170\",\"FILE:application/pdf;170286;\",null],[\"234566767544\",\"FILE:application/pdf;456378;\",null],[\"233456767544\",\"FILE:application/pdf;456378;\",null]]}">>).
{ok,[{<<"headers">>,[<<"CNPJ">>,<<"PDF">>,<<"error">>]},
{<<"rows">>,
[[<<"17192451000170">>,<<"FILE:application/pdf;170286;">>,
undefined],
[<<"234566767544">>,<<"FILE:application/pdf;456378;">>,
undefined],
[<<"233456767544">>,<<"FILE:application/pdf;456378;">>,
undefined]]}]}
(hbd#crayon2.yoonka.com)44> Json = binary:list_to_bin(yajler:encode(Parsed)).
<<"{\"headers\":[\"CNPJ\",\"PDF\",\"error\"],\"rows\":[[\"17192451000170\",\"FILE:application/pdf;170286;\",\"undefined\"],[\"2345667675"...>>
Yajler is an Erlang NIF so it is using a C library, in this case called yajl, to do the actual parsing, but I imagine a similar result you would get from other Erlang applications that can parse JSON.