I'm returning this kind os data to an variable:
{"numbers":[0.8832325122263557,0.9905363563950811, ...]}
How can i remove this string "numbers": and stay just with the [] ?
Here's a script that
Uses Mix.install/2 to instal Jason. In a mix project you would add jason to your deps in mix.exs instead.
Uses the ~S sigil to quote the JSON without needing to escape the " characters.
Uses Jason.decode!/2 to parse the JSON.
Uses Map.get/3 to get the value of the numbers key from the resulting map.
Inspects the value with IO.inspect/2, causing it to print out.
Uses |> pipes to pass the data between function calls succinctly.
Mix.install([:jason])
~S({"numbers":[0.8832325122263557,0.9905363563950811]})
|> Jason.decode!()
|> Map.get("numbers")
|> IO.inspect()
Running and output:
$ elixir example.exs
[0.8832325122263557, 0.9905363563950811]
Just to propose an alternative solution without a JSON parser, I would use Regex.
Captures any string inside brackets, e.g. [numbers]
Splits the string by ,
Converts the string numbers to floats
string = ~S({"numbers":[0.8832325122263557,0.9905363563950811]})
~r/(?:.+\[)(?'numbers'.+)(?:\].+)/
|> Regex.scan(string, capture: ["numbers"])
|> List.first()
|> List.first()
|> String.split(",")
|> Enum.map(&String.to_float/1)
This works well if your input is always expected to be like {"numbers":[0.8832325122263557,0.9905363563950811, ...]}.
Related
I'm calling an API from Go and trying to push json string data from another api call into it.
I can hand craft the calls using a payload like
payload := strings.NewReader('[{"value1":333, "value2":444}]'
and everything is happy.
I'm now trying to covert this to take the json string {"value1":333, "value2":444} as an input parameter of type string to a function, but when I try and use that as the payload, the api is responding with
expected type: JSONArray, found: JSONObject
I naively tried setting the input to the function as []string and appending the data to an array as the input, but then strings.NewReader complained that it was being fed an array.. which is was.
I'm at a loss to work out how to convert a string of json into a json array that the api will be happy with.
I tried just surrounding the string with [] but the compiler threw a fit about incorrect line termination.
Must have been doing something wrong with the string, surrounding the {} with [] let the function pass the data, but there must be a better way than this.
Any ideas, or am I making this harder than it should be?
You were on the right track with the brackets, but you actually need to append the characters to the string. For example:
str := `{"value1":333, "value2":444}`
str = "[" + str + "]"
// [{"value1":333, "value2":444}]
https://play.golang.org/p/rWHCLDCAngd
If you use brackets outside a string or rune literal, then it is parsed as Go language syntax.
I'm learning Elm and one thing that has puzzled me is 'Json.Decode.succeed'. According to the docs
succeed : a -> Decoder a
Ignore the JSON and produce a certain Elm value.
decodeString (succeed 42) "true" == Ok 42
decodeString (succeed 42) "[1,2,3]" == Ok 42
decodeString (succeed 42) "hello" == Err ...
I understand that (although, as a beginner, I don't yet see its use). But this method is also used in a Decode pipeline, thus:
somethingDecoder : Maybe Wookie -> Decoder Something
somethingDecoder maybeWookie =
Json.Decode.succeed Something
|> required "caterpillar" Caterpillar.decoder
|> required "author" (Author.decoder maybeWookie)
What is going on here? That is, if 'succeed' ignores the JSON that's passed to it, how is it used to read JSON and turn it into Elm values? Any clues appreciated!
Just to start, the intuition for a decoder pipeline is that it acts like a curried function where piping with required and optional applies arguments one-by-one. Expect that everything, both the function, its arguments and the return value are all wrapped in Decoders.
So as an example:
succeed Something
|> required (succeed 42)
|> required (succeed "foo")
is equivalent to
succeed (Something 42 "foo")
and
decodeString (succeed (Something 42 "foo")) whatever
will return Ok (Something 42 "foo") as long as whatever is valid JSON.
When everything succeeds it's just a really convoluted function call. The more interesting aspect of decoders, and the reason we use them in the first place, is in the error path. But since 'succeed' is what's of interest here, we'll ignore that and save a lot of time, text and brain cells. Just know that without considering the error path this will all seem very contrived.
Anyway, let's try to recreate this to see how it works.
Decode.map2
The key to the pipelines, apart form the pipe operator, is the Decode.map2 function. You've probably already used it, or its siblings, if you've tried writing JSON decoders without using pipelines. We can implement our example above using map2 like this:
map2 Something
(succeed 42)
(succeed "foo")
This will work exactly like the example above. But the problem with this, from a user POV, is that if we need to add another argument we also have to change map2 to map3. And also Something isn't wrapped in a decoder, which is boring.
Calling functions wrapped in Decoders
The reason this is useful anyway is because it gives us access to several values at the same time, and the ability to combine them in whatever way we want. We can use this to call a function inside a Decoder with an argument inside a Decoder and have the result also wrapped in a Decoder:
map2 (\f x -> f x)
(succeed String.fromInt)
(succeed 42)
Currying and partial application
Unfortunately this still has the problem of needing to change the map function if we need more arguments. If only there was a way to apply arguments to a function one at a time... like if we had currying and partial application. Since we have a way to call functions wrapped in decoders now, what if we return a partially applied function instead and apply the remaining arguments later?
map2 (\f x -> f x)
(succeed Something)
(succeed 42)
will return a Decoder (string -> Something), so now we just have to rinse and repeat with this and the last argument:
map2 (\f x -> f x)
(map2 (\f x -> f x)
(succeed Something)
(succeed 42))
(succeed "")
Et voila, we have now recreated JSON decode pipelines! Although it might not look like it on the surface.
Ceci n'est pas une pipe
The final trick is to use map2 with the pipe operator. The pipe is essentially defined as \x f -> f x. See how similar this looks to the function we've been using? The only difference is that the arguments are swapped around, so we need to swap the order we pass arguments as well:
map2 (|>)
(succeed "")
(map2 (|>)
(succeed 42)
(succeed Something))
and then we can use the pipe operator again to reach the final form
succeed Something
|> map2 (|>)
(succeed 42)
|> map2 (|>)
(succeed "")
It should now be apparent that required is just an alias for map2 (|>).
And that's all there is to it!
Here I have a payload coming to my controller action endpoint:
%{
"mandrill_events" => "[{\"event\":\"send\",\"msg\":{\"ts\":1365109999,\"subject\":\"This an example webhook message\",\"email\":\"example.webhook#mandrillapp.com\",\"sender\":\"example.sender#mandrillapp.com\",\"tags\":[\"webhook-example\"],\"opens\":[],\"clicks\":[],\"state\":\"sent\",\"metadata\":{\"user_id\":111},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa\",\"_version\":\"exampleaaaaaaaaaaaaaaa\"},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa\",\"ts\":1518203456},{\"event\":\"send\",\"msg\":{\"ts\":1365109999,\"subject\":\"This an example webhook message\",\"email\":\"example.webhook#mandrillapp.com\",\"sender\":\"example.sender#mandrillapp.com\",\"tags\":[\"webhook-example\"],\"opens\":[],\"clicks\":[],\"state\":\"sent\",\"metadata\":{\"user_id\":111},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa1\",\"_version\":\"exampleaaaaaaaaaaaaaaa\"},\"_id\":\"exampleaaaaaaaaaaaaaaaaaaaaaaaaa1\",\"ts\":1518203456}]"
}
I am trying to decode the content of mandrill_events, so that I can then access some values, but I think the bracket is throwing it off.
get_in(payload, ["mandrill_events"]) |> Base.url_decode64 |> Poison.decode!
But that didn't work either.
** (ArgumentError) argument error
:erlang.iolist_to_binary(:error)
(poison) lib/poison/parser.ex:35: Poison.Parser.parse/2
(poison) lib/poison/parser.ex:51: Poison.Parser.parse!/2
(poison) lib/poison.ex:83: Poison.decode!/2
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
(iex) lib/iex/evaluator.ex:250: IEx.Evaluator.handle_eval/5
(iex) lib/iex/evaluator.ex:230: IEx.Evaluator.do_eval/3
(iex) lib/iex/evaluator.ex:208: IEx.Evaluator.eval/3
(iex) lib/iex/evaluator.ex:94: IEx.Evaluator.loop/1
(iex) lib/iex/evaluator.ex:24: IEx.Evaluator.init/4
Short answer: get_in(a, ["mandrill_events"]) |> Poison.decode! should give you what you want.
The reason as to why the answer provided here includes the operation |> Base.url_decode64 is because that question was dealing with base64 encoded payloads. Base64 encoding is simply a way of mapping a payload to a subset of the ASCII characters that are guaranteed to be recognized by every router, so that the payload value does not get corrupted when in transit.
For example, you could do:
get_in(a, ["mandrill_events"]) |> Base.url_encode64
which will render something like this:
"W3siZXZlbnQiOiJzZW5kIiwibXNnIjp7InRzIjoxMzY1MTA5OTk5LCJzdWJqZWN0IjoiVGhpcyBhbiBleGFtcGxlIHdlYmhvb2sgbWVzc2FnZSIsImVtYWlsIjoiZXhhbXBsZS53ZWJob29rQG1hbmRyaWxsYXBwLmNvbSIsInNlbmRlciI6ImV4YW1wbGUuc2VuZGVyQG1hbmRyaWxsYXBwLmNvbSIsInRhZ3MiOlsid2ViaG9vay1leGFtcGxlIl0sIm9wZW5zIjpbXSwiY2xpY2tzIjpbXSwic3RhdGUiOiJzZW50IiwibWV0YWRhdGEiOnsidXNlcl9pZCI6MTExfSwiX2lkIjoiZXhhbXBsZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJfdmVyc2lvbiI6ImV4YW1wbGVhYWFhYWFhYWFhYWFhYWEifSwiX2lkIjoiZXhhbXBsZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJ0cyI6MTUxODIwMzQ1Nn0seyJldmVudCI6InNlbmQiLCJtc2ciOnsidHMiOjEzNjUxMDk5OTksInN1YmplY3QiOiJUaGlzIGFuIGV4YW1wbGUgd2ViaG9vayBtZXNzYWdlIiwiZW1haWwiOiJleGFtcGxlLndlYmhvb2tAbWFuZHJpbGxhcHAuY29tIiwic2VuZGVyIjoiZXhhbXBsZS5zZW5kZXJAbWFuZHJpbGxhcHAuY29tIiwidGFncyI6WyJ3ZWJob29rLWV4YW1wbGUiXSwib3BlbnMiOltdLCJjbGlja3MiOltdLCJzdGF0ZSI6InNlbnQiLCJtZXRhZGF0YSI6eyJ1c2VyX2lkIjoxMTF9LCJfaWQiOiJleGFtcGxlYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYTEiLCJfdmVyc2lvbiI6ImV4YW1wbGVhYWFhYWFhYWFhYWFhYWEifSwiX2lkIjoiZXhhbXBsZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWExIiwidHMiOjE1MTgyMDM0NTZ9XQ=="
When you are dealing with base64 encoded payload, you will need to first decode it so you get a JSON string which in turn you can deserialize using Poison.
As a full sanity test, the following would work as well:
get_in(a, ["mandrill_events"]) |> Base.url_encode64 |> Base.url_decode64 |> Poison.decode!
Of course, if the string is not base64 encoded, and you try to base64 decode it accordingly as you currently are doing, then it will throw an :error which Poison does not know how to convert to an elixir term as its input is to be a JSON string, not an atom
A web service returns a response as ByteString
req <- parseUrl "https://api.example.com"
res <- withManager $ httpLbs $ configReq req
case (HashMap.lookup "result" $ responseBody res) of .... -- error - responseBody returns ByteString
where
configReq r = --......
To be more specific, responseBody returns data in ByteString, although it's actually valid JSON. I need to find a value in it. Obviously, it would be easier to find it if it was JSON and not ByteString.
If that's the case, how do I convert it to JSON?
UPDATE:
decode $ responseBody resp :: IO (Either String Aeson.Value)
error:
Couldn't match expected type `IO (Either String Value)'
with actual type `Maybe a0'
You'll find several resources for converting bytestring to JSON. The simplest use cases are on the hackage page itself, and the rest you can infer using type signatures of the entities involved.
https://hackage.haskell.org/package/aeson-0.7.0.6/docs/Data-Aeson.html
But here's a super quick intro to JSON with Aeson:
In most languages, you have things like this:
someString = '{ "name" : ["value1", 2] }'
aDict = json.loads(someString)
This is obviously great, because JSON has a nearly one to one mapping with a fundamental data-structure of the language. Containers in most dynamic languages can contain values of any type, and so moving from JSON to data structure is a single step.
However, that is not the case with Haskell. You can't put things of arbitrary types into a container like type (A list, or a dictionary).
So Aeson does a neat thing. It defines an intermediate Haskell type for you, that maps directly to JSON.
A fundamental unit in Aeson is a Value. The Value can contain many things. Like an integer, string, an array, or an object.
https://hackage.haskell.org/package/aeson-0.7.0.6/docs/Data-Aeson.html#t:Value
An aeson array is a Vector (like a list but better) of Values and an aeson object is a HashMap of Text to Values
The next interesting step is that you can define functions that will convert an Aeson value to your Haskell type. This completes the loop. ByteString to Value to a custom type.
So all you do is implement parseJSON and toJSON functions that convert aeson Values to your type and vice-versa. The bit that converts a bytestring into a valid aeson value is implemented by aeson. So the heavy lifting is all done.
Just important to note, that Aeson bytestring is a lazy bytestring, so you might need some strict to lazy helpers.
stringToLazy :: String -> ByteString
stringToLazy x = Data.Bytestring.Lazy.fromChunks [(Data.ByteString.Char8.pack x)]
lazyToString :: ByteString -> String
lazyToString x = Data.ByteString.Char8.unpack $ Data.ByteString.Char8.concat $ Data.ByteString.Lazy.toChunks
That should be enough to get started with Aeson.
--
Common decoding functions with Aeson:
decode :: ByteString -> Maybe YourType
eitherDecode :: ByteString -> Either String YourType.
In your case, you're looking for eitherDecode.
I'm trying to figure out Haskell's json library. However, I'm running into a bit of an issue in ghci:
Prelude> import Text.JSON
Prelude Text.JSON> decode "[1,2,3]"
<interactive>:1:0:
Ambiguous type variable `a' in the constraint:
`JSON a' arising from a use of `decode' at <interactive>:1:0-15
Probable fix: add a type signature that fixes these type variable(s)
I think this has something to do with the a in the type signature:
decode :: JSON a => String -> Result a
Can someone show me:
How to decode a string?
What's going with the type system here?
You need to specify which type you want to get back, like this:
decode "[1,2,3]" :: Result [Integer]
-- Ok [1,2,3]
If that line was part of a larger program where you would go on and use the result of decode the type could just be inferred, but since ghci doesn't know which type you need, it can't infer it.
It's the same reason why read "[1,2,3]" doesn't work without a type annotation or more context.
The decode function is defined as follows:
decode :: JSON a => String -> Result a
In a real program, the type inference engine can usually figure out what type to expect from decode. For example:
userAge :: String -> Int
userAge input = case decode input of
Result a -> a
_ -> error $ "Couldn't parse " ++ input
In this case, the type of userAge causes the typechecker to infer that decode's return value, in this particular case, is Result Int.
However, when you use decode in GHCi, you must specify the type of the value, e.g.:
decode "6" :: Result Int
=> Ok 6
A quick glance at the docs seems to suggest that the purpose of this function is to allow you to read JSON into any Haskell data structure of a supported type, so
decode "[1, 2, 3]" :: Result [Int]
ought to work