I need to consume a json source that represents floats as strings* and I can't figure out how.
It is almost easy:
Json.Decode.map String.toFloat Json.Decode.string
However, that produces a Maybe Float and I'd prefer it fail altogether if it cant decode the string.
(*) The reason for this is that the real datatype is Decimal, so "1.5" != "1.50". My application doesn't have to care though.
You can either install elm-community/json-extra and use Json.Decode.Extra.parseFloat
or just copy its implementation
fromMaybe : String -> Maybe a -> Decode.Decoder a
fromMaybe error val =
case val of
Just v ->
Decode.succeed v
Nothing ->
Decode.fail error
parseFloat : Decode.Decoder Float
parseFloat =
Decode.string |> Decode.andThen (String.toFloat >> fromMaybe "failed to parse as float")
Another option which makes fromMaybe unnecessary:
floatDecoder : Json.Decoder Float
floatDecoder =
Json.string |> Json.andThen (String.toFloat >> Maybe.withDefault 0.0 >> Json.succeed)
Just in case it helps someone else ;)
Related
In Elm, there's no native way to encode/decode a custom type. This makes send and receive custom-typed values to JS difficult. I was confused for a while and wonder how to handle simple custom types like the one below?
type MyCustomType
= A
| B
| C
Here's a quick and simple demo of how to encode and decode any custom types.
Say you have a custom type like this:
type MyCustomType
= A
| B
| C
You can encode MyCustomType directly as a string:
encodeMyCustomType : MyCustomType -> Encode.Value
encodeMyCustomType myCustomType =
Encode.string <|
case myCustomType of
A -> "A"
B -> "B"
C -> "C"
Decoding MyCustomType is slightly more involved. You need to use Decode.andThen to check which variant is found and use Decode.fail in case no valid variant is found:
Decode.string |>
Decode.andThen
(\str ->
case str of
"A" -> Decode.succeed A
"B" -> Decode.succeed B
"C" -> Decode.succeed C
_ -> Decode.fail "Invalid MyCustomType"
)
Is there an easy way to make Json.Decode case insensitive in elm (0.18)?
decodeDepartmentDate : Json.Decode.Decoder DepartmentDate
decodeDepartmentDate =
Json.Decode.map6 DepartmentDate
(field "nameOfDay" Json.Decode.string)
(field "orderDate" Convert.datePart)
(field "mealTimeID" Json.Decode.string)
(field "mealTime" Json.Decode.string)
(field "departmentID" Json.Decode.string)
(field "department" Json.Decode.string)
I want to be able to use the same elm SPA against multiple back ends and avoid issues like this by default:
BadPayload "Expecting an object with a field named `nameOfDay` at _[11]
but instead got: {\"NameOfDay\":\"Wednesday\",\"OrderDate\":\"2018-09-05T00:00:00\",
\"MealTimeID\":\"546ccee0-e070-403e-a15b-63f4e1366054\",\"MealTime\":\"All Day\",
\"StartTime\":\"2018/06/05 05:04:38\",\"DepartmentID\":\"066a1c9f-97da-487e-b82f-f933b159c042\",
\"Department\":\"Side walk\"}"
Thanks
As far as I'm aware, there's no ready-made solution for doing so. But you can make your own!
The easiest way is probably to just generate the different casings and make your own field decoder using oneOf:
myField name decoder =
Decode.oneOf
[ Decode.field name decoder
, Decode.field (String.toLower) decoder
]
Another approach would be to decode the object as key/value pairs without decoding the values, transforming the keys and then re-encoding it to be able to use the existing JSON decoders on it:
lowerCaseKeys =
Decode.keyValuePairs Decode.value
|> Decode.map (List.map (\(key, value) -> (String.toLower key, value)))
|> Decode.map (Encode.object)
But since the value is now wrapped in a Decoder you'd have to use decodeValue on that and ultimately end up with a double-wrapped Result, which isn't very nice. I might be missing some elegant way of making this work though.
Instead it seems better to not re-encode it, but just make your own field decoder to work on the dict. This will also allow you to ignore casing on the keys you specify.
lowerCaseKeys : Decode.Decoder (Dict.Dict String Decode.Value)
lowerCaseKeys =
Decode.keyValuePairs Decode.value
|> Decode.map (List.map (\( key, value ) -> ( String.toLower key, value )))
|> Decode.map Dict.fromList
myField : String -> Decode.Decoder a -> Dict.Dict String Decode.Value -> Decode.Decoder a
myField name decode dict =
case Dict.get (String.toLower name) dict of
Just value ->
case Decode.decodeValue decode value of
Ok v ->
Decode.succeed v
Err e ->
e |> Decode.errorToString |> Decode.fail
Nothing ->
Decode.fail "missing key"
result =
Decode.decodeString (lowerCaseKeys |> Decode.andThen (myField "fOO" Decode.int)) """{ "Foo": 42 }"""
You can define a variant of field that disregards case.
fieldInsensitive : String -> Decode.Decoder a -> Decode.Decoder a
fieldInsensitive f d =
let
flow = String.toLower f
in
Decode.keyValuePairs Decode.value |> Decode.andThen
(\ l -> l |> List.filter (\(k, v) -> String.toLower k == flow)
|> List.map (\(k, v) -> v)
|> List.head
|> Maybe.map Decode.succeed
|> Maybe.withDefault (Decode.fail "field not found")
) |> Decode.andThen
(\ v -> case Decode.decodeValue d v of
Ok w -> Decode.succeed w
Err e -> Decode.fail (Decode.errorToString e)
)
This is more or less the same code as #glennsl's answer, but wrapped up in a self-contained function. The advantage is a simpler interface, the disadvantage is that if you lookup multiple fields in the same object you will be repeating work.
Note that this code makes a rather arbitrary decision if there are multiple fields with the same key up to case! For more reliable code, it might be a better idea to fail if a key exists more than once.
I'm working with json code. And I can not extract from it the value due to an error in the type.And to get this error I'm using a function eitherDecode
Code part:
extractValues :: BSL.ByteString -> FullWeather
extractValues rawJSON = do
let result = eitherDecode rawJSON :: Either String FullWeather
case result of
Left problem -> return problem
Right ok -> return ok
Eroor:
Couldn't match expected type ‘FullWeather’
with actual type ‘m0 String’
25 Left problem -> return problem
Couldn't match expected type ‘FullWeather’
with actual type ‘m1 FullWeather’
26 Right ok -> return ok
Three problems I can spot:
do-notation is for monadic values and is syntax sugar for applications of (>>=) and (>>). Don't use do-notation for function that don't return a monadic value (like IO a, [a], Maybe a...). Outside of a do-notation the correct syntax is let ... in ....
I suggest following a Haskell tutorial like Learn you a Haskell, which will teach you the correct usage of do-notation.
The function return has type return :: Monad m => a -> m a.
return is not a keyword.
If your function returns a FullWeather, it can't return a string too. This is what Either or Maybe are for.
Another thing you can do is throw an error.
There are 3 basic solutions we can use for this:
Throw an error in the case of a problem.
extractValues :: BSL.ByteString -> FullWeather
extractValues rawJSON =
let result = eitherDecode rawJSON :: Either String FullWeather
in case result of
Left problem -> error problem
Right ok -> ok
Return a Maybe FullWeather instead of FullWeather.
extractValues :: BSL.ByteString -> Maybe FullWeather
extractValues rawJSON =
let result = eitherDecode rawJSON :: Either String FullWeather
in case result of
Left problem -> Nothing
Right ok -> Just ok
Return an Either String FullWeather instead of FullWeather.
extractValues :: BSL.ByteString -> Either String FullWeather
extractValues = eitherDecode
This is actually in continuation of the question I asked a few days back. I took the applicative functors route and made my own instances.
I need to parse a huge number of json statements all in a file, one line after the other. An example json statement is something like this -
{"question_text": "How can NBC defend tape delaying the Olympics when everyone has
Twitter?", "context_topic": {"followers": 21, "name": "NBC Coverage of the London
Olympics (July & August 2012)"}, "topics": [{"followers": 2705,
"name": "NBC"},{"followers": 21, "name": "NBC Coverage of the London
Olympics (July & August 2012)"},
{"followers": 17828, "name": "Olympic Games"},
{"followers": 11955, "name": "2012 Summer Olympics in London"}],
"question_key": "AAEAABORnPCiXO94q0oSDqfCuMJ2jh0ThsH2dHy4ATgigZ5J",
"__ans__": true, "anonymous": false}
sorry for the json formatting. It got bad
I have about 10000 such json statements and I need to parse them. The code I have written is
something like this -
parseToRecord :: B.ByteString -> Question
parseToRecord bstr = (\(Ok x) -> x) decodedObj where decodedObj = decode (B.unpack bstr) :: Result Question
main :: IO()
main = do
-- my first line in the file tells how many json statements
-- are there followed by a lot of other irrelevant info...
ts <- B.getContents >>= return . fst . fromJust . B.readInteger . head . B.lines
json_text <- B.getContents >>= return . tail . B.lines
let training_data = take (fromIntegral ts) json_text
let questions = map parseToRecord training_data
print $ questions !! 8922
This code gives me a runtime error Non-exhaustive patterns in lambda. The error references to \(Ok x) -> x in the code. By hit and trial, I came to the conclusion that the program works ok till the 8921th index and fails on the 8922th iteration.
I checked the corresponding json statement and tried to parse it standalone by calling the function on it and it works. However, it doesn't work when I call map. I don't really understand what is going on. Having learnt a little bit of haskell in "learn haskell for a great good", I wanted to dive into a real world programming project but seem to have got stuck here.
EDIT :: complete code is as follows
{-# LANGUAGE BangPatterns #-}
{-# OPTIONS_GHC -O2 -optc-O2 #-}
{-# OPTIONS_GHC -fno-warn-incomplete-uni-patterns #-}
import qualified Data.ByteString.Lazy.Char8 as B
import Data.Maybe
import NLP.Tokenize
import Control.Applicative
import Control.Monad
import Text.JSON
data Topic = Topic
{ followers :: Integer,
name :: String
} deriving (Show)
data Question = Question
{ question_text :: String,
context_topic :: Topic,
topics :: [Topic],
question_key :: String,
__ans__ :: Bool,
anonymous :: Bool
} deriving (Show)
(!) :: (JSON a) => JSObject JSValue -> String -> Result a
(!) = flip valFromObj
instance JSON Topic where
-- Keep the compiler quiet
showJSON = undefined
readJSON (JSObject obj) =
Topic <$>
obj ! "followers" <*>
obj ! "name"
readJSON _ = mzero
instance JSON Question where
-- Keep the compiler quiet
showJSON = undefined
readJSON (JSObject obj) =
Question <$>
obj ! "question_text" <*>
obj ! "context_topic" <*>
obj ! "topics" <*>
obj ! "question_key" <*>
obj ! "__ans__" <*>
obj ! "anonymous"
readJSON _ = mzero
isAnswered (Question _ _ _ _ status _) = status
isAnonymous (Question _ _ _ _ _ status) = status
parseToRecord :: B.ByteString -> Question
parseToRecord bstr = handle decodedObj
where handle (Ok k) = k
handle (Error e) = error (e ++ "\n" ++ show bstr)
decodedObj = decode (B.unpack bstr) :: Result Question
--parseToRecord bstr = (\(Ok x) -> x) decodedObj where decodedObj = decode (B.unpack bstr) :: Result Question
main :: IO()
main = do
ts <- B.getContents >>= return . fst . fromJust . B.readInteger . head . B.lines
json_text <- B.getContents >>= return . tail . B.lines
let training_data = take (fromIntegral ts) json_text
let questions = map parseToRecord training_data
let correlation = foldr (\x acc -> if (isAnonymous x == isAnswered x) then (fst acc + 1, snd acc + 1) else (fst acc, snd acc + 1)) (0,0) questions
print $ fst correlation
here's the data which can be given as input to the executable. I'm using ghc 7.6.3. If the program name is ans.hs, I followed these steps.
$ ghc --make ans.hs
$ ./ans < path/to/the/file/sample/answered_data_10k.in
thanks a lot!
The lambda function (\(Ok x) -> x) is partial in that it will only be able to match objects that were successfully decoded. If you are experiencing this, it indicates that your JSON parser is failing to parse a record, for some reason.
Making the parseToRecord function more informative would help you find the error. Try actually reporting the error, rather than reporting a failed pattern match.
parseToRecord :: B.ByteString -> Question
parseToRecord bstr = handle decodedObj
where handle (Ok k) = k
handle (Error e) = error e
decodedObj = decode (B.unpack bstr) :: Result Question
If you want more help, it might be useful to include the parser code.
Update
Based on your code and sample JSON, it looks like your code is first failing
when it encounters a null in the context_topic field of your JSON.
Your current code cannot handle a null, so it fails to parse. My fix would
be something like the following, but you could come up with other ways to
handle it.
data Nullable a = Null
| Full a
deriving (Show)
instance JSON a => JSON (Nullable a) where
showJSON Null = JSNull
showJSON (Full a) = showJSON a
readJSON JSNull = Ok Null
readJSON c = Full `fmap` readJSON c
data Question = Question
{ question_text :: String,
context_topic :: Nullable Topic,
topics :: [Topic],
question_key :: String,
__ans__ :: Bool,
anonymous :: Bool
} deriving (Show)
It also seems to fail on line 9002, where there is a naked value of "1000" on
that line, and it seems that several JSON values after that line lack the
'__ans__' field.
I would have suggestion to use Maybe in order to parse the null values:
data Question = Question
{ question_text :: String
, context_topic :: Maybe Topic
, topics :: [Topic]
, question_key :: String
, __ans__ :: Bool
, anonymous :: Bool
} deriving (Show)
And then change the readJSON function as follows (in addition, the missing ans-fields can be fixed by returning False on an unsuccessful parsing attempt):
instance JSON Question where
-- Keep the compiler quiet
showJSON = undefined
readJSON (JSObject obj) = Question <$>
obj ! "question_text" <*>
(fmap Just (obj ! "context_topic") <|> return Nothing) <*>
obj ! "topics" <*>
obj ! "question_key" <*>
(obj ! "__ans__" <|> return False) <*>
obj ! "anonymous"
readJSON _ = mzero
After getting rid of the 1000 in line 9000-something (like sabauma mentioned), I got 4358 as result. So maybe these slight changes are enough?
My question is simple. Why wrong pattern matching does not throw exception in Maybe monad. For clarity :
data Task = HTTPTask {
getParams :: [B.ByteString],
postParams :: [B.ByteString],
rawPostData :: B.ByteString
} deriving (Show)
tryConstuctHTTPTask :: B.ByteString -> Maybe Task
tryConstuctHTTPTask str = do
case decode str of
Left _ -> fail ""
Right (Object trie) -> do
Object getP <- DT.lookup (pack "getParams") trie
Object postP <- DT.lookup (pack "postParams") trie
String rawData <- DT.lookup (pack "rawPostData") trie
return $ HTTPTask [] [] rawData
Look at tryConstuctHTTPTask function. I think that when the pattern does not match (for example "Object getP") we must get something like "Prelude.Exception", instead i get the "Nothing". I like this behavior but i am not understand why.
Thanks.
Doing pattern <- expression in a do-block, will call fail when the pattern does not match. So it is equivalent to doing
expression >>= \x ->
case x of
pattern -> ...
_ -> fail
Since fail is defined as Nothing in the Maybe monad, you get Nothing for failed pattern matches using <-.