I would like to parse the values of a json object to a list.
Here's my current approach (simplified and the newtype is based on the results of: Aeson: derive some (but not all) fields of a struct (means I need it)):
Json:
{"v1": 1, "v2": 2}
Wanted result:
Test [1,2]
Current approach:
import Data.Aeson
import Data.HashMap.Strict (elems)
newtype Test = Test [Int]
instance FromJSON Test where
parseJSON (Object o) =
mapM parseJSON (elems o)
Compilation error:
• Couldn't match type ‘[b0]’ with ‘Test’
Expected type: aeson-1.1.2.0:Data.Aeson.Types.Internal.Parser Test
Actual type: aeson-1.1.2.0:Data.Aeson.Types.Internal.Parser [b0]
• In the expression: mapM parseJSON (elems o)
mapM parseJSON (elems o) returns Parser [Int], but you need Parser (Test [Int]), so the correct way to do that is:
instance FromJSON Test where
parseJSON (Object o) = Test <$> mapM parseJSON (elems o)
However, the type of argument of parseJSON is Value, the value of argument may not be Object, it can also be an Array, String or etc, so it is better to use withObject to check what is it as:
instance FromJSON Test where
parseJSON val = withObject "Test"
(\o -> Test <$> mapM parseJSON (elems o))
val
withObject will print a meaningful error message when the value of type Value is not Object.
Related
If I have a custom data type for parsing JSON with Aeson
data Response = Response
{ response :: [Body]
} deriving (Show)
instance FromJSON Response where
parseJSON (Object v) = Response <$> v .: "response"
parseJSON _ = mzero
data Body = Body
{ body_id :: Int
, brandId :: Int
} deriving (Show)
instance FromJSON Body where
parseJSON (Object v) = Body
<$> v .: "id"
<*> v .: "brandId"
parseJSON _ = mzero
raw :: BS.ByteString
raw = "{\"response\":[{\"id\":5977,\"brandId\":87}]}"
giving:
λ> decode raw :: Maybe Response
Just (Response {response = [Body {body_id = 5977, brandId = 87}]})
How do I derive the instances for FromJSON automatically?
I have tried:
data Response = Response
{ response :: [Body]
} deriving (Show,Generic)
data Body = Body
{ body_id :: Int
, brandId :: Int
} deriving (Show,Generic)
instance FromJSON Response
instance FromJSON Body
as has been suggested from some tutorials, but that gives:
λ> :l response.hs
[1 of 1] Compiling Response ( response.hs, interpreted )
response.hs:19:22:
Can't make a derived instance of `Generic Response':
You need DeriveGeneric to derive an instance for this class
In the data declaration for `Response'
response.hs:24:22:
Can't make a derived instance of `Generic Body':
You need DeriveGeneric to derive an instance for this class
In the data declaration for `Body'
Failed, modules loaded: none.
what am I doing wrong?
What the error is telling you is that you have to enable the DeriveGeneric extension in order for this to work. So you have to add:
{-# LANGUAGE DeriveGeneric #-}
right at the top of your file, or compile using the -XDeriveGeneric flag.
for some reason I can't wrap my head around arbitrarilly successful parses in Aeson, without making the whole system bork and cause a space leak.
Here's my issue:
newtype Foo = Foo
{ getFoo :: [(String, Maybe String)]
} deriving (Show, Eq)
instance ToJSON Foo where
toJSON (Foo xs) = object $
map (\(k,mv) -> T.pack k .= mv) xs
so far, encoding a Foo is fine and dandy. But, I want to make a parser that rejects a couple of keys, if they exist. Right now, I have a pseudo-rejection going on, and that's why I think I'm getting a bad outcome:
import qualified Data.HashMap as HM
-- the "duck-tape and chewing gum" approach
instance FromJSON Foo where
parseJSON (Object o) = Foo <$> parseJSON (Object theRest)
where
theRest = foldr HM.delete o [ "foo"
, "bar"
]
parseJSON _ = empty
This version is what caused me to think that manipulating the internal object was incorrect, because the parser may be getting "more" data in the HashMap, outside of the parser (because of the lazy bytestring being fed into it), but I am clearly not sure about this. So, I tried a different approach:
instance FromJSON Foo where
parseJSON (Object o) =
(Foo . filter (\(k,_) -> k `elem` toIgnore)) <$>
parseJSON (Object o)
where
toIgnore = ["foo", "bar"]
parseJSON _ = empty
But this also seems to cause a deadlock / space leak (not sure exactly what to diagnose this halting of execution). What would be the advised way to accept everything except a few keys of the object? I need to pattern-match on the (Object o) structure because I'm manually looking up o .: "foo" and o .: "bar" in a different component for my data type. Ideally, I would like to just remove those keys from the content and continue parsing, because I already accounted for them (hence - "the rest").
Is there any hope?
For your PartialAppContact example here is a more mundane approach which seems to work:
{-# LANGUAGE OverloadedStrings, QuasiQuotes #-}
import Data.Aeson
import qualified Data.Text as T
import qualified Data.HashMap.Strict as HM
import Control.Monad
import Text.Heredoc
type RequiredField = String
type OptionalField = String
data PartialAppContact = PartialAppContact
{ partialAppContactRequired :: [(RequiredField, String)]
, partialAppContactOptional :: [(OptionalField, Maybe String)]
} deriving (Show, Eq)
instance FromJSON PartialAppContact where
parseJSON (Object o) = do
let required = [ "firstName", "lastName", "email", "phoneNumber" ]
reqPairs <- forM required $ \k -> do
v <- o .: k
s <- parseJSON v
return (T.unpack k, s)
nonReqPairs <- forM [ (k,v) | (k,v) <- HM.toList o, k `notElem` required ] $ \(k,v) -> do
s <- parseJSON v
return (T.unpack k, s)
return $ PartialAppContact reqPairs nonReqPairs
test1 = Data.Aeson.eitherDecode "{\"firstName\":\"Athan'\"}" :: Either String PartialAppContact
input = [str|
| { "firstName": "a first name"
| , "lastName": "a last name"
| , "email": "asasd#asd.com"
| , "phoneNumber": "123-123-123"
| , "another field": "blah blah" }
|]
test2 = Data.Aeson.eitherDecode "{\"firstName\":\"Athan'\" }" :: Either String PartialAppContact
test3 = Data.Aeson.eitherDecode input :: Either String PartialAppContact
Update
Based on your comments, consider this idea for writing the instance:
import Data.List (partition)
instance FromJSON PartialAppContact where
parseJSON (Object o) = do
let required = [ "firstName", "lastName", "email", "phoneNumber" ]
let (group1, group2) = partition (\(k,_) -> k `elem` required) (HM.toList o)
reqFields <- forM group1 $ \(k,v) -> do s <- parseJSON v; return (T.unpack k, s)
otherFields <- forM group2 (\(k,v) -> (T.unpack k,) <$> parseJSON v)
return $ PartialAppContact reqFields otherFields
I found a working implementation requires the use of (.:?), to correctly implement optional, known fields. From there, you can freely decompose the HashMap and re-parseJSON it's subfields:
instance FromJSON Foo where
parseJSON (Object o) = do
mfoo <- o .:? "foo"
mbar <- o .:? "bar"
let foundFields = catMaybes [mfoo, mbar]
rest <- mapM (\(k,v) -> (T.unpack k,) <$> parseJSON v)
(toList theRest)
return $ Foo rest -- assuming you're done with `foundFields`
where
theRest = foldr HM.delete o ["foo", "bar"]
To see the final implementation of the issue discussed in the comments, see this commit.
I have a JSON doc that looks like:
{ "series": [[1,2], [2,3], [3,4]] }
I'd like to parse this into a set of data types:
data Series = Series [DataPoint]
data DataPoint = DataPoint Int Int -- x and y
I'm having lots of problems trying to write the FromJSON instance for DataPoint.
instance FromJSON DataPoint where
parseJSON (Array a) = ???
I've tried using Lens to destruct the DataPoint record, but it doesn't compile:
case a ^.. values . _Integer of -}
[x,y] -> DataPoint <$> x <*> y
_ -> mzero
That fails with this error (the first two lines I get even absent the lens trickery, just trying to create a DataPoint <$> 1 <*> 2):
Couldn't match type ‘aeson-0.7.0.6:Data.Aeson.Types.Internal.Parser
Integer’
with ‘Integer’
Expected type: (aeson-0.7.0.6:Data.Aeson.Types.Internal.Parser
Integer
-> Const
(Data.Monoid.Endo
[aeson-0.7.0.6:Data.Aeson.Types.Internal.Parse
(aeson-0.7.0.6:Data.Aeson.Types.Internal.Parser I
-> Value
-> Const
(Data.Monoid.Endo
[aeson-0.7.0.6:Data.Aeson.Types.Internal.Parser
Value
Actual type: (Integer
-> Const
(Data.Monoid.Endo
[aeson-0.7.0.6:Data.Aeson.Types.Internal.Parse
Integer)
-> Value
-> Const
(Data.Monoid.Endo
[aeson-0.7.0.6:Data.Aeson.Types.Internal.Parser
Value
In the second argument of ‘(.)’, namely ‘_Integer’
In the second argument of ‘(^..)’, namely ‘values . _Integer’
Is there a better way to do this?
Does anybody have an example of parsing arrays of values into a more detailed structure?
Aeson have instance for list, so I think it is not necessary to deal with vectors.
{-# LANGUAGE LambdaCase #-}
import Data.Aeson
data Series = Series [DataPoint]
data DataPoint = DataPoint Int Int
instance FromJSON DataPoint where
parseJSON jsn = do
[x,y] <- parseJSON jsn
return $ DataPoint x y
instance FromJSON Series where
parseJSON = \case
Object o -> (o .: "series") >>= fmap Series . parseJSON
x -> fail $ "unexpected json: " ++ show x
The trick here is getting the instance for FromJSON DataPoint correct, which takes a little bit of matching but isn't too bad. I came up with
instance FromJSON DataPoint where
parseJSON (Array v)
| V.length v == 2 = do
x <- parseJSON $ v V.! 0
y <- parseJSON $ v V.! 1
return $ DataPoint x y
| otherwise = mzero
parseJSON _ = mzero
Which will fail to parse cleanly if it isn't able to pull two Ints out for x and y. Then you just have to define the instance for Series:
instance FromJSON Series where
parseJSON (Object o) = do
pts <- o .: "series"
ptsList <- mapM parseJSON $ V.toList pts
return $ Series ptsList
parseJSON _ = mzero
Which, again, will cleanly fail if the data is malformed anywhere. To test:
> decode "{\"series\": [[1, 2], [3, 4]]}" :: Maybe Series
Just (Series [DataPoint 1 2, DataPoint 3 4])
> decode "{\"series\": [[1, 2], [3, {}]]}" :: Maybe Series
Nothing
So it looks like it works.
EDIT: As #maxtaldykin has pointed out, you can just take advantage of the FromJSON a => FromJSON [a] instance with
instance FromJSON DataPoint where
parseJSON obj = do
[x, y] <- parseJSON obj
return $ DataPoint x y
instance FromJSON Series where
parseJSON (Object o) = do
pts <- o .: "series"
fmap Series $ parseJSON pts
parseJSON _ = mzero
Which is greatly simplified from my original answer. Kudos to Max.
I'm trying to parse some bibliographic data, more specifically, pull out the 'subject' field for each item. The data is json and looks something like this:
{"rows": [
{"doc":{"sourceResource": {"subject": ["fiction", "horror"]}}},
{"doc":{"sourceResource": {"subject": "fantasy"}}}
]}
I can pull out 'subject' if every entry is either Text or [Text], but I'm stumped as to how to accommodate both. Here is my program in its current state:
{-# LANGUAGE OverloadedStrings#-}
import Debug.Trace
import Data.Typeable
import Data.Aeson
import Data.Text
import Control.Applicative
import Control.Monad
import qualified Data.ByteString.Lazy as B
import Network.HTTP.Conduit (simpleHttp)
import qualified Data.HashMap.Strict as HM
import qualified Data.Map as Map
jsonFile :: FilePath
jsonFile = "bib.json"
getJSON :: IO B.ByteString
getJSON = B.readFile jsonFile
data Document = Document { rows :: [Row]}
deriving (Eq, Show)
data Row = SubjectList [Text]
| SubjectText Text
deriving (Eq, Show)
instance FromJSON Document where
parseJSON (Object o) = do
rows <- parseJSON =<< (o .: "rows")
return $ Document rows
parseJSON _ = mzero
instance FromJSON Row where
parseJSON (Object o) = do
item <- parseJSON =<< ((o .: "doc") >>=
(.: "sourceResource") >>=
(.: "subject"))
-- return $ SubjectText item
return $ SubjectList item
parseJSON _ = mzero
main :: IO ()
main = do
d <- (decode <$> getJSON) :: IO (Maybe Document)
print d
Any help would be appreciated.
Edit:
the working FromJSON Row instance:
instance FromJSON Row where
parseJSON (Object o) =
(SubjectList <$> (parseJSON =<< val)) <|>
(SubjectText <$> (parseJSON =<< val))
where
val = ((o .: "doc") >>=
(.: "sourceResource") >>=
(.: "subject"))
parseJSON _ = mzero
First, look at the type of
((o .: "doc") >>=
(.: "sourceResource") >>=
(.: "subject")) :: FromJSON b => Parser b
We can get out of it anything that's an instance of FromJSON. Now, clearly, this can work for Text or [Text] individually, but your problem is that you want to get either Text or [Text]. Fortunately, it should be fairly easy to deal with this. Rather than letting it decode it for you further, just get a Value out of it. Once you've got a Value, you could decode it as a Text and put it in a SubjectText:
SubjectText <$> parseJSON val :: Parser Row
Or as a [Text] and put it in a SubjectList:
SubjectList <$> parseJSON val :: Parser Row
But wait, either one of these will do, and they have the same output type. Notice that Parser is an instance of Alternative, which lets us say exactly that (“either one will do”). Thus,
(SubjectList <$> parseJSON val) <|> (SubjectText <$> parseJSON val) :: Parser Row
Ta-da! (Actually, it wasn't necessary to pull it out as a Value; we could have instead embedded that long ((o .: "doc") >>= (.: "sourceResource") >>= (.: "subject")) chain into each subexpression. But that's ugly.)
I'm working on the I/O aspect of my json server, and there's a method I just can't get right.
First, I'll give the error, then the code and datatypes involved and some commentary about the problem afterwards.
("X-Response-Body-Start","<!DOCTYPE html>\n<html><head><title>Invalid Arguments</title></head><body><h1>Invalid Arguments</h1><ul><li>when expecting a unit constructor (U1), encountered String instead</li></ul></body></html>")
Expecting unit contructor?
Okay here's some relevant code. Let's see if we can see where I go wrong
from Datatypes.hs
data JobID = JobID Project Int deriving Generic
data Project = BNAP deriving (Show,Generic) -- one day to be an ADT
instance ToJSON Project where
toJSON = toJSON . show
instance FromJSON Project
instance FromJSON JobID
instance ToJSON JobID
The test code
testReadR :: IO Value
testReadR = do
req <- parseUrl readURI
manager <- newManager def
pBody <- runResourceT $ do
reqBody <- readObject
liftIO $ print reqBody
Response _ _ _ body <- http (buildReq req reqBody) manager
pBody <- body $$+- sinkParser json
return pBody -- (return wraps it up)
closeManager manager
return pBody
buildReq :: forall a (m :: * -> *) (t :: * -> *).
ToJSON a =>
Request t -> a -> Request m
buildReq req reqBody =
let reqBodyBS = Data.Aeson.encode reqBody
rHeaders = [(hContentType,pack "application/json")]
in req {method = fromString "POST"
, requestBody = RequestBodyLBS reqBodyBS
,requestHeaders=rHeaders
}
readObject :: ResourceT IO Value
readObject = do -- I took a bunch out because I thought simplifiying would help me
-- solve this
return $ Data.Aeson.toJSON $ JobID BNAP 306
The Handler
postReadR :: Handler RepJson
postReadR = do
conf <- parseJsonBody_ :: Handler JobID
liftIO $ print conf
testJ <- jsonToRepJson $ toJSON $ JobID BNAP 305
jValue <- jsonToRepJson conf -- to be replaced with
-- Either ErrorReport Response
-- (or something like that)
return jValue
when I change the line to
conf <- parseJsonBody_ :: Handler Value
print conf yields
Array (fromList [String "BNAP",Number 306])
So it seems the problem lies with String "BNAP" but I don't know why. Any ideas on how I can suss this out? Is there an obvious answer I'm not seeing?
Update : I have a new error. I'm sure I borked the FromJSON instance.
test: ResponseTimeout
instance FromJSON Project where
parseJSON (String p) = parseJSON $ toJSON p
parseJSON _ = mzero
The challenge here was that Project is a unary type. None of the examples I studied seemed to address that. But I know p is a Text, and toJSON can make a Value out of that, and parseJSON makes a Parser out of the Value. So it's all good right? Well, I still get the above error which is not informative at all. Any ideas?
instance ToJSON Project where
toJSON = toJSON . show
instance FromJSON Project
The instances don't work together. The generic FromJSON instance expects the generic unit constructor U1, but the ToJSON instance produces a String "BNAP".
If you have a hand-written ToJSON instance, you also need a hand-written FromJSON instance.
instance FromJSON Project where
parseJSON (String p) = parseJSON $ toJSON p
parseJSON _ = mzero
p is a Text, and we have
instance ToJSON Text where
toJSON = String
so the above instance for Project loops, since it expands parseJSON (String p) = parseJSON (String p).
For the type as it stands now,
instance FromJSON Project where
parseJSON (String "BNAP") = return BNAP
parseJSON _ = mzero
should work if you have OverloadedStrings enabled, if not,
instance FromJSON Project where
parseJSON (String p)
| p == pack "BNAP" = return BNAP
parseJSON _ = mzero