I'm using an external API which returns JSON responses. One of the responses is an array of objects and these objects are identified by the field value inside them. I'm having some trouble understanding how the parsing of such JSON response could be done with Aeson.
Here is a simplified version of my problem:
newtype Content = Content { content :: [Media] } deriving (Generic)
instance FromJSON Content
data Media =
Video { objectClass :: Text
, title :: Text } |
AudioBook { objectClass :: Text
, title :: Text }
In API documentation it is said that the object can be identified by the field objectClass which has value "video" for our Video object and "audiobook" for our AudioBook and so on. Example JSON:
[{objectClass: "video", title: "Some title"}
,{objectClass: "audiobook", title: "Other title"}]
The question is how can this type of JSON be approached using Aeson?
instance FromJSON Media where
parseJSON (Object x) = ???
You basically need a function Text -> Text -> Media:
toMedia :: Text -> Text -> Media
toMedia "video" = Video "video"
toMedia "audiobook" = AudioBook "audiobook"
The FromJSON instance is now really simple (using <$> and <*> from Control.Applicative):
instance FromJSON Media where
parseJSON (Object x) = toMedia <$> x .: "objectClass" <*> x .: "title"
However, at this point you're redundant: the objectClass field in Video or Audio doesn't give you more information than the actual type, so you might remove it:
data Media = Video { title :: Text }
| AudioBook { title :: Text }
toMedia :: Text -> Text -> Media
toMedia "video" = Video
toMedia "audiobook" = AudioBook
Also note that toMedia is partial. You probably want to catch invalid "objectClass" values:
instance FromJSON Media where
parseJSON (Object x) =
do oc <- x .: "objectClass"
case oc of
String "video" -> Video <$> x .: "title"
String "audiobook" -> AudioBook <$> x .: "title"
_ -> empty
{- an alternative using a proper toMedia
toMedia :: Alternative f => Text -> f (Text -> Media)
toMedia "video" = pure Video
toMedia "audiobook" = pure AudioBook
toMedia _ = empty
instance FromJSON Media where
parseJSON (Object x) = (x .: "objectClass" >>= toMedia) <*> x .: "title"
-}
And last, but not least, remember that valid JSON uses strings for the name.
The default translation for a data type like:
data Media = Video { title :: Text }
| AudioBook { title :: Text }
deriving Generic
is actually very close to what you want. (For the simplicity of my examples, I define ToJSON instances and encode the examples to see what kind of JSON we get.)
aeson, default
So, with the default instance we have (view the complete source file which produces this output):
[{"tag":"Video","title":"Some title"},{"tag":"AudioBook","title":"Other title"}]
Let's see whether we can get even closer with custom options...
aeson, custom tagFieldName
With custom options:
mediaJSONOptions :: Options
mediaJSONOptions =
defaultOptions{ sumEncoding =
TaggedObject{ tagFieldName = "objectClass"
-- , contentsFieldName = undefined
}
}
instance ToJSON Media
where toJSON = genericToJSON mediaJSONOptions
we get:
[{"objectClass":"Video","title":"Some title"},{"objectClass":"AudioBook","title":"Other title"}]
(Think yourself what you want to do with an undefined field in the real code.)
aeson, custom constructorTagModifier
Adding
, constructorTagModifier = fmap Char.toLower
to mediaJSONOptions gives:
[{"objectClass":"video","title":"Some title"},{"objectClass":"audiobook","title":"Other title"}]
Great! Exactly what you specified!
decoding
Simply add an instance with the same options to be able to decode from this format:
instance FromJSON Media
where parseJSON = genericParseJSON mediaJSONOptions
Example:
*Main> encode example
"[{\"objectClass\":\"video\",\"title\":\"Some title\"},{\"objectClass\":\"audiobook\",\"title\":\"Other title\"}]"
*Main> decode $ fromString "[{\"objectClass\":\"video\",\"title\":\"Some title\"},{\"objectClass\":\"audiobook\",\"title\":\"Other title\"}]" :: Maybe [Media]
Just [Video {title = "Some title"},AudioBook {title = "Other title"}]
*Main>
Complete source file.
generic-aeson, default
To get a more complete picture, let's also look at what generic-aeson package would give (at hackage). It has also nice default translations, different in some respects from those from aeson.
Doing
import Generics.Generic.Aeson -- from generic-aeson package
and defining:
instance ToJSON Media
where toJSON = gtoJson
gives the result:
[{"video":{"title":"Some title"}},{"audioBook":{"title":"Other title"}}]
So, it's different from all what we've seen when using aeson.
generic-aeson's options (Settings) are not interesting for us (they allow only to strip a prefix).
(The complete source file.)
aeson, ObjectWithSingleField
Apart from lower-casing the first letter of the constructor names, generic-aeson's translation seems similar to an option available in aeson:
Let's try this:
mediaJSONOptions =
defaultOptions{ sumEncoding = ObjectWithSingleField
, constructorTagModifier = fmap Char.toLower
}
and yes, the result is:
[{"video":{"title":"Some title"}},{"audiobook":{"title":"Other title"}}]
the rest of options: (aeson, TwoElemArray)
One available option for sumEncoding has been left out from consideration above, because it gives an array which is not quite similar to the JSON representation asked about. It's TwoElemArray. Example:
[["video",{"title":"Some title"}],["audiobook",{"title":"Other title"}]]
is given by:
mediaJSONOptions =
defaultOptions{ sumEncoding = TwoElemArray
, constructorTagModifier = fmap Char.toLower
}
Related
Say I have the following structure:
data AddressDto = AddressDto
{ addressDtoId :: UUID
, addressDtoCode :: Text
, addressDtoCity :: Maybe Text
, addressDtoStreet :: Text
, addressDtoPostCode :: Text
} deriving (Eq, Show, Generic)
instance FromJSON AddressDto where
parseJSON = genericParseJSON $ apiOptions "addressDto"
instance ToJSON AddressDto where
toJSON = genericToJSON $ apiOptions "addressDto"
toEncoding = genericToEncoding $ apiOptions "addressDto"
This works as you would expect.
Now say I want to parse a JSON structure of the format:
{ UUID: AddressDto, UUID: AddressDto, UUID: AddressDto }
A reasonable Haskell representation would seem to be:
data AddressListDto = AddressListDto [(UUID, AddressDto)]
Creating a helper function like so:
keyAndValueToList :: Either ServiceError AddressListDto -> Text -> DiscountDto -> Either ServiceError AddressListDto
keyAndValueToList (Left err) _ _ = Left err
keyAndValueToList (Right (AddressListDto ald)) k v = do
let maybeUUID = fromString $ toS k
case maybeUUID of
Nothing -> Left $ ParseError $ InvalidDiscountId k
Just validUUID -> Right $ AddressListDto $ (validUUID, v) : ald
and finally an instance of FromJSON:
instance FromJSON AddressListDto where
parseJSON = withObject "tuple" $ \o -> do
let result = foldlWithKey' keyAndValueToList (Right $ AddressListDto []) o
case result of
Right res -> pure res
Left err -> throwError err
This fails to compile with:
Couldn't match type ‘aeson-1.4.5.0:Data.Aeson.Types.Internal.Value’
with ‘AddressDto’
Expected type: unordered-containers-0.2.10.0:Data.HashMap.Base.HashMap
Text AddressDto
Two questions:
1) How do I make sure the nested values in the hashmap get parsed correctly to an AddressDto.
2) How do I avoid forcing the initial value into an either? Is there a function I could use instead of foldlWithKey' that didn't make me wrap the initial value like this?
Funny answer. To implement parseJSON, you can use
parseJSON :: Value -> Parser (HashMap Text AddressDto)
...but even better, why not just use a type alias instead of a fresh data type, so:
type AddressListDto = HashMap UUID AddressDto
This already has a suitable FromJSON instance, so then you don't even need to write any code yourself.
I recently started using Data.Aeson for one of my projects. And I am recently new to Haskell as well. So I am trying to figure out how the implementation of parseJSON function in FromJSON typeclass works.
So I have a code from my codebase.
data MyProfile = MyProfile { name :: String, age :: Int } deriving Show
instance FromJSON MyProfile where
parseJSON (Object m) = MyProfile <$>
m .: "name" <*>
m .: "age"
parseJSON x = fail ("not an object: " ++ show x)
And the YAML file I am trying to read is pretty simple as well.
profile:
name: "Foo"
age: 16
I am trying to understand the working of that applicative functor. I browsed through the Data.Aeson module and found that (.:) returns a Parser (FromJSON a).
So the facts that I have understood is,
Object m holds the profile: section of the yaml
MyProfile corresponds to profile
name in ParseJSON is trying to get the value for the key in the JSON object m
Similar with age as well
And each <*> returns a Parser (FromJSON) which is then applied over to the next <*>
What I am not understanding is,
How does MyProfile gets mapped to the profile section? What if I have a huge yaml file and multiple data defined in my program?
In the code MyProfile <$> m .: "name", shouldn't the first argument of <$> be a function? I perceive that <$> is similar to fmap and hence the first argument must be a function (which is applied to the second argument). But MyProfile is a data! Confusing!
How is the yaml values, in this case, Foo and 16, added to the MyProfile data?
Please correct me if any of my understandings is wrong.
I have been working on a small library against an JSON-based API. This library makes use of an "options" object, a series of key-value pairs that specify advanced behaviour:
{
"id": 1234
...
"options": {
"notify_no_data": True,
"no_data_timeframe": 20,
"notify_audit": False,
"silenced": {"*" :1428937807}
}
}
I've represented the options concept in Haskell using a list of options:
data Option = NotifyNoData NominalDiffTime -- notify after xyz
| NotifyAudit -- notify on changes
| Silenced (Maybe UTCTime) -- silence all notifications ("*") until xyz (or indefinitely)
newtype Options = Options [Option]
Implementing toJSON was simple enough; I instantiated toJSON for the Option type, and then used those as helpers for the Options type.
instance ToJSON Option where
toJSON (Silenced mtime) =
Object $ Data.HashMap.fromList [("silenced", mapping)]
where stamp = maybe Null (jsonTime . floor . utcTimeToPOSIXSeconds) mtime
mapping = Object $ Data.HashMap.singleton "*" stamp
toJSON (NotifyNoData difftime) =
Object $ Data.HashMap.fromList [("notify_no_data", Bool True)
,("no_data_timeframe", stamp)]
where stamp = jsonTime $ floor (difftime / 60)
toJSON NotifyAudit =
Object $ Data.HashMap.fromList [("notify_audit", Bool True)]
instance ToJSON Options where
toJSON (Options options) = Object $ Data.HashMap.unions $ reverse $ (opts:) $ map ((\(Object o) -> o) . toJSON) options
where opts = Data.HashMap.fromList [("silenced", Object Data.HashMap.empty)
,("notify_no_data", Bool False)
,("notify_audit", Bool False)]
The problem I am running into is in the fromJSON implementation for Options. All the use cases I've seen before supply a fairly simple json-object-representation to data-representation mapping. What I need to do is convert an object to an object of options to a list of data (Option) representations. For example, the sample JSON under "options" that I gave at the start would have to become:
Options [NotifyNoData 20, Silenced (Just (posixSecondsToUTCTime 1428937807))]
FromJSON requires an implementation of parseJSON :: FromJSON a => Value -> Parser a. I am having trouble understanding how to build a Parser using this optional object structure given by the API. Is there a standard approach to parsing a JSON object to a list like this? It may be that I am failing to fully understand the Parser typeclass.
You can maybe use the parser monad to extract all the information, something like:
parseJSON (Object v) = do
maybeNotify <- v .:? "notify_no_data"
maybeTimeFrame <- v .:? "no_data_timeframe"
let nots = case (maybeNotify,maybeTimeFrame) of
(Just True,Just stamp) -> [NotifyNoData $ fromJsontime stamp]
_ -> []
return $ Options nots
(fromJsontime is just a helper function to convert from the JSON value to your NominalDiffTime, I suppose you have something for that).
Do the same for the other types of options, concatenating the result.
Using the json-stream parser (which I am the author of), it would look like this (from my head, untested):
option = NotifyNoData <$> "no_data_timeframe" .: value
<* filterI id ("notify_no_data" .: bool)
<|> const NotifyAudit <$> filterI id ("notify_audit" .: bool)
<|> Silenced <$> "silenced" .: "*" .:? value
optionList = Options <$> toList option
The code expects you have FromJSON instance for UTCTime (or you could just use 'integer' and some kind of fromposixtime etc.).
I'm writing a JSON service for JIRA, and I've come across a requirement that conflicts with Haskell's namespace.
I have this record
data Assignee = Assignee {name :: Text} deriving Generic
instance ToJSON Assignee
This is dictated by what JIRA wants, unfortunately it wants the same field for a different object.
data Reporter = Reporter {name :: Text} deriving Generic
instance ToJSON Reporter
I see a few options:
Maybe I can circumvent the compiler's complaining with template Haskell, but how?
I could simply not have a Reporter record, and change the reporter field with a seperate service after the ticket has been created. That I know how to do, but is it the best way?
Create the JSON object by hand, but I form it from this record:
data Fields = Fields
{ project :: HashMap Key Project
, summary :: Text
, issuetype :: HashMap Name Task
, versions :: [HashMap Name Text]
, description :: Text
, assignee :: Assignee
} deriving (Generic)
The thought of making this by hand gives me the wiggins. If I must I will.
So, my question now is, if there is no other better way than the ones I've presented, which of these is the best course of action?
The most straightforward way is to enable the -XDisambiguateRecordFields extension.
If DisambiguateRecordFields and/or keeping the records in separate modules works for you, that's excellent.
If not, then a common way to work around this problem is to prefix record field labels in some way, to disambiguate:
data Assignee = Assignee {assigneeName :: Text} deriving Generic
data Reporter = Reporter {reporterName :: Text} deriving Generic
You can still use GHC Generics to derive the JSON translation functions, but you have to configure it so that the field labels are changed, for example like this:
stripPrefix :: Eq a => [a] -> [a] -> [a]
stripPrefix p x = case splitAt (length p) x of
(y, z)
| y == p -> z
| otherwise -> x
lower :: String -> String
lower [] = []
lower (x : xs) = toLower x : xs
stripPrefixOptions :: String -> Options
stripPrefixOptions p = defaultOptions {
fieldLabelModifier = lower . stripPrefix p
}
Then you can say:
data Assignee = Assignee {assigneeName :: Text} deriving Generic
instance ToJSON Assignee where
toJSON = genericToJSON (stripPrefixOptions "assignee")
data Reporter = Reporter {reporterName :: Text} deriving Generic
instance ToJSON Reporter where
toJSON = genericToJSON (stripPrefixOptions "reporter")
Testing in GHCi:
GHCi> > encode (Assignee { assigneeName = "foo" })
"{\"name\":\"foo\"}"
I'm struggling to understand this (I'm still a bit new to Haskell) but I'm finding the documentation for the Text.JSON package to be a little confusing. Basically I have this data record type: -
data Tweet = Tweet
{
from_user :: String,
to_user_id :: String,
profile_image_url :: String,
created_at :: String,
id_str :: String,
source :: String,
to_user_id_str :: String,
from_user_id_str :: String,
from_user_id :: String,
text :: String,
metadata :: String
}
and I have some tweets in JSON format that conform to the structure of this type. The thing that I'm struggling with is how to map the above to what gets returned from the following code
decode tweet :: Result JSValue
into the above datatype. I understand that I'm supposed to create an instance of instance JSON Tweet but I don't know where to go from there.
Any pointers would be greatly appreciated, thanks!
I'd recommend that you use the new aeson package instead of the json package, as the former performs much better. Here's how you'd convert a JSON object to a Haskell record, using aeson:
{-# LANGUAGE OverloadedStrings #-}
module Example where
import Control.Applicative
import Control.Monad
import Data.Aeson
data Tweet = Tweet {
from_user :: String,
to_user_id :: String,
profile_image_url :: String,
created_at :: String,
id_str :: String,
source :: String,
to_user_id_str :: String,
from_user_id_str :: String,
from_user_id :: String,
text :: String,
metadata :: String
}
instance FromJSON Tweet where
parseJSON (Object v) =
Tweet <$> v .: "from_user"
<*> v .: "to_user_id"
<*> v .: "profile_image_url"
<*> v .: "created_at"
<*> v .: "id_str"
<*> v .: "source"
<*> v .: "to_user_id_str"
<*> v .: "from_user_id_str"
<*> v .: "from_user_id"
<*> v .: "text"
<*> v .: "metadata"
-- A non-Object value is of the wrong type, so use mzero to fail.
parseJSON _ = mzero
Then use Data.Aeson.json to get a attoparsec parser that converts a ByteString into a Value. The call fromJSON on the Value to attempt to parse it into your record. Note that there are two different parsers involved in these two steps, a Data.Attoparsec.Parser parser for converting the ByteString into a generic JSON Value and then a Data.Aeson.Types.Parser parser for converting the JSON value into a record. Note that both steps can fail:
The first parser can fail if the ByteString isn't a valid JSON value.
The second parser can fail if the (valid) JSON value doesn't contain one of the fields you mentioned in your fromJSON implementation.
The aeson package prefers the new Unicode type Text (defined in the text package) to the more old school String type. The Text type has a much more memory efficient representation than String and generally performs better. I'd recommend that you change the Tweet type to use Text instead of String.
If you ever need to convert between String and Text, use the pack and unpack functions defined in Data.Text. Note that such conversions require O(n) time, so avoid them as much as possible (i.e. always use Text).
You need to write a showJSON and readJSON method, for your type, that builds your Haskell values out of the JSON format. The JSON package will take care of parsing the raw string into a JSValue for you.
Your tweet will be a JSObject containing a map of strings, most likely.
Use show to look at the JSObject, to see how the fields are laid out.
You can lookup each field using get_field on the JSObject.
You can use fromJSString to get a regular Haskell strings from a JSString.
Broadly, you'll need something like,
{-# LANGUAGE RecordWildCards #-}
import Text.JSON
import Text.JSON.Types
instance JSON Tweet where
readJSON (JSObject o) = return $ Tweet { .. }
where from_user = grab o "from_user"
to_user_id = grab o "to_user_id"
profile_image_url = grab o "proile_image_url"
created_at = grab o "created_at"
id_str = grab o "id_str"
source = grab o "source"
to_user_id_str = grab o "to_user_id_str"
from_user_id_str = grab o "from_user_id_str"
from_user_id = grab o "from_user_id"
text = grab o "text"
metadata = grab o "metadata"
grab o s = case get_field o s of
Nothing -> error "Invalid field " ++ show s
Just (JSString s') -> fromJSString s'
Note, I'm using the rather cool wild cards language extension.
Without an example of the JSON encoding, there's not much more I can advise.
Related
You can find example instances for the JSON encoding via instances
in the source, for
simple types. Or in other packages that depend on json.
An instance for AUR messages is here, as a (low level) example.
Import Data.JSon.Generic and Data.Data, then add deriving (Data) to your record type, and then try using decodeJSON on the tweet.
I support the answer by #tibbe.
However, I would like to add How you check put some default value in case, the argument misses in the JSON provided.
In tibbe's answer you can do the following:
Tweet <$> v .: "from_user"
<*> v .:? "to_user_id" .!= "some user here"
<*> v .: "profile_image_url" .!= "url to image"
<*> v .: "created_at"
<*> v .: "id_str" != 232131
<*> v .: "source"
this will the dafault parameters to be taken while parsing the JSON.