Aeson: derive some (but not all) fields of a struct - json

I have a large struct which I need to be an instance of FromJSON so that I can parse my json data into it.
I would like to derive automatically, but a single field needs "special care" in that it is an object in json and I want it to be an array of the values in my struct. How can I do this without writing a huge FromJson implementation repeating all the fields?
Example json:
{"myobject": {"one": 1, "two": 2}, ...many_more_fields...}
Example struct:
data MyStruct = MyStruct {
myobject :: [Int],
...many_more_fields,...
} deriving (Generic)
How do I do this elegantly?

You should create a newtype for your special field:
newtype MySpecialType = MySpecialType [Int]
instance FromJSON MySpecialType where ....
data MyStruct = MyStruct {
myobject:: MySpecialType,
...
}
Now the instance for MyStruct becomes entirely regular and can be handed off to Template Haskell in the normal way.

To avoid carrying the newtype from Paul Johnson's very good answer all across the codebase, you can also generalize your type as follows, making the type of myobject a parameter:
data MyStruct_ intList = MyStruct {
myobject :: intlist,
...
} deriving (Functor, Generic)
type MyStruct = MyStruct [Int]
instance FromJSON MyStruct where
parseJSON = (fmap . fmap) (\(MySpecialType i) -> i)
. genericParseJSON defaultOptions
genericParseJSON above gets instantiated with MyStruct MySpecialType, and then the field gets unwrapped via fmap (noting MyStruct_ is a Functor)
I also just wrote a blogpost about "type surgery", applied to this kind of problem so that you can keep the original type unmodified.
The generic-data-surgery library can derive a generic type with the same Generic structure as MyStruct_ MySpecialType above, to be used by aeson's genericParseJSON. The surgery modifyRField then applies the function \(MySpecialType i) -> i to the myobject field, finally yielding MyStruct.
import Generic.Data.Surgery (fromOR, toOR', modifyRField)
-- The original type
data MyStruct = MyStruct {
myobject :: [Int],
...
} deriving (Generic)
instance FromJSON MyStruct where
parseJSON = fmap (fromOR . modifyRField #"myobject" (\(MySpecialType i) -> i) . toOR')
. genericParseJSON defaultOptions

Related

reading Csv as custom data types

I have a csv at some filePath with two columns with no headers
john,304
sarah,300
...
I have been able to read the csv as such:
import Data.Csv as Csv
import Data.ByteString.Lazy as BL
import Data.Vector as V
...
results <- fmap V.toList . Csv.decode #(String,Integer) Csv.NoHeader <$> BL.readFile filePath
-- Right [("john",300),("sarah",302)]
If I have custom data type for the csv columns as such:
data PersonnelData = PersonnelData
{ name :: !String
, amount :: !Integer
} deriving (Show, Generic, Csv.FromRecord)
How can I modify the above to decode / read the file for this data type?
Where you have this:
Csv.decode #(String,Integer)
You are are using a visible type application to explicitly tell Csv.decode that its first type parameter should be the type (String, Integer). Let's have a look at the signature for decode to see what that means:
decode :: FromRecord a  
=> HasHeader
-> ByteString
-> Either String (Vector a)
There's only one type parameter, and it's pretty clear from where it appears (in the output) that it is the type that decode is decoding into. So Csv.decode #(String,Integer) is a function that explicitly decodes CSV records to (String, Integer).
So the only change you need to make to your code is to explicitly tell it you want to decode to PersonnelData instead of (String, Integer). Just use Csv.decode #PersonnelData. (You need a FromRecord instance, but you already have provided that by deriving it)

Can aeson handle JSON with imprecise types?

I have to deal with JSON from a service that sometimes gives me "123" instead of 123 as the value of field. Of course this is ugly, but I cannot change the service. Is there an easy way to derive an instance of FromJSON that can handle this? The standard instances derived by means of deriveJSON (https://hackage.haskell.org/package/aeson-1.5.4.1/docs/Data-Aeson-TH.html) cannot do that.
One low-hanging (although perhaps not so elegant) option is to define the property as an Aeson Value. Here's an example:
{-#LANGUAGE DeriveGeneric #-}
module Q65410397 where
import GHC.Generics
import Data.Aeson
data JExample = JExample { jproperty :: Value } deriving (Eq, Show, Generic)
instance ToJSON JExample where
instance FromJSON JExample where
Aeson can decode a JSON value with a number:
*Q65410397> decode "{\"jproperty\":123}" :: Maybe JExample
Just (JExample {jproperty = Number 123.0})
It also works if the value is a string:
*Q65410397> decode "{\"jproperty\":\"123\"}" :: Maybe JExample
Just (JExample {jproperty = String "123"})
Granted, by defining the property as Value this means that at the Haskell side, it could also hold arrays and other objects, so you should at least have a path in your code that handles that. If you're absolutely sure that the third-party service will never give you, say, an array in that place, then the above isn't the most elegant solution.
On the other hand, if it gives you both 123 and "123", there's already some evidence that maybe you shouldn't trust the contract to be well-typed...
Assuming you want to avoid writing FromJSON instances by hand as much as possible, perhaps you could define a newtype over Int with a hand-crafted FromJSON instance—just for handling that oddly parsed field:
{-# LANGUAGE TypeApplications #-}
import Control.Applicative
import Data.Aeson
import Data.Text
import Data.Text.Read (decimal)
newtype SpecialInt = SpecialInt { getSpecialInt :: Int } deriving (Show, Eq, Ord)
instance FromJSON SpecialInt where
parseJSON v =
let fromInt = parseJSON #Int v
fromStr = do
str <- parseJSON #Text v
case decimal str of
Right (i, _) -> pure i
Left errmsg -> fail errmsg
in SpecialInt <$> (fromInt <|> fromStr)
You could then derive FromJSON for records which have a SpecialInt as a field.
Making the field a SpecialInt instead of an Int only for the sake of the FromJSON instance feels a bit intrusive though. "Needs to be parsed in an odd way" is a property of the external format, not of the domain.
In order to avoid this awkwardness and keep our domain types clean, we need a way to tell GHC: "hey, when deriving the FromJSON instance for my domain type, please treat this field as if it were a SpecialInt, but return an Int at the end". That is, we want to deal with SpecialInt only when deserializing. This can be done using the "generic-data-surgery" library.
Consider this type
{-# LANGUAGE DeriveGeneric #-}
import GHC.Generics
data User = User { name :: String, age :: Int } deriving (Show,Generic)
and imagine we want to parse "age" as if it were a SpecialInt. We can do it like this:
{-# LANGUAGE DataKinds #-}
import Generic.Data.Surgery (toOR', modifyRField, fromOR, Data)
instance FromJSON User where
parseJSON v = do
r <- genericParseJSON defaultOptions v
-- r is a synthetic Data which we must tweak in the OR and convert to User
let surgery = fromOR . modifyRField #"age" #1 getSpecialInt . toOR'
pure (surgery r)
Putting it to work:
{-# LANGUAGE OverloadedStrings #-}
main :: IO ()
main = do
print $ eitherDecode' #User $ "{ \"name\" : \"John\", \"age\" : \"123\" }"
print $ eitherDecode' #User $ "{ \"name\" : \"John\", \"age\" : 123 }"
One limitation is that "generic-data-surgery" works by tweaking Generic representations, so this technique won't work with deserializers generated using Template Haskell.

Use of typeclasses in the JSON example form Real World Haskell

I have found Real World Haskell - Chapter 5 mainly confusing, and it seems it is stinging me also in Chapter 6.
In Chapter 6, up to just before Typeclasses at work: making JSON easier to use, everything seems clear to me; then the book shows a portion of a JSON file, as well as a Haskell source that defines a variable result holding (most of) that JSON example.
Then, with reference to the preceding chunk of code, it makes the difference between JSON objects, which can contain elements of different types, and Haskell lists, which cannot. This justifies the use of JValue's constructors (JNumber, JBool, ...) in the aforementioned code.
So far so good. Then it starts becoming confusing form me.
This limits our flexibility: if we want to change the number 3920 to a string "3,920", we must change the constructor that we use to wrap it from JNumber to JString.
Yeah, so what? If my intention is to make this change, I will have to change, for instance, this line
("esitmatedCount", JNumber 3920)
to this
("esitmatedCount", JString "3,920")
which corresponds to changing 3920 to "3,920" in the actual JSON file. So what? If I had the chance of putting different types in a Haskell list, I would still have to wrap the number in the double quotes and add the comma.
I don't get where the loss of flexibility is.
Then a tempting solution is proposed (tempting? Than it is no good... Where's the drawback? Some comments in the online book pose the same question.)
type JSONError = String
class JSON a where
toJValue :: a -> JValue
fromJValue :: JValue -> Either JSONError a
instance JSON JValue where
toJValue = id
fromJValue = Right
Now, instead of applying a constructor like JNumber to a value to wrap it, we apply the toJValue function. If we change a value's type, the compiler will choose a suitable implementation of toJValue to use with it.
This makes me thing that the intention is to use the toJValue function instead of the constructor JNumber, but I don't see how toJValue 3920 could work.
How should I make use of the code above?
Update: I added answers to your follow-up comments in a section at the end.
I think the authors, in writing the type classes chapter, wanted to maintain continuity with the example from the previous chapter. They may also have had in mind some real code they'd written where type classes were used for working with JSON in Haskell. (I see that Bryan O'Sullivan, one of the authors of RWH, is also the author of the excellent aeson JSON parsing library which uses type classes extremely effectively.) I think that they were also a little frustrated that their best example of the need for type classes (BasicEq) was something that was already implemented, which forced readers to pretend that the language designers had left a key feature out of the language in order to see the need for type classes. They also realized that the JSON example was rich and complex enough to allow them to introduce some difficult new concepts (type synonym and overlapping instances, the open world assumption, newtype wrappers, etc.).
So, they tried to add the JSON example as a realistic, reasonably complex example that related back to the earlier material and could be used for pedagogical purposes to introduce a bunch of new material.
Unfortunately, they realized a little too late that the motivation for the example was weak, at least without introducing a bunch of advanced new concepts and techniques. So, they mumbled something about "lack of flexibility", forged ahead anyway, and let the example peter out at the end, never really getting back to how someone would use toJValue or fromJValue for anything.
Here's a demonstration of why the JSON class is useful, motivated by the aeson package. Note that it uses several more advanced features that haven't been covered by the first five chapters of RWH, so you might not be able to follow it all yet.
So we're on the same page, suppose we have the following code, a slightly simplified version of the type class and instances from Chapter 6. There are a few extra the language extensions here that are needed for the code below.
{-# LANGUAGE TypeSynonymInstances, FlexibleInstances, RankNTypes, RecordWildCards #-}
module JSONClass where
data JValue = JString String
| JNumber Double
| JBool Bool
| JNull
| JObject [(String, JValue)]
| JArray [JValue]
deriving (Show)
class JSON a where
toJValue :: a -> JValue
fromJValue :: JValue -> Maybe a
instance JSON Bool where
toJValue = JBool
fromJValue (JBool b) = Just b
fromJValue _ = Nothing
instance {-# OVERLAPPING #-} JSON String where
toJValue = JString
fromJValue (JString s) = Just s
fromJValue _ = Nothing
instance JSON Double where
toJValue = JNumber
fromJValue (JNumber x) = Just x
fromJValue _ = Nothing
instance {-# OVERLAPPABLE #-} (JSON a) => JSON [a] where
toJValue = JArray . map toJValue
fromJValue (JArray vals) = mapM fromJValue vals
fromJValue _ = Nothing
Also suppose that we have some Haskell data types representing search results, patterned after the result example given in the "Typeclasses at work" section:
data Search = Search
{ query :: String
, estimatedCount :: Double
, moreResults :: Bool
, results :: [Result]
} deriving (Show)
data Result = Result
{ title :: String
, snippet :: String
, url :: String
} deriving (Show)
It would be nice to convert these to JSON. Using the RecordWildCards extension which "blows up" the fields of a parameter into separate variables, we can write this quite cleanly:
resultToJValue :: Result -> JValue
resultToJValue Result{..}
= JObject [("title", JString title), ("snippet", JString snippet), ("url", JString url)]
searchToJValue :: Search -> JValue
searchToJValue Search{..}
= JObject [("query", JString query),
("estimatedCount", JNumber estimatedCount),
("moreResults", JBool moreResults),
("results", JArray $ map resultToJValue results)]
It's a little cluttered with constructors. We could "simplify" this by replacing some of the constructors with toJValue which would give us:
resultToJValue :: Result -> JValue
resultToJValue Result{..}
= JObject [("title", toJValue title), ("snippet", toJValue snippet),
("url", toJValue url)]
searchToJValue :: Search -> JValue
searchToJValue Search{..}
= JObject [("query", toJValue query),
("estimatedCount", toJValue estimatedCount),
("moreResults", toJValue moreResults),
("results", JArray $ map resultToJValue results)]
You could easily argue that's really no less cluttered. However, the type class allows us to define a helper function:
(.=) :: (JSON a) => String -> a -> (String, JValue)
infix 0 .=
k .= v = (k, toJValue v)
which introduces a nice, clean syntax:
resultToJValue :: Result -> JValue
resultToJValue Result{..}
= JObject [ "title" .= title
, "snippet" .= snippet
, "url" .= url ]
searchToJValue :: Search -> JValue
searchToJValue Search{..}
= JObject [ "query" .= query
, "estimatedCount" .= estimatedCount
, "moreResults" .= moreResults
, ("results", JArray $ map resultToJValue results)]
and the only reason the last line looks ugly is that we didn't give Result its JSON instance:
instance JSON Result where
toJValue = resultToJValue
which would allow us to write:
searchToJValue :: Search -> JValue
searchToJValue Search{..}
= JObject [ "query" .= query
, "estimatedCount" .= estimatedCount
, "moreResults" .= moreResults
, "results" .= results ]
In fact, we don't need the functions resultToJValue and searchToJValue at all, as their definitions can just be given directly in the instances. So, all the code above after the definitions of the Search and Result data types can be collapsed down to:
(.=) :: (JSON a) => String -> a -> (String, JValue)
infix 0 .=
k .= v = (k, toJValue v)
instance JSON Result where
toJValue Result{..}
= JObject [ "title" .= title
, "snippet" .= snippet
, "url" .= url ]
instance JSON Search where
toJValue Search{..}
= JObject [ "query" .= query
, "estimatedCount" .= estimatedCount
, "moreResults" .= moreResults
, "results" .= results ]
which provides support for:
search = Search "awkward squad haskell" 3920 True
[ Result "Simon Peyton Jones: papers"
"Tackling the awkward squad..."
"http://..."
]
main = print (toJValue search)
What about converting a JSON JValue back to Result and Search? You might want to try writing this up without using the type classes and see what it looks like. The solution with type classes uses a mind-bending helper function (that requires the RankNTypes language extension):
withObj :: (JSON a) => JValue ->
((forall v. JSON v => String -> Maybe v) -> Maybe a) -> Maybe a
withObj (JObject lst) template = template v
where v k = fromJValue =<< lookup k lst
after which the instances are easily written using applicative syntax (<$> and <*>) which allow us to combine a bunch of Maybe values as paramters to a function call, returning Nothing if any of the parameters are Nothing (i.e., unexpected type in the JSON) and calling the function otherwise:
instance JSON Result where
fromJValue o = withObj o $ \v -> Result <$> v "title" <*> v "snippet" <*> v "url"
instance JSON Search where
fromJValue o = withObj o $ \v -> Search <$> v "query" <*> v "estimatedCount"
<*> v "moreResults" <*> v "results"
Without type classes, this sort of uniform treatment of different field types using the helper functions (.=) and withObj wouldn't have been possible and the final syntax for writing these marshalling functions would have been substantially more complicated.
This example couldn't have been introduced as-is in RWH Chapter 6, as it involves applicatives (the <*> syntax), higher rank types (withObj), and probably a bunch of other things I've forgotten about. I'm not sure if it could be simplified enough to make the final syntax look nice enough that the advantages of using type classes would be clear.
Anyway, here's the full code. You might want to browse the documentation for the aeson package to see what a real library based on this approach would look like.
{-# LANGUAGE TypeSynonymInstances, FlexibleInstances, RankNTypes, RecordWildCards #-}
module JSONClass where
-- JSON type
data JValue = JString String
| JNumber Double
| JBool Bool
| JNull
| JObject [(String, JValue)]
| JArray [JValue]
deriving (Show)
-- Type classes and instances
class JSON a where
toJValue :: a -> JValue
fromJValue :: JValue -> Maybe a
instance JSON Bool where
toJValue = JBool
fromJValue (JBool b) = Just b
fromJValue _ = Nothing
instance {-# OVERLAPPING #-} JSON String where
toJValue = JString
fromJValue (JString s) = Just s
fromJValue _ = Nothing
instance JSON Double where
toJValue = JNumber
fromJValue (JNumber x) = Just x
fromJValue _ = Nothing
instance {-# OVERLAPPABLE #-} (JSON a) => JSON [a] where
toJValue = JArray . map toJValue
fromJValue (JArray vals) = mapM fromJValue vals
fromJValue _ = Nothing
-- helpers
(.=) :: (JSON a) => String -> a -> (String, JValue)
infix 0 .=
k .= v = (k, toJValue v)
withObj :: (JSON a) => JValue ->
((forall v. JSON v => String -> Maybe v) -> Maybe a) -> Maybe a
withObj (JObject lst) template = template v
where v k = fromJValue =<< lookup k lst
-- our new data types
data Search = Search
{ query :: String
, estimatedCount :: Double
, moreResults :: Bool
, results :: [Result]
} deriving (Show)
data Result = Result
{ title :: String
, snippet :: String
, url :: String
} deriving (Show)
-- JSON instances to marshall them in and out of JValues
instance JSON Result where
toJValue Result{..}
= JObject [ "title" .= title
, "snippet" .= snippet
, "url" .= url ]
fromJValue o = withObj o $ \v -> Result <$> v "title" <*> v "snippet" <*> v "url"
instance JSON Search where
toJValue Search{..}
= JObject [ "query" .= query
, "estimatedCount" .= estimatedCount
, "moreResults" .= moreResults
, "results" .= results ]
fromJValue o = withObj o $ \v -> Search <$> v "query" <*> v "estimatedCount"
<*> v "moreResults" <*> v "results"
-- a test
search :: Search
search = Search "awkward squad haskell" 3920 True
[ Result "Simon Peyton Jones: papers"
"Tackling the awkward squad..."
"http://..."
]
main :: IO ()
main = do
let jsonSearch = toJValue search
print jsonSearch
let search' = fromJValue jsonSearch :: Maybe Search
print search'
Answers to Questions from the Comments
You asked a bunch of follow-up questions in the comments. I've tried to answer them here, in slightly different order:
Q: The book uses Either and you use Maybe. I would say that's just because you are using Nothing to say something went wrong and the book suggests using an explanatory String to give details about the wrong. Ok, but the book's definition of toJValue and fromJValue are so different from yours: I can't see how toJValue = id can be useful as the type of input and output cannot be different, based on id's signature; and fromJValue, given any JValue returns Right that JValue, whereas you deconstruct to return the Haskell type wrapped in it.
A: Yes, I used Maybe instead of Either to signal errors, and only because I thought it made my example a little bit simpler. The book talks about this in the "More helpful errors" section, noting that Maybe could have been used, too, but Either allows more helpful error messages to be provided: the book's Left is like my Nothing but with an additional explanatory note.
Maybe my plan to simplify things backfired because my version is supposed to look similar to the book's version. I think you're just comparing the wrong instances. Consider the class definitions first:
-- from book
class JSON a where
toJValue :: a -> JValue
fromJValue :: JValue -> Either JSONError a
-- mine
class JSON a where
toJValue :: a -> JValue
fromJValue :: JValue -> Maybe a
The only intended difference here is that fromJValue can return Left errmsg or Right answer in the book's version versus Nothing or Just answer in my version. For a particular instance, like the Bool instance, we have:
-- from book
instance JSON Bool where
toJValue = JBool
fromJValue (JBool b) = Right b
fromJValue _ = Left "not a JSON boolean"
-- mine
instance JSON Bool where
toJValue = JBool
fromJValue (JBool b) = Just b
fromJValue _ = Nothing
Again, these match except that Right becomes Just and Left "message" becomes Nothing. I think all that's throwing you off is that the book defines this extra instance for the JValue type:
instance JSON JValue where
toJValue = id
fromJValue = Right
that I decided not to define because I didn't need it for anything. This instance is weird and different from all the other instances. All the other instances involve translating other Haskell types to and from a corresponding JValue representation. This instance "translates" a JValue to and from itself. So, toJValue is just id because no transformation is actually needed. For fromJValue, we'd be tempted to use id, too, but the general fromJValue function is allowed to fail by returning Left errmsg (or Nothing, for my version) if the types don't match. But, a JValue is always the right type to be "translated" to a JValue, so we can always return a Right answer. My version of this instance would look like:
instance JSON JValue where
toJValue = id
fromJValue = Just -- use Just instead of Right; we never return Nothing
Q: Also, you put fromJValue _ = Nothing in every instance, whereas the book doesn't even mention Left, not before "More Helpful Errors". Maybe I have to keep reading a bit, as understanding half of your answer has hopefully unlocked something in my understanding.
A: Well, the book only introduces one instance before the "More helpful errors" section, and it's that instance JSON JValue. My version wouldn't have needed Nothing, the same way the book's didn't need Left. As soon as we start defining instances that can fail, then we need either Left or Nothing.
Q: The other concerns mapM: why is that used? I understand that vals is a valid list in input to the JArray constructor (otherwise we couldn't be on the lines which deconstructs it, right?), so applying fromJValue to each element of the list should return a Maybe (all Just, right?) just as the signature of the function requires.
A: Yes, you're correct. You're just missing one step. To be concrete, let's say we're trying to read a Haskell list of doubles ([Double]) from a JSON value like:
JArray [JNumber 1.0, JNumber 2.0, JNumber 3.0]
so we have vals = [JNumber 1.0, JNumber 2.0, JNumber 3.0]. As you say, we want to apply fromJValue to each element of this list. If we did that using map, like:
map fromJValue vals
we'd get:
[Just 1.0, Just 2.0, Just 3.0] :: [Maybe Double]
but that return value doesn't actually match the type signature. That's a [Maybe Double] value, but we want a Maybe [Double] value, more like:
Just [1.0, 2.0, 3.0] :: Maybe [Double]
The purpose of mapM here is to pull that "Just" outside of the list. It serves a second purpose, too. If we were trying to read a list like:
[JNumber 1.0, JNumber 2.0, JString "three point zero"]
then applying map fromJValue vals (in a context where fromJValue has been specialized to the Double instance) would give:
[Just 1.0, Just 2.0, Nothing] :: [Maybe Double]
Here, even though some of the elements were successfully transformed to doubles, some couldn't be, so we actually want to indicate overall failure by transforming that whole thing into the final result:
Nothing :: Maybe [Double]
The mapM function is a general monadic map, but for the specific monad I'm using (the Maybe monad), it has signature:
mapM :: (a -> Maybe b) -> [a] -> Maybe [b]
It's best understood as using a function a -> Maybe b that can "succeed" by returning Just or "fail" by returning Nothing, and applying that function to a list [a]. If all applications succeed, it returns Just the list of the results (pulling the Just outside the list); if any fail, it returns a global Nothing failure value.
It's actually the same idea as the function mapEithers in RWH. That function applies a function to the list [a], and if they all succeed (by returning Rights), it returns Right the list of the results (pulling the Right outside the list); if any fail (by returning Left), it returns a Left failure value (using the "first" error message encountered, if multiple errors are generated). In fact, mapEithers didn't need to be defined. It could have been replaced with mapM, because mapM works with both the Maybe monad and the Either errmsg monads and has the same behavior as mapEithers for the Either errmsg monad.
Q: One concerns JNull, which is the only constructor not taking any parameters, so there's nothing to deconstruct and, therefore, no type that could be made an instance of JSON. How does this fit the overall picture?
A: The most literal way of translating JNull would be to translate it into a Haskell type that contains no information. There actually is such a type. It's named "unit" and written () in source code. You've probably seen it used in various contexts. The appropriate instance would be:
instance JSON () where
toJValue () = JNull
fromJValue JNull = Just ()
fromJValue _ = Nothing
I didn't include this instance because it's pretty useless. It would only be helpful for translating to and from some JSON where a particular field always had the explicit value null, but that's not the way null is used in JSON.
At the risk of complicating things further, here is a potential use for JNull. Suppose I wanted to add an optional field to my Result type, for the URL of a "favorites icon" or something.
data Result = Result
{ title :: String
, snippet :: String
, url :: String
, favicon :: Maybe String -- new, optional field
} deriving (Show)
Now, my beautiful instance JSON Result is broken, because I don't have a way of handling a Maybe String field. But, I can introduce a new instance to handle Maybe values that uses JNull as the equivalent of Nothing:
instance JSON a => JSON (Maybe a) where
toJValue Nothing = JNull
toJValue (Just x) = toJValue x
fromJValue JNull = Just Nothing
fromJValue x = Just <$> fromJValue x
There are many complicated things going on here, and I won't try to explain them all. However, it may help to understand that the return value of fromJValue is the rather weird type Maybe (Maybe a) where the two Maybes serve different purposes: the outer Maybe says whether the transformation succeeded, and the inner Maybe says whether the optional value was available or missing. So Just Nothing is a value representing a successful transformation to a missing value!
The point of that instance is that we can update our Result instance to include the new field:
instance JSON Result where
toJValue Result{..}
= JObject [ "title" .= title
, "snippet" .= snippet
, "url" .= url
, "favicon" .= favicon ]
fromJValue o = withObj o $ \v -> Result <$> v "title" <*> v "snippet"
<*> v "url" <*> v "favicon"
and the instance can now (sort of) handle an optional value:
> toJValue (Result "mytitle" "mysnippet" "myurl" Nothing)
JObject [("title",JString "mytitle"),("snippet",JString "mysnippet"),("url",JString "myurl"),("favicon",JNull)]
> toJValue (Result "mytitle" "mysnippet" "myurl" (Just "myfavicon"))
JObject [("title",JString "mytitle"),("snippet",JString "mysnippet"),("url",JString "myurl"),("favicon",JString "myfavicon")]
though in generating real JSON, you probably would leave out any fields that looked like { ..., favicon: null, ... }, so you'd want to introduce some filtering to remove null fields from the final JSON value. Also, fromJValue doesn't actually handle a truly missing optional field:
> fromJValue (JObject [("title",JString "mytitle"),("snippet",JString "mysnippet"),("url",JString "myurl")]) :: Maybe Result
Nothing
Instead, it needs an explicit JNull to work correctly:
> fromJValue (JObject [("title",JString "mytitle"),("snippet",JString "mysnippet"),("url",JString "myurl"),("favicon",JNull)]) :: Maybe Result
Just (Result {title = "mytitle", snippet = "mysnippet", url = "myurl", favicon = Nothing})
so we'd need to do a bit more coding to get this to work properly.

Haskell Data.Decimal as Aeson type

Is it possible to parse a Data.Decimal from JSON using the Aeson package?
Suppose I have the following JSON:
{
"foo": 5.231,
"bar": "smth"
}
And the following record type:
data test { foo :: Data.Decimal
, bar :: String } deriving Generic
with
instance FromJSON test
instance ToJSON test
This would work, if it weren't for the Data.Decimal value "foo".
From what I understand I would need to manually create a FromJSON and ToJSON (for converting back to JSON) instance of Data.Decimal, since it doesn't derive from Generic. How can I do that?
Thanks in advance.
I know this is an old thread, but might help someone. Here is how I am converting Decimal to/from json (I assembled this code from a bunch of other code which I don't have the source for now):
instance A.ToJSON (DecimalRaw Integer) where
-- maybe this is not the best, but it works
toJSON d = A.toJSON $ show d
instance A.FromJSON (DecimalRaw Integer) where
parseJSON = A.withText "Decimal" (go . (read :: String -> [(DecimalRaw Integer, String)]) . T.unpack)
where
go [(v, [])] = return v
go (_ : xs) = go xs
go _ = fail "Could not parse number"

Override how Data.Aeson handles only one field of my record

I am making a REST API for university courses:
data Course = Course {
id :: Maybe Text,
name :: Text,
deleted :: Bool
} deriving(Show, Generic)
instance FromJSON Course
instance ToJSON Course
I would like to allow deleted to be optional in the serialized JSON structure, but not in my application. I want to set deleted to False if it isn't specified when parsing.
I could write a manual instance for FromJSON, but I don't want to have to write it out for all the fields. I want to declare how deleted is handled and let the automatic instance handle everything else.
How would I do this?
To my knowledge there is not a way to customize the generic instance, but you could structure your type a bit differently:
data Course = Course
{ courseId :: Maybe Text -- Don't use `id`, it's already a function
, name :: Text
} deriving (Show, Generic)
data Deletable a = Deletable
{ deleted :: Bool
, item :: a
} deriving (Show)
instance FromJSON Course
instance ToJSON Course
instance FromJSON a => FromJSON (Deletable a) where
parseJSON (Object v) = do
i <- parseJSON (Object v)
d <- v .:? "deleted" .!= False
return $ Deletable d i
parseJSON _ = mzero
Now you can do
> let noDeleted = "{\"name\":\"Math\",\"courseId\":\"12345\"}" :: Text
> let withDeleted = "{\"name\":\"Math\",\"courseId\":\"12345\",\"deleted\":true}" :: Text
> decode noDeleted :: Maybe (Deletable Course)
Just (Deletable {deleted = False, item = Course {courseId = Just "12345", name = "Math"}})
> decode noDeleted :: Maybe Course
Just (Course {courseId = Just "12345", name = "Math"})
> decode withDeleted :: Maybe (Deletable Course)
Just (Deletable {deleted = True, item = Course {courseId = Just "12345", name = "Math"}})
> decode withDeleted :: Maybe Course
Just (Course {courseId = Just "12345", name = "Math"})
And now you can just optionally tag a course as deletable when you need it, and the FromJSON instances take care of everything.