How to handle variability of JSON objects in Haskell? - json

Question

Some REST service has variable returning JSONs, for example some fields can appear or disappear depending on the parameters of the request, the structure itself may change, nesting, etc.
So, this leads to avalanche-type growth in the number of types (along with FromJSON instances). Options are to:
try to make a lot of fields under Maybe (but this does not help very much with the variability in structure)
to introduce a lot of types
to create different phantom types (actually no big difference with prev.)
The 1. has drawback that if your call with some fixed parameters always returns good knows fields, you have to handle Nothing cases too, code becomes more complex. The 2. and 3. is tiring.
What is the most simple/convenient way to handle such variability in Haskell (if you use Aeson, sure, another option is to avoid Aeson usage)?

Answer 1

A possible solution to the existing/non-existing fields problem using type-level computation.
Some required extensions and imports:
{-# LANGUAGE DeriveGeneric, ScopedTypeVariables, DataKinds, KindSignatures,
TypeApplications, TypeFamilies, TypeOperators, FlexibleContexts #-}
import Data.Aeson
import Data.Proxy
import GHC.Generics
import GHC.TypeLits
Here's a data type (to be used promoted) that indicates if some field is absent or present. Also a type family that maps absent types to ():
data Presence = Present
| Absent
type family Encode p v :: * where
Encode Present v = v
Encode Absent v = ()
Now we can define a parameterized record containing all possible fields, like this:
data Foo (a :: Presence)
(b :: Presence)
(c :: Presence) = Foo {
field1 :: Encode a Int,
field2 :: Encode b Bool,
field3 :: Encode c Char
} deriving Generic
instance (FromJSON (Encode a Int),
FromJSON (Encode b Bool),
FromJSON (Encode c Char)) => FromJSON (Foo a b c)
One problem: writing the full type for each combination of occurrences/absences would be tedious, especially if only a few fields are present each time. But perhaps we could define an auxiliary type synonym FooWith that let us mention only those fields that are present:
type family Mentioned (ns :: [Symbol]) (n :: Symbol) :: Presence where
Mentioned '[] _ = Absent
Mentioned (n ': _) n = Present
Mentioned (_ ': ns) n = Mentioned ns n
-- the field names are repeated as symbols, how to avoid this?
type FooWith (ns :: [Symbol]) = Foo (Mentioned ns "field1")
(Mentioned ns "field2")
(Mentioned ns "field3")
Example of use:
ghci> :kind! FooWith '["field2","field3"]
FooWith '["field2","field3"] :: * = Foo 'Absent 'Present 'Present
Another problem: for each request, we must repeat the list of required fields two times: one in the URL ("fields=a,b,c...") and another in the expected type. It would be better to have a single source of truth.
We can deduce the term-level list of fields to be added to the URL from the type-level list of fields, by using an auxiliary type class Demote:
class Demote (ns :: [Symbol]) where
demote :: Proxy ns -> [String]
instance Demote '[] where
demote _ = []
instance (KnownSymbol n, Demote ns) => Demote (n ': ns) where
demote _ = symbolVal (Proxy #n) : demote (Proxy #ns)
For example:
ghci> demote (Proxy #["field2","field3"])
["field2","field3"]