How to validate JSON with schema in Haskell? - json

I want to validate JSON with a schema. hjsonschema seemed like a good choice as it is fairly new and supports the latest draft.
But the plotly json schema always gives me valid responses.
I may be misunderstanding something here but this should not be valid JSON
bad.json
{
"fjsdklj" : 5
}
even though it is considered valid by the following code
module Main where
import Control.Applicative
import Data.Aeson
import Data.HashMap.Strict (HashMap)
import qualified Data.HashMap.Strict as H
import Data.Monoid
import qualified Data.ByteString.Lazy as B
import qualified Data.JsonSchema as JS
import Data.Maybe
main :: IO ()
main = do
schemaJSON <- (fromJust . decode) <$> B.readFile "simple-schema.json"
bad <- (fromJust . decode) <$> B.readFile "bad.json"
let schemaData = JS.RawSchema {
JS._rsURI = Nothing,
JS._rsData = schemaJSON
}
schema <- compileSchema (JS.SchemaGraph schemaData H.empty) schemaData
checkResults (JS.validate schema bad)
compileSchema :: JS.SchemaGraph -> JS.RawSchema -> IO (JS.Schema JS.Draft4Failure)
compileSchema graph rs =
case JS.compileDraft4 graph rs of
Left failure -> error $ "Not a valid schema: " <> show failure
Right schema -> return schema
checkResults :: [JS.ValidationFailure JS.Draft4Failure] -> IO ()
checkResults [] = putStrLn "Just fine"
checkResults x = putStrLn $ "ERROR: " ++ show x
simple-schema.json is the plotly schema and bad.json the snippet I posted above.

It's nothing about Haskell.
Your schema doesn't have a required property at top level so an empty json object is acceptable.
Also it contains no "additionalProperties": false property so anything that does not fit into one of the defined patterns are ignored.
BTW I doubt whether it's a valid Draft4 json schema. It passed the validation of the Draft4 meta-schema but the syntax is a little bit different, maybe something Python-specific. You'd better run the test suites which came along with the hjsonscheme package to see if everything works fine.

Related

Haskell - how do I convert piped-in JSON-based to a data record?

I'm building a reinforcement learning library where I'd like to pass certain instance information into the executables via a piped JSON.
Using aeson's Simplest.hs, I'm able to get the following basic example working as intended. Note that the parameters are sitting in Main.hs as a String params as a placeholder.
I tried to modify Main.hs so I would pipe the Nim game parameters in from a JSON file via getContents, but am running into the expected [Char] vs. IO String issue. I've tried to read up as much as possible about IO, but can't figure out how to lift my JSON parsing method to deal with IO.
How would I modify the below so that I can work with piped-in JSON?
Main.hs
module Main where
import qualified System.Random as Random
import qualified Data.ByteString.Lazy.Char8 as BL
import qualified Games.Engine as Engine
import qualified Games.IO.Nim as NimIO
import qualified Games.Rules.Nim as Nim
import qualified Games.Learn.ValueIteration as VI
main :: IO ()
main = do
let params = "{\"players\":[\"Bob\", \"Alice\", \"Charlie\"], \"initialPiles\": [3, 4, 5], \"isMisere\": false}"
let result = NimIO.decode $ BL.pack params :: Maybe NimIO.NimGame
case result of
Nothing -> putStrLn "Parameter errors."
Just game -> do
putStrLn "Let's play some Nim! Remainder of code omitted"
Games.IO.Nim.hs
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RecordWildCards #-}
module Games.IO.Nim
( decode
, NimGame
, players
, initialPiles
, isMisere
) where
import Control.Applicative (empty)
import qualified Data.ByteString.Lazy.Char8 as BL
import Data.Aeson
( pairs,
(.:),
object,
FromJSON(parseJSON),
Value(Object),
KeyValue((.=)),
ToJSON(toJSON, toEncoding),
decode)
data NimGame = NimGame
{ players :: [String]
, initialPiles :: [Int]
, isMisere :: Bool
} deriving (Show)
instance ToJSON NimGame where
toJSON (NimGame playersV initialPilesV isMisereV) = object [ "players" .= playersV,
"initialPiles" .= initialPilesV,
"isMisere" .= isMisereV]
toEncoding NimGame{..} = pairs $
"players" .= players <>
"initialPiles" .= initialPiles <>
"isMisere" .= isMisere
instance FromJSON NimGame where
parseJSON (Object v) = NimGame <$>
v .: "players" <*>
v .: "initialPiles" <*>
v .: "isMisere"
parseJSON _ = empty
Alternative Main.hs that generates compile error
module Main where
import qualified System.Random as Random
import qualified Data.ByteString.Lazy.Char8 as BL
import qualified Games.Engine as Engine
import qualified Games.IO.Nim as NimIO
import qualified Games.Rules.Nim as Nim
import qualified Games.Learn.ValueIteration as VI
main :: IO ()
main = do
--let params = "{\"players\":[\"Bob\", \"Alice\", \"Charlie\"], \"initialPiles\": [3, 4, 5], \"isMisere\": false}"
let params = getContents
let result = NimIO.decode $ BL.pack params :: Maybe NimIO.NimGame
case result of
Nothing -> putStrLn "Parameter errors."
Just game -> do
putStrLn "Let's play some Nim!"
Compile Error
(base) randm#pearljam ~/Projects/gameshs $ stack build
gameshs-0.1.0.0: unregistering (local file changes: app/Nim.hs)
gameshs> configure (lib + exe)
Configuring gameshs-0.1.0.0...
gameshs> build (lib + exe)
Preprocessing library for gameshs-0.1.0.0..
Building library for gameshs-0.1.0.0..
Preprocessing executable 'nim-exe' for gameshs-0.1.0.0..
Building executable 'nim-exe' for gameshs-0.1.0.0..
[2 of 2] Compiling Main
/home/randm/Projects/gameshs/app/Nim.hs:17:41: error:
• Couldn't match expected type ‘[Char]’
with actual type ‘IO String’
• In the first argument of ‘BL.pack’, namely ‘params’
In the second argument of ‘($)’, namely ‘BL.pack params’
In the expression:
NimIO.decode $ BL.pack params :: Maybe NimIO.NimGame
|
17 | let result = NimIO.decode $ BL.pack params :: Maybe NimIO.NimGame
| ^^^^^^
-- While building package gameshs-0.1.0.0 (scroll up to its section to see the error) using:
/home/randm/.stack/setup-exe-cache/x86_64-linux-tinfo6/Cabal-simple_mPHDZzAJ_3.2.1.0_ghc-8.10.4 --builddir=.stack-work/dist/x86_64-linux-tinfo6/Cabal-3.2.1.0 build lib:gameshs exe:nim-exe --ghc-options " -fdiagnostics-color=always"
Process exited with code: ExitFailure 1
getContents returns not a String as you apparently expect, but IO String, which is a "program", which, when executed, will produce a String. So when you're trying to parse this program with decode, of course that doesn't work: decode parses a String, it cannot parse a program.
So how do you execute this program to obtain the String? There are two ways: either you make it part of another program or you call it main and it becomes your entry point.
In your case, the sensible thing to do would be to make getContent part of your main program. To do that, use the left arrow <-, like this:
main = do
params <- getContents
let result = NimIO.decode $ BL.pack params :: Maybe NimIO.NimGame
...

GHC API: Find all functions (and their types) in scope

I'm trying to list (print) all functions (and their types) that are in scope for a given module.
For example, I have this module:
{-# LANGUAGE NoImplicitPrelude #-}
module Reverse where
import Prelude ((==), String)
myString :: String
myString = "string"
I'm trying to use the GHC API (8.0.2), but I can't seem to find where the information that I'm looking for is stored.
I have managed to find all functions in scope, (but not with their types,) with this code:
import Data.IORef
import DynFlags
import GHC
import GHC.LanguageExtensions
import GHC.Paths (libdir)
import HscTypes
import NameEnv
import OccName
import Outputable
import RdrName
import TcRnTypes
main = runGhc (Just libdir) $ do
liftIO $ print sets
dflags <- getSessionDynFlags
let compdflags =
(foldl xopt_set dflags [Cpp, ImplicitPrelude, MagicHash])
setSessionDynFlags compdflags
target <- guessTarget "Reverse.hs" Nothing
setTargets [target]
load LoadAllTargets
modSum <- getModSummary $ mkModuleName "Reverse"
parsedModule <- parseModule modSum
tmod <- typecheckModule parsedModule
let (tcenv, moddets) = tm_internals_ tmod
printO $ map (map gre_name) $ occEnvElts $ tcg_rdr_env tcenv
printO
:: (GhcMonad m, Outputable a)
=> a -> m ()
printO a = do
dfs <- getProgramDynFlags
liftIO $ putStrLn $ showPpr dfs a
I get this output:
[[String], [==], [myString]]
Of course, this is only half of the way to the data I need.
The GHC API is rather confusing, you have to get used to a lot of abbreviations, type-synonyms and a style-heterogenous codebase, but finding the the name and type of everything in scope should be possible.
Otherwise GHC couldn't tell you that your function is not in scope if you make a typo.
Indeed, once you have type-checked a module, all the relevant information is available.
First you need all the Names of your functions, which you can get with this code:
parsedModule <- parseModule modSum
tmod <- typecheckModule parsedModule
let (tcenv, moddets) = tm_internals_ tmod
let names = concatMap (map gre_name) $ occEnvElts $ tcg_rdr_env tcenv
Then you need to lookup the Names to get TyThings with lookupName.
You'll get a Maybe TyThing (Nothing if the Name is not found), and when the name refers to a function, the TyThing will be AnId i where i is the thing you're looking for.
An Id is just a name with a type.
You can then get the type with varType.
You could argue that all these types make this problem a lot harder, but they've made it possible for me to figure out what I need to do without looking into the code and with no documentation.

Mustache not rendering JSON value as JSON encoded string

The full example is here:
{-# LANGUAGE OverloadedStrings #-}
module Test2 where
import Data.Aeson
import Text.Mustache
main :: IO ()
main = do
let example = Data.Aeson.object [ "key" .= (5 :: Integer), "somethingElse" .= (2 :: Integer) ]
print . encode $ example
print ("Start" :: String)
case compileTemplate "" "{{{jsonData}}}" of
Right x -> do
print $ substituteValue x (Text.Mustache.object ["jsonData" ~= example])
Left e -> error . show $ e
The above produces the following output:
"{\"somethingElse\":2,\"key\":5}"
"Start"
"fromList [(\"somethingElse\",2.0),(\"key\",5.0)]"
My expectation is it would produce:
"{\"somethingElse\":2,\"key\":5}"
"Start"
"{\"somethingElse\":2,\"key\":5}"
Mustache doesn't seem to support straight substitution of JSON objects. Setting up a similar example here, I receive
[object Object]
as output. Not identical to yours, but it indicates that the issue is not necessarily with the Haskell implementation.
In other words, I believe the issue is with your template {{{jsonData}}}.
If you change it to {{{jsonData.somethingElse}}}, it works fine (I know this is not what you want).
Alternatively, encode the JSON data as text prior to passing it to the substitution function as suggested here. Basically something like this:
substituteValue x (Text.Mustache.object ["jsonData" ~= (encodeToLazyText jsonData)])
This results in your desired output. encodeToLazyText is found in Data.Aeson.Text.
Working code:
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Data.Aeson ((.=))
import qualified Data.Aeson as A
import Data.Aeson.Text (encodeToLazyText)
import Text.Mustache ((~=))
import qualified Text.Mustache as M
import qualified Text.Mustache.Types as M
main :: IO ()
main = do
print . A.encode $ jsonData
putStrLn "Start"
case M.compileTemplate "" "in mustache: {{{jsonData}}}" of
Right template ->
print (M.substituteValue template mustacheVals)
Left e ->
error . show $ e
jsonData :: A.Value
jsonData =
A.object
[ "key" .= (5 :: Integer)
, "somethingElse" .= (2 :: Integer)
]
mustacheVals :: M.Value
mustacheVals =
M.object
[ "jsonData" ~= encodeToLazyText jsonData
]

Creating a csv file in haskell

I suspect this a bit of a noobie question...
So I have a list myList that I would like to write to a csv file, using the cassava library:
import qualified Data.ByteString.Lazy as BL
import qualified Data.Vector as V
import Data.Csv
main = BL.writeFile "myFileLocation" (encode $ V.fromList myList)
as far as I can establish, encode has type ToRecord a => Vector a -> ByteString yet I'm getting the following error:
Couldn't match expected type `[a0]'
with actual type `V.Vector (a,b,c,d)'
In the return type of a call of `V.fromList'
In the second argument of `($)', namely `V.fromList myList'
In the second argument of `BL.writeFile', namely
`(encode $ V.fromList myList)'
I'm confused!

Fault tolerant JSON parsing

I'm using Data.Aeson to parse some JSON into a Record type. From time to time data is added to the JSON and this breaks my code as Aeson complains something to the effect of:
expected Object with 21
name/value pairs but got 23 name/value
I'd really prefer to parse the JSON in a fault tolerant way -- I don't care if more fields are added to the JSON at a later date, just parse whatever you can! Is there a way to achieve this fault tolerance? Here's my code:
myRecordFromJSONString :: BS.ByteString -> Maybe MyRecord
myRecordFromJSONString s = case Data.Attoparsec.parse json s of
Done _rest res -> Data.Aeson.Types.parseMaybe parseJSON res
_ -> Nothing
I should add that I'm using deriveJSON from Data.Aeson.TH to generate the parsing code. If I write the FromJSON code manually it's fault tolerant but I'd like to not have to do that...
If you are using GHC 7.2 or 7.4, the new generics support in aeson doesn't check for extra fields. I'm not sure if this is by design or not but we use it for the same reason.
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
import Data.Aeson
import qualified Data.Aeson.Types
import Data.Attoparsec
import qualified Data.ByteString as BS
import Data.ByteString.Char8 ()
import GHC.Generics
data MyRecord = MyRecord
{ field1 :: Int
} deriving (Generic, Show)
instance FromJSON MyRecord
myRecordFromJSONString :: BS.ByteString -> Maybe MyRecord
myRecordFromJSONString s = case Data.Attoparsec.parse json s of
Done _rest res -> Data.Aeson.Types.parseMaybe parseJSON res
_ -> Nothing
main :: IO ()
main = do
let parsed = myRecordFromJSONString "{ \"field1\": 1, \"field2\": 2 }"
print parsed
Running this would fail with the TH derived instance due to 'field2' not existing in the record. The Generic instance returns the desired result:
Just (MyRecord {field1 = 1})