Haskell: Parsing and processing stream of json requests - json

Following code tries to execute a stream of Request objects.
{-# LANGUAGE DeriveGeneric, DeriveDataTypeable #-}
--OverloadedStrings,
import Data.Aeson
import Data.Data
import Data.ByteString.Lazy.Internal
import GHC.Generics
import qualified Data.Map as Map
data Request a = Request {
name :: String,
payload :: a
} deriving (Show, Generic)
instance FromJSON a => FromJSON (Request a)
instance ToJSON a => ToJSON (Request a)
class Exe a where
exe :: (Request a) -> String
data Createuser = Createuser {
userName :: String
} deriving (Show, Generic)
instance FromJSON Createuser
instance ToJSON Createuser
instance Exe Createuser where
exe r = "creating user"
parseCreateUser json = parse json :: Request Createuser
data Deleteuser = Deleteuser {
userName2 :: String
} deriving (Show, Generic)
instance FromJSON Deleteuser
instance ToJSON Deleteuser
instance Exe Deleteuser where
exe r = "deleting user"
parseDeleteUser json = parse json :: Request Deleteuser
tojson name payload = encode (Request name payload)
parse bs = let Just p = decode bs in p
sampleDeleteUser = tojson "deleteUser" (Deleteuser "Jigar Gosar")
sampleCreateUser = tojson "createUser" (Createuser "Jigar Gosar")
exeTuple tuple = case tuple of
("createUser", a) -> exe $ parseCreateUser $ a
("deleteUser", a) -> exe $ parseDeleteUser $ a
main = do
-- This list would be read from socket.
let tuples = [("createUser", sampleCreateUser), ("deleteUser", sampleDeleteUser)]
putStr $ unlines [exeTuple tup | tup <- tuples]
print "end"
Is there any way to remove the following duplication?
parseCreateUser json = parse json :: Request Createuser
parseDeleteUser json = parse json :: Request Deleteuser
I will have to write this for every (Request p) I create.
Also I will have to keep modifying following function every time I create new (Request p)
exeTuple tuple = case tuple of
("createUser", a) -> exe $ parseCreateUser $ a
("deleteUser", a) -> exe $ parseDeleteUser $ a
Is this idiomatic haskell code?

Related

Haskell Aeson json encoding bytestrings

I need to serialize a record in Haskell, and am trying to do it with Aeson. The problem is that some of the fields are ByteStrings, and I can't work out from the examples how to encode them. My idea is to first convert them to text via base64. Here is what I have so far (I put 'undefined' where I didn't know what to do):
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
module Main where
import qualified Data.Aeson as J
import qualified Data.ByteString as B
import qualified Data.ByteString.Base64 as B64
import qualified Data.Text as T
import qualified Data.Text.Encoding as E
import qualified GHC.Generics as G
data Data = Data
{ number :: Int
, bytestring :: B.ByteString
} deriving (G.Generic, Show)
instance J.ToJSON Data where
toEncoding = J.genericToEncoding J.defaultOptions
instance J.FromJSON Data
instance J.FromJSON B.ByteString where
parseJSON = undefined
instance J.ToJSON B.ByteString where
toJSON = undefined
byteStringToText :: B.ByteString -> T.Text
byteStringToText = E.decodeUtf8 . B64.encode
textToByteString :: T.Text -> B.ByteString
textToByteString txt =
case B64.decode . E.encodeUtf8 $ txt of
Left err -> error err
Right bs -> bs
encodeDecode :: Data -> Maybe Data
encodeDecode = J.decode . J.encode
main :: IO ()
main = print $ encodeDecode $ Data 1 "A bytestring"
It would be good if it was not necessary to manually define new instances of ToJSON and FromJSON for every record, because I have quite a few different records with bytestrings in them.
parseJson needs to return a value of type Parser B.ByteString, so you just need to call pure on the return value of B64.decode.
import Control.Monad
-- Generalized to any MonadPlus instance, not just Either String
textToByteString :: MonadPlus m => T.Text -> m B.ByteString
textToByteString = case B64.decode (E.encodeUtf8 x) of
Left _ -> mzero
Right bs -> pure bs
instance J.FromJSON B.ByteString where
parseJSON (J.String x) = textToByteString x
parseJSON _ = mzero
Here, I've chosen to return mzero both if you try to decode anything other than a JSON string and if there is a problem with the base-64 decoding.
Likewise, toJSON needs just needs to encode the Text value you create from the base64-encoded ByteString.
instance J.ToJSON B.ByteString where
toJSON = J.toJSON . byteStringToText
You might want to consider using a newtype wrapper instead of defining the ToJSON and FromJSON instances on B.ByteString directly.

Convert JSON back to Data in Haskell

I have the following Haskell to call my WCF Service:
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE DeriveGeneric #-}
module Main where
import Data.Aeson
import Data.Dynamic
import Data.Aeson.Lens
import Data.ByteString.Lazy as BS
import GHC.Generics
import Network.Wreq
import Control.Lens
data Point = Point { x :: Int, y :: Int } deriving (Generic, Show)
instance ToJSON Point
instance FromJSON Point
data Rectangle = Rectangle { width :: Int, height :: Int, point :: Point } deriving (Generic, Show)
instance ToJSON Rectangle
instance FromJSON Rectangle
main = do
let p = Point 1 2
let r = Rectangle 10 20 p
let url = "http://localhost:8000/Rectangle"
let opts = defaults & header "Content-Type" .~ ["application/json"]
r <- postWith opts url (encode r)
let returnData = r ^? responseBody
case (decode returnData) of
Nothing -> BS.putStrLn "Error decoding JSON"
Just json -> BS.putStrLn $ show $ decode json
The output in this case is:
Just "{\"height\":20,\"point\":{\"x\":1,\"y\":2},\"width\":10}"
I already tried it with fromJSON:
print $ fromJSON returnData
and got this error:
Couldn't match expected type `Value'
with actual type `Maybe
bytestring-0.10.6.0:Data.ByteString.Lazy.Internal.ByteString'
In the first argument of `fromJSON', namely `returnData'
In the second argument of `($)', namely `fromJSON returnData'
Failed, modules loaded: none.
My question is now how to convert this JSON string back to an object of type "Rectangle"?
EDIT 1: I changed my code due to Janos Potecki's answer and get now the following error:
Couldn't match type `[Char]' with `ByteString'
Expected type: ByteString
Actual type: String
In the second argument of `($)', namely `show $ decode json'
In the expression: BS.putStrLn $ show $ decode json
In a case alternative:
Just json -> BS.putStrLn $ show $ decode json
Failed, modules loaded: none.
EDIT 2: i changed it to:
main = do
let point = Point 1 2
let rectangle = Rectangle 10 20 point
let url = "http://localhost:8000/Rectangle/Move/100,200"
let opts = defaults & header "Content-Type" .~ ["application/json"]
r <- postWith opts url (encode rectangle)
let returnData = (r ^? responseBody) >>= decode
case returnData of
Nothing -> BS.putStrLn "Error decoding JSON"
Just json -> BS.putStrLn json
and now i get:
No instance for (FromJSON ByteString)
arising from a use of `decode'
In the second argument of `(>>=)', namely `decode'
In the expression: (r ^? responseBody) >>= decode
In an equation for `returnData':
returnData = (r ^? responseBody) >>= decode
working solution
r' <- asJSON =<< postWith opts url (encode rectangle) :: IO Res
case r' of
Nothing -> print "Error decoding JSON"
Just x -> print x
For performance I'd suggest you add the following to your instance ToJSON:
instance ToJSON Point where
toEncoding = genericToEncoding defaultOptions
and the same for Rectangle

Parsing JSON data from a URL in Haskell using Aeson

I'm trying to create a web app in Haskell that takes some JSON input from a URL.
{-# LANGUAGE OverloadedStrings, DeriveGeneric #-}
import Data.Aeson as Q
import Data.Text
import Control.Applicative
import Control.Monad
import qualified Data.ByteString.Lazy as B
import Network.HTTP.Conduit (simpleHttp)
import GHC.Generics
--import Data.DateTime
data Temperatures =
Temperatures { date :: String
, temperature :: Int
} deriving (Show, Generic)
instance FromJSON Temperatures
instance ToJSON Temperatures
jsonURL :: String
jsonURL = "http://www.phoric.eu/temperature"
getJSON :: IO B.ByteString
getJSON = simpleHttp jsonURL
main :: IO ()
main = do
d <- (eitherDecode <$> getJSON) :: IO (Either String Temperatures)
case d of
Left e -> putStrLn e
Right stuff -> print stuff
However, I'm getting the error at runtime:
Bradley$ runhaskell test.hs
The key "date" was not found
The JSON is below # the URL which is in the code.
{"temperatures":[
{"date":"2015-02-28T20:16:12+00:00", "temperature":0},
{"date":"2015-01-01T21:46:55+00:00", "temperature":2},
{"date":"2015-04-08T21:46:53+00:00", "temperature":3},
{"date":"2015-04-09T21:46:01+00:00", "temperature":4},
{"date":"2015-04-10T21:46:40+00:00", "temperature":5},
{"date":"2015-04-11T21:46:36+00:00", "temperature":6},
{"date":"2015-04-12T20:36:25+00:00", "temperature":7}
]}
I have no idea why it is not recognising the keys when they're clearly present in the JSON object, any ideas?
In its current form, your code can only parse a single line of the form {"date":"2015-04-12T20:36:25+00:00", "temperature":7} : it tries to parse the input into a Temperatures, but can't find a date key into the root object because it only has a temperatures key.
The problem is that your Temperatures datatype does not really match your input. Here is something that should match:
data DataPoint = DataPoint { date :: String
, temperature :: Int
} deriving (Show, Generic)
data Temperatures = Temperatures { temperatures :: [DataPoint]
} deriving (Show, Generic)

Conduit with aeson / attoparsec, how to exit cleanly without exception once source has no more data

I'm using aeson / attoparsec and conduit / conduit-http connected by conduit-attoparsec to parse JSON data from a file / webserver. My problem is that my pipeline always throws this exception...
ParseError {errorContexts = ["demandInput"], errorMessage = "not enough bytes", errorPosition = 1:1}
...once the socket closes or we hit EOF. Parsing and passing on the resulting data structures through the pipeline etc. works just fine, but it always ends with the sinkParser throwing this exception. I invoke it like this...
j <- CA.sinkParser json
...inside of my conduit that parses ByteStrings into my message structures.
How can I have it just exit the pipeline cleanly once there is no more data (no more top-level expressions)? Is there any decent way to detect / distinguish this exception without having to look at error strings?
Thanks!
EDIT: Example:
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Applicative
import qualified Data.ByteString as B
import qualified Data.ByteString.Char8 as B8
import qualified Data.Conduit.Attoparsec as CA
import Data.Aeson
import Data.Conduit
import Data.Conduit.Binary
import Control.Monad.IO.Class
data MyMessage = MyMessage String deriving (Show)
parseMessage :: (MonadIO m, MonadResource m) => Conduit B.ByteString m B.ByteString
parseMessage = do
j <- CA.sinkParser json
let msg = fromJSON j :: Result MyMessage
yield $ case msg of
Success r -> B8.pack $ show r
Error s -> error s
parseMessage
main :: IO ()
main =
runResourceT $ do
sourceFile "./input.json" $$ parseMessage =$ sinkFile "./out.txt"
instance FromJSON MyMessage where
parseJSON j =
case j of
(Object o) -> MyMessage <$> o .: "text"
_ -> fail $ "Expected Object - " ++ show j
Sample input (input.json):
{"text":"abc"}
{"text":"123"}
Outputs:
out: ParseError {errorContexts = ["demandInput"], errorMessage = "not enough bytes", errorPosition = 3:1}
and out.txt:
MyMessage "abc"MyMessage "123"
This is a perfect use case for conduitParserEither:
parseMessage :: (MonadIO m, MonadResource m) => Conduit B.ByteString m B.ByteString
parseMessage =
CA.conduitParserEither json =$= awaitForever go
where
go (Left s) = error $ show s
go (Right (_, msg)) = yield $ B8.pack $ show msg ++ "\n"
If you're on FP Haskell Center, you can clone my solution into the IDE.

JSON parsing befuddlement

I'm working on the I/O aspect of my json server, and there's a method I just can't get right.
First, I'll give the error, then the code and datatypes involved and some commentary about the problem afterwards.
("X-Response-Body-Start","<!DOCTYPE html>\n<html><head><title>Invalid Arguments</title></head><body><h1>Invalid Arguments</h1><ul><li>when expecting a unit constructor (U1), encountered String instead</li></ul></body></html>")
Expecting unit contructor?
Okay here's some relevant code. Let's see if we can see where I go wrong
from Datatypes.hs
data JobID = JobID Project Int deriving Generic
data Project = BNAP deriving (Show,Generic) -- one day to be an ADT
instance ToJSON Project where
toJSON = toJSON . show
instance FromJSON Project
instance FromJSON JobID
instance ToJSON JobID
The test code
testReadR :: IO Value
testReadR = do
req <- parseUrl readURI
manager <- newManager def
pBody <- runResourceT $ do
reqBody <- readObject
liftIO $ print reqBody
Response _ _ _ body <- http (buildReq req reqBody) manager
pBody <- body $$+- sinkParser json
return pBody -- (return wraps it up)
closeManager manager
return pBody
buildReq :: forall a (m :: * -> *) (t :: * -> *).
ToJSON a =>
Request t -> a -> Request m
buildReq req reqBody =
let reqBodyBS = Data.Aeson.encode reqBody
rHeaders = [(hContentType,pack "application/json")]
in req {method = fromString "POST"
, requestBody = RequestBodyLBS reqBodyBS
,requestHeaders=rHeaders
}
readObject :: ResourceT IO Value
readObject = do -- I took a bunch out because I thought simplifiying would help me
-- solve this
return $ Data.Aeson.toJSON $ JobID BNAP 306
The Handler
postReadR :: Handler RepJson
postReadR = do
conf <- parseJsonBody_ :: Handler JobID
liftIO $ print conf
testJ <- jsonToRepJson $ toJSON $ JobID BNAP 305
jValue <- jsonToRepJson conf -- to be replaced with
-- Either ErrorReport Response
-- (or something like that)
return jValue
when I change the line to
conf <- parseJsonBody_ :: Handler Value
print conf yields
Array (fromList [String "BNAP",Number 306])
So it seems the problem lies with String "BNAP" but I don't know why. Any ideas on how I can suss this out? Is there an obvious answer I'm not seeing?
Update : I have a new error. I'm sure I borked the FromJSON instance.
test: ResponseTimeout
instance FromJSON Project where
parseJSON (String p) = parseJSON $ toJSON p
parseJSON _ = mzero
The challenge here was that Project is a unary type. None of the examples I studied seemed to address that. But I know p is a Text, and toJSON can make a Value out of that, and parseJSON makes a Parser out of the Value. So it's all good right? Well, I still get the above error which is not informative at all. Any ideas?
instance ToJSON Project where
toJSON = toJSON . show
instance FromJSON Project
The instances don't work together. The generic FromJSON instance expects the generic unit constructor U1, but the ToJSON instance produces a String "BNAP".
If you have a hand-written ToJSON instance, you also need a hand-written FromJSON instance.
instance FromJSON Project where
parseJSON (String p) = parseJSON $ toJSON p
parseJSON _ = mzero
p is a Text, and we have
instance ToJSON Text where
toJSON = String
so the above instance for Project loops, since it expands parseJSON (String p) = parseJSON (String p).
For the type as it stands now,
instance FromJSON Project where
parseJSON (String "BNAP") = return BNAP
parseJSON _ = mzero
should work if you have OverloadedStrings enabled, if not,
instance FromJSON Project where
parseJSON (String p)
| p == pack "BNAP" = return BNAP
parseJSON _ = mzero