I want to parse a JSON object and create a JSONEvent with the given name and args
I'm using Aeson, and right now I'm stucked on converting "args":[{"a": "b"}] to a [(String, String)].
Thank's in advance!
{-# LANGUAGE OverloadedStrings #-}
import Control.Applicative
import Data.Aeson
data JSONEvent = JSONEvent [(String, String)] (Maybe String) deriving Show
instance FromJSON JSONEvent where
parseJSON j = do
o <- parseJSON j
name <- o .:? "name"
args <- o .:? "args" .!= []
return $ JSONEvent args name
let decodedEvent = decode "{\"name\":\"edwald\",\"args\":[{\"a\": \"b\"}]}" :: Maybe JSONEvent
Here's a bit more elaborate example based on ehird's example. Note that the explicit typing on calls to parseJSON is unnecessary but I find them useful for documentation and debugging purposes. Also I'm not sure what you intended, but with args with multiple values I simply concatenate all the args together like so:
*Main> decodedEvent2
Just (JSONEvent [("a","b"),("c","d")] (Just "edwald"))
*Main> decodedEvent3
Just (JSONEvent [("a","b"),("c","d")] (Just "edwald"))
Here's the code:
{-# LANGUAGE OverloadedStrings #-}
import Control.Applicative
import qualified Data.Text as T
import qualified Data.Foldable as F
import qualified Data.HashMap.Lazy as HM
import qualified Data.Vector as V
import Data.Aeson
import qualified Data.Attoparsec as P
import Data.Aeson.Types (Parser)
import qualified Data.Aeson.Types as DAT
import qualified Data.String as S
data JSONEvent = JSONEvent [(String, String)] (Maybe String) deriving Show
instance FromJSON JSONEvent where
parseJSON = parseJSONEvent
decodedEvent = decode "{\"name\":\"edwald\",\"args\":[{\"a\": \"b\"}]}" :: Maybe JSONEvent
decodedEvent2 = decode "{\"name\":\"edwald\",\"args\":[{\"a\": \"b\"}, {\"c\": \"d\"}]}" :: Maybe JSONEvent
decodedEvent3 = decode "{\"name\":\"edwald\",\"args\":[{\"a\": \"b\", \"c\": \"d\"}]}" :: Maybe JSONEvent
emptyAesonArray :: Value
emptyAesonArray = Array $ V.fromList []
parseJSONEvent :: Value -> Parser JSONEvent
parseJSONEvent v =
case v of
Object o -> do
name <- o .:? "name"
argsJSON <- o .:? "args" .!= emptyAesonArray
case argsJSON of
Array m -> do
parsedList <- V.toList <$> V.mapM (parseJSON :: Value -> Parser (HM.HashMap T.Text Value)) m
let parsedCatList = concatMap HM.toList parsedList
args <- mapM (\(key, value) -> (,) <$> (return (T.unpack key)) <*> (parseJSON :: Value -> Parser String) value) parsedCatList
return $ JSONEvent args name
_ -> fail ((show argsJSON) ++ " is not an Array.")
_ -> fail ((show v) ++ " is not an Object.")
-- Useful for debugging aeson parsers
decodeWith :: (Value -> Parser b) -> String -> Either String b
decodeWith p s = do
value <- P.eitherResult $ (P.parse json . S.fromString) s
DAT.parseEither p value
I'm not an aeson expert, but if you have Object o, then o is simply a HashMap Text Value; you could use Data.HashMap.Lazy.toList to convert it into [(Text, Value)], and Data.Text.unpack to convert the Texts into Strings.
So, presumably you could write:
import Control.Arrow
import Control.Applicative
import qualified Data.Text as T
import qualified Data.Foldable as F
import qualified Data.HashMap.Lazy as HM
import Data.Aeson
instance FromJSON JSONEvent where
parseJSON j = do
o <- parseJSON j
name <- o .:? "name"
Object m <- o .:? "args" .!= []
args <- map (first T.unpack) . HM.toList <$> F.mapM parseJSON m
return $ JSONEvent args name
Related
An example is easier to explain so I would like to parse this data into a datatype with cassava
title;authors
Cambridge Economic History;Ian MorrisWalter,ScheidelRichard,P Saller
I tried to do the following but it does not work (minimal not-working example) :
{-# LANGUAGE OverloadedStrings, DeriveGeneric #-}
module Library where
import Data.Csv
import Data.List.Split
import qualified Data.Vector as V
import qualified Data.Text as T
import GHC.Generics
data Book = Book {
title :: T.Text,
authors :: Authors
} deriving (Generic, Show)
type Authors = [T.Text]
instance FromNamedRecord Book
instance FromNamedRecord Authors
parseField "authors" =
pure $ splitOn "," ???
opts = defaultDecodeOptions {
decDelimiter = fromIntegral (ord ';')
}
main c = do
csvData <- BL.readFile "data.csv"
let res = decodeByNameWith opts csvData :: Either String (Header, V.Vector Book)
Is it possible to do it in Cassava ? Thanks!
The easiest way to do this is to write a customized FromNamedRecord instance for Book, instead of deriving the Generic one. It would look something like:
instance FromNamedRecord Book where
parseNamedRecord m = Book <$>
m .: "title" <*>
(T.splitOn "," <$> m .: "authors")
Here, m .: "authors" retrieves the authors field as a Text record, and the T.splitOn "," is fmapped (<$>) over that result to split the Text into [Text] by commas.
The full program:
{-# LANGUAGE OverloadedStrings #-}
module Library where
import Data.Char
import Data.Csv
import qualified Data.Vector as V
import qualified Data.Text as T
import Data.ByteString.Lazy as BL
data Book = Book {
title :: T.Text,
authors :: Authors
} deriving (Show)
type Authors = [T.Text]
instance FromNamedRecord Book where
parseNamedRecord m = Book <$>
m .: "title" <*>
(T.splitOn "," <$> m .: "authors")
opts = defaultDecodeOptions {
decDelimiter = fromIntegral (ord ';')
}
main = do
csvData <- BL.readFile "data.csv"
let res = decodeByNameWith opts csvData :: Either String (Header, V.Vector Book)
print res
giving:
λ> main
Right (["title","authors"],[Book {title = "Cambridge Economic History",
authors = ["Ian MorrisWalter","ScheidelRichard","P Saller"]}])
Note that this doesn't allow you to handle per-author quoting in the author list, so if you need to parse an author with an embedded comma, like:
Another Book;John Smith,"Anne Douglas, Jr."
you'll be out of luck. Cassava will refuse to parse an "authors" field with embedded quotes like this, and you'll end up having to write your own specialized CSV parser, I think.
I need to serialize a record in Haskell, and am trying to do it with Aeson. The problem is that some of the fields are ByteStrings, and I can't work out from the examples how to encode them. My idea is to first convert them to text via base64. Here is what I have so far (I put 'undefined' where I didn't know what to do):
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
module Main where
import qualified Data.Aeson as J
import qualified Data.ByteString as B
import qualified Data.ByteString.Base64 as B64
import qualified Data.Text as T
import qualified Data.Text.Encoding as E
import qualified GHC.Generics as G
data Data = Data
{ number :: Int
, bytestring :: B.ByteString
} deriving (G.Generic, Show)
instance J.ToJSON Data where
toEncoding = J.genericToEncoding J.defaultOptions
instance J.FromJSON Data
instance J.FromJSON B.ByteString where
parseJSON = undefined
instance J.ToJSON B.ByteString where
toJSON = undefined
byteStringToText :: B.ByteString -> T.Text
byteStringToText = E.decodeUtf8 . B64.encode
textToByteString :: T.Text -> B.ByteString
textToByteString txt =
case B64.decode . E.encodeUtf8 $ txt of
Left err -> error err
Right bs -> bs
encodeDecode :: Data -> Maybe Data
encodeDecode = J.decode . J.encode
main :: IO ()
main = print $ encodeDecode $ Data 1 "A bytestring"
It would be good if it was not necessary to manually define new instances of ToJSON and FromJSON for every record, because I have quite a few different records with bytestrings in them.
parseJson needs to return a value of type Parser B.ByteString, so you just need to call pure on the return value of B64.decode.
import Control.Monad
-- Generalized to any MonadPlus instance, not just Either String
textToByteString :: MonadPlus m => T.Text -> m B.ByteString
textToByteString = case B64.decode (E.encodeUtf8 x) of
Left _ -> mzero
Right bs -> pure bs
instance J.FromJSON B.ByteString where
parseJSON (J.String x) = textToByteString x
parseJSON _ = mzero
Here, I've chosen to return mzero both if you try to decode anything other than a JSON string and if there is a problem with the base-64 decoding.
Likewise, toJSON needs just needs to encode the Text value you create from the base64-encoded ByteString.
instance J.ToJSON B.ByteString where
toJSON = J.toJSON . byteStringToText
You might want to consider using a newtype wrapper instead of defining the ToJSON and FromJSON instances on B.ByteString directly.
I'm new to Haskell and in order to learn the language I am working on a project that involves dealing with JSON. I am currently getting the feeling Haskell is the wrong language for the job, but that isn't the point here.
I've been struggling to understand how this works for a few days. I have searched and everything I have found does not seem to work. Here's the issue:
I have some JSON in the following format:
>>>less "path/to/json"
{
"stringA1_stringA2": {"stringA1":floatA1,
"stringA2":foatA2},
"stringB1_stringB2": {"stringB1":floatB1,
"stringB2":floatB2}
...
}
Here floatX1 and floatX2 are actually strings of the form "0.535613567", "1.221362183" etc. What I want to do is parse this into the following data
data Mydat = Mydat { name :: String, num :: Float} deriving (Show)
where name would correspond to "stringX1_stringX2" and num to floatX1 for X = A,B,...
So far I have reached a 'solution' which feels fairly hackish and convoluted and doesn't work properly.
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE DeriveGeneric #-}
import Data.Functor
import Data.Monoid
import Data.Aeson
import Data.List
import Data.Text
import Data.Map (Map)
import qualified Data.HashMap.Strict as DHM
--import qualified Data.HashMap as DHM
import qualified Data.ByteString.Lazy as LBS
import System.Environment
import GHC.Generics
import Text.Read
data Mydat = Mydat {name :: String, num :: Float} deriving (Show)
test s = do
d <- LBS.readFile s
let v = decode d :: Maybe (DHM.HashMap String Object)
case v of
-- Just v -> print v
Just v -> return $ Prelude.map dataFromList $ DHM.toList $ DHM.map (DHM.lookup "StringA1") v
good = ['1','2','3','4','5','6','7','8','9','0','.']
f x = elem x good
dataFromList :: (String, Maybe Value) -> Mydat
dataFromList (a,b) = Mydat a (read (Prelude.filter f (show b)) :: Float)
Now I can compile this and run
test "path/to/json"
in ghci and it prints a list of Mydat's in the case where "stringX1"="stringA1" for all X. In reality there are two values for "stringX1" so aside from the hackyness this is not satisfactory. There must be a better way to do this. I get that I need to write my own parser probably but I am confused about how this works so any suggestions would be great. Thanks in advance.
The structure of your JSON is pretty nasty, but here's a basic working solution:
#!/usr/bin/env stack
-- stack --resolver lts-11.5 script --package containers --package aeson
{-# LANGUAGE OverloadedStrings #-}
import qualified Data.Map as Map
import qualified Data.Aeson as Aeson
data Mydat = Mydat { name :: String
, num :: Float
} deriving (Show)
instance Eq Mydat where
(Mydat _ x1) == (Mydat _ x2) = x1 == x2
instance Ord Mydat where
(Mydat _ x1) `compare` (Mydat _ x2) = x1 `compare` x2
type MydatRaw = Map.Map String (Map.Map String String)
processRaw :: MydatRaw -> [Mydat]
processRaw = Map.foldrWithKey go []
where go key value accum =
accum ++ (Mydat key . read <$> Map.elems value)
main :: IO ()
main =
do let json = "{\"stringA1_stringA2\":{\"stringA1\":\"0.1\",\"stringA2\":\"0.2\"}}"
print $ fmap processRaw (Aeson.eitherDecode json)
Note that read is partial and generally not a good idea. But I'll leave it to you to flesh out a safer version :)
As I commented, the best thing would probably be to make your JSON file well-formed in the sense that the float fields should really be floats, not strings.
If that's not an option, I would recommend you phrase out the type that the JSON file seems to represent as simple as possible (but without dynamic Objects), and then convert that to the type you actually want.
import Data.Map (Map)
import qualified Data.Map as Map
type GarbledJSON = Map String (Map String String)
-- ^ you could also stick with hash maps for everything, but
-- usually `Map` is actually more sensible in Haskell.
data MyDat = MyDat {name :: String, num :: Float} deriving (Show)
test :: FilePath -> IO [MyDat]
test s = do
d <- LBS.readFile s
case decode d :: Maybe GarbledJSON of
Just v -> return [ MyDat iName ( read . filter (`elem`good)
$ iVals Map.! valKey )
| (iName, iVals) <- Map.toList v
, let valKey = takeWhile (/='_') iName ]
Note that this will crash completely if any of the items don't contain the first part of the name as a string of float format, and likely give bogus items when you filter out characters that aren't good. If you just want to ignore any malformed items (which is also not a very clean approach...), you can do it this way:
test :: FilePath -> IO [MyDat]
test s = do
d <- LBS.readFile s
return $ case decode d :: Maybe GarbledJSON of
Just v -> [ MyDat iName iVal
| (iName, iVals) <- Map.toList v
, let valKey = takeWhile (/='_') iName
, Just iValStr <- [iVals Map.!? valKey]
, [(iVal,"")] <- [reads iValStr] ]
Nothing -> []
I'm trying to parse a csv file where I want to ignore the first line and the last line, as in:
Someheader
foo, 1000,
bah, 2000,
somefooter
I wrote some Haskell using the cassava library:
{-# LANGUAGE OverloadedStrings #-}
import Control.Applicative
import qualified Data.ByteString.Lazy as BL
import Data.Csv
import qualified Data.Vector as V
import Control.Monad (mzero)
data Demand = Demand
{ name :: !String
, amount :: !Int
} deriving Show
instance FromRecord Demand where
parseRecord r
| length == 2 = Demand <$> r .! 0
<*> r .! 1
| otherwise = mzero
main :: IO ()
main = do
csvData <- BL.readFile "demand.csv"
case decode HasHeader csvData of
Left err -> putStrLn err
Right (_, v) -> V.forM_ v $ \ p ->
putStrLn $ (name p) ++ " amount " ++ show (amount p)
When I run this get a type mismatch, that I can't figure out:
parser.hs:34:15: error:
• Couldn't match expected type ‘V.Vector a2’
with actual type ‘(a1, V.Vector Demand)’
• In the pattern: (_, v)
In the pattern: Right (_, v)
My guess is that I haven't unpacked the Vector in the record correctly? Any help, gratefully received.
decode has the type FromRecord a => HasHeader -> ByteString-> Either String (Vector a) based on the documentation for cassava.
So the correct pattern would be Right v instead of Right (_, v).
Another problem in the code, is that length is a function, and you didn't apply it to anything, in the guard | length == 2 = .... I believe the correct code should instead be | length r == 2 = ...
Here's the complete code after those changes:
{-# LANGUAGE OverloadedStrings #-}
import Control.Applicative
import qualified Data.ByteString.Lazy as BL
import Data.Csv
import qualified Data.Vector as V
import Control.Monad (mzero)
data Demand = Demand
{ name :: !String
, amount :: !Int
} deriving Show
instance FromRecord Demand where
parseRecord r
| length r == 2 = Demand <$> r .! 0
<*> r .! 1
| otherwise = mzero
main :: IO ()
main = do
csvData <- BL.readFile "demand.csv"
case decode HasHeader csvData of
Left err -> putStrLn err
Right v -> V.forM_ v $ \ p ->
putStrLn $ (name p) ++ " amount " ++ show (amount p)
I am struggling with a function that would take String JSON as an input and return RoseTree structure as an output.
My code is as follows:
import qualified Data.Aeson as JSON
import qualified Data.Text as T
import qualified Data.Aeson.Types as AT
import Control.Applicative
import qualified Data.ByteString.Char8 as BS
import Data.Attoparsec.ByteString.Char8
data RoseTree a = Empty | RoseTree a [RoseTree a]
deriving (Show)
instance (Show a) => JSON.ToJSON (RoseTree a) where
toJSON (RoseTree n cs) =
JSON.object [T.pack "value" JSON..= show n
, T.pack "children" JSON..= JSON.toJSON cs]
instance (Show a, JSON.FromJSON a) => JSON.FromJSON (RoseTree a) where
parseJSON (JSON.Object o) =
RoseTree <$> o JSON..: T.pack "value"
<*> o JSON..: T.pack "children"
parseRoseTreeFromJSON :: (Show a, JSON.FromJSON a) => String -> (RoseTree a)
parseRoseTreeFromJSON json =
let bs = BS.pack json in case parse JSON.json bs of
(Done rest r) -> case AT.parseMaybe JSON.parseJSON r of
(Just x) -> x
Nothing -> Empty
_ -> Empty
It converts RoseTree structure into JSON perfectly fine. For example, when I run JSON.encode $ JSON.toJSON $ RoseTree 1 [], it returns "{\"children\":[],\"value\":\"1\"}", running JSON.encode $ JSON.toJSON $ RoseTree 1 [RoseTree 2 []] returns "{\"children\":[{\"children\":[],\"value\":\"2\"}],\"value\":\"1\"}".
But when I try to supply that JSON I have got on the previous step into the parser, it always returns Empty:
*Main> parseRoseTreeFromJSON "{\"children\":[],\"value\":1}"
Empty
*Main> parseRoseTreeFromJSON "{\"children\":[{\"children\":[],\"value\":\"2\"}],\"value\":\"1\"}"
Empty
Similarly,
*Main> JSON.decode $ BLS.pack "{\"children\":[{\"children\":[],\"value\":\"2\"}],\"value\":\"1\"}"
Nothing
I suspect that my parser is incorrectly defined. But I cannot tell what exactly is wrong. Is the approach I am using the correct one to approach recursive parsing? What would be the right way to do it recursively?
Weird GHCi typing rules strike again! Everything works as expected if you add type annotations:
ghci> :set -XOverloadedStrings
ghci> JSON.decode "{\"children\":[],\"value\":1}" :: Maybe (RoseTree Int)
Just (RoseTree 1 [])
Without knowing what type you are trying to parse (it sees FromJSON a => Maybe a), GHCi defaults a ~ (). You can try this out by testing the FromJSON () instance out and noticing that it succeeds!
ghci> JSON.decode "[]"
Just ()
Couple side notes on your code if this is actually for a project (and not just for fun and learning):
I encourage you to check out OverloadedStrings (you can basically drop almost all uses of T.pack from your code since string literals will infer whether they should have type String, lazy/strict Text, lazy/strict ByteString etc.)
You can use DeriveGeneric and DeriveAnyClass to get JSON parsing almost for free (although I concede the behavior will be slightly different from what you currently have)
With those suggestions, here is a rewrite of your code:
{-# LANGUAGE OverloadedStrings, DeriveGeneric, DeriveAnyClass #-}
import qualified Data.Aeson as JSON
import qualified Data.ByteString.Lazy.Char8 as BS
import GHC.Generics (Generic)
import Data.Maybe (fromMaybe)
data RoseTree a = RoseTree { value :: a, children :: [RoseTree a] }
deriving (Show, Generic, JSON.FromJSON, JSON.ToJSON)
Note that decode from Data.Aeson does (almost) what parseRoseTreeFromJSON does...
ghci> :set -XOverloadedStrings
ghci> JSON.encode (RoseTree 1 [])
"{\"children\":[],\"value\":1}"
ghci> JSON.encode (RoseTree 1 [RoseTree 2 []])
"{\"children\":[{\"children\":[],\"value\":2}],\"value\":1}"
ghci> JSON.decode "{\"children\":[],\"value\":1}" :: Maybe (RoseTree Int)
Just (RoseTree {value = 1, children = []})
ghci> JSON.decode "{\"children\":[{\"children\":[],\"value\":2}],\"value\":1}" :: Maybe (RoseTree Int)
Just (RoseTree {value = 1, children = [RoseTree {value = 2, children = []}]})