Elm: decoding json from http response and showing it - json

I'm kind of new to Elm and I find it very hard to decode a json from a http response.
The app I'm making is doing a call to gravatar and receives a profile.
I'd like to extract some fields from the response and put in in a record, which in turn in shown in the view.
This is my code:
-- MODEL
type alias MentorRecord =
{ displayName : String
, aboutMe : String
, currentLocation : String
, thumbnailUrl : String
}
type alias Model =
{ newMentorEmail : String
, newMentor : MentorRecord
, mentors : List MentorRecord
}
init : ( Model, Cmd Msg )
init =
( Model "" (MentorRecord "" "" "" "") [], Cmd.none )
-- UPDATE
type Msg
= MentorEmail String
| AddMentor
| GravatarMentor (Result Http.Error MentorRecord)
| RemoveMentor
update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
case msg of
MentorEmail newEmail ->
( { model | newMentorEmail = newEmail }, Cmd.none )
AddMentor ->
( model, getGravatarMentor model.newMentorEmail )
GravatarMentor (Ok addedMentor) ->
( Model "" addedMentor (addedMentor :: model.mentors)
, Cmd.none
)
GravatarMentor (Err _) ->
( model, Cmd.none )
RemoveMentor ->
( model, Cmd.none )
-- VIEW
view : Model -> Html Msg
view model =
div []
[ input [ placeholder "Email adress mentor", onInput MentorEmail ] []
, button [ onClick AddMentor ] [ text "Add Mentor" ]
, br [] []
, img [ src (createIconUrl model.newMentorEmail) ] []
, div [] [ text model.newMentor.displayName ]
, div [] [ toHtmlImgList model.mentors ]
]
toHtmlImgList : List MentorRecord -> Html Msg
toHtmlImgList mentors =
ul [] (List.map toLiImg mentors)
toLiImg : MentorRecord -> Html Msg
toLiImg mentor =
li [] [ img [ src mentor.thumbnailUrl ] [] ]
-- HTTP
getGravatarMentor : String -> Cmd Msg
getGravatarMentor newMentorEmail =
Http.send GravatarMentor
(Http.get (createProfileUrl newMentorEmail) decodeGravatarResponse)
createProfileUrl : String -> String
createProfileUrl email =
"https://en.gravatar.com/" ++ MD5.hex email ++ ".json"
createIconUrl : String -> String
createIconUrl email =
"https://www.gravatar.com/avatar/" ++ MD5.hex email
decodeGravatarResponse : Decoder MentorRecord
decodeGravatarResponse =
let
mentorDecoder =
Json.Decode.Pipeline.decode MentorRecord
|> Json.Decode.Pipeline.required "displayName" string
|> Json.Decode.Pipeline.required "aboutMe" string
|> Json.Decode.Pipeline.required "currentLocation" string
|> Json.Decode.Pipeline.required "thumbnailUrl" string
in
at [ "entry" ] mentorDecoder
If a valid email address if filled in (i.e. one with a gravatar profile), you see the icon. But what this code also should do is extract name, location, about me info, thumbnailUrl from another http response, put it in a list, and show it in the view. And that's not happening if you click on 'Add mentor'
So I guess the decoding part isn't going very well, but I'm not sure (maybe because the nested element is in a list?).
A response from gravatar looks like this (removed some fields in entry):
{ "entry": [
{
"preferredUsername": "bla",
"thumbnailUrl": "https://secure.gravatar.com/avatar/hashinghere",
"displayName": "anne",
"aboutMe": "Something...",
"currentLocation": "Somewhere",
}
]}
Code in Ellie app: https://ellie-app.com/n5dxHhvQPa1/1

entry is an array. To decode the contents of the first element of the array, you need to use Json.Decode.index.
Change:
(at [ "entry" ]) mentorDecoder
to
(at [ "entry" ] << index 0) mentorDecoder
But the bigger problem here is that Gravatar does not support cross origin requests (CORS) but only JSONP. elm-http doesn't support JSONP. You can either use ports for that or use a third party service which enables you to make CORS requests to arbitrary sites. I've used the latter in the ellie link below but you should use ports or your own CORS proxy in a real production application.
I also made aboutMe and currentLocation optional as they weren't present in the profile I checked. Here's the link: https://ellie-app.com/pS2WKpJrFa1/0
The original functions:
createProfileUrl : String -> String
createProfileUrl email =
"https://en.gravatar.com/" ++ MD5.hex email ++ ".json"
decodeGravatarResponse : Decoder MentorRecord
decodeGravatarResponse =
let
mentorDecoder =
Json.Decode.Pipeline.decode MentorRecord
|> Json.Decode.Pipeline.required "displayName" string
|> Json.Decode.Pipeline.required "aboutMe" string
|> Json.Decode.Pipeline.required "currentLocation" string
|> Json.Decode.Pipeline.required "thumbnailUrl" string
in
at [ "entry" ] mentorDecoder
The changed functions:
createProfileUrl : String -> String
createProfileUrl email =
"https://crossorigin.me/https://en.gravatar.com/" ++ MD5.hex email ++ ".json"
decodeGravatarResponse : Decoder MentorRecord
decodeGravatarResponse =
let
mentorDecoder =
Json.Decode.Pipeline.decode MentorRecord
|> Json.Decode.Pipeline.required "displayName" string
|> Json.Decode.Pipeline.optional "aboutMe" string ""
|> Json.Decode.Pipeline.optional "currentLocation" string ""
|> Json.Decode.Pipeline.required "thumbnailUrl" string
in
(at [ "entry" ] << index 0) mentorDecoder

Related

Modify json values from list of maps and save output

I have the following example json that I read in using the play framework.
{
"field_a": "dummy",
"field_b": "dummy",
"nest": {
"nest_a": "dummy",
"nest_b": 87
},
"field_c": null,
"field_d": null,
"field_e": "chocolate",
"field_f": "sugar",
"array": [
"dummy entry"
],
"id": "Anything"
}
I Then have the following List of Maps that I want to swap out data with which is my Input:
val substitutionsList: List[mutable.Map[String, String]] = List(
mutable.Map("field_b" -> "dummy string", "field_d" -> "2016-01-01", "field_f" -> "2011-01-01"),
mutable.Map("field_b" -> "dummy string", "field_d" -> "2018-01-01", "field_f" -> "2018-01-01"),
mutable.Map("field_b" -> "dummy string", "field_d" -> "2018-04-01", "field_f" -> "2018-04-01"),
mutable.Map("field_b" -> "dummy string", "field_d" -> "2016-01-01", "field_f" -> "2016-01-01")
)
I am reading in the json as follows:
def parseSchemaJson(schemaContent: String) = Json.parse(schemaContent).as[JsObject]
val baseSchemaInput = parseSchemaJson(Source.fromFile("/dummy.json").mkString)
I want to iterate over my Input and swap out the values in the json for the values in my map and after each one is done, create a new .json file.
private def replaceField(json: JsObject, fieldToReplace: String): Option[String] = (json \ fieldToReplace).asOpt[String]
println(replaceField(baseSchemaInput, "field_a")) //prints dummy
I can list out the value in my json using something like this but I have no idea how to swap the value from my list into each respective bit and write out a json file.
First occurance of the expected output
{
"field_a": "dummy",
"field_b": "dummy string",
"nest": {
"nest_a": "dummy",
"nest_b": 87
},
"field_c": null,
"field_d": "2016-01-01",
"field_e": "chocolate",
"field_f": "2011-01-01",
"array": [
"dummy entry"
],
"id": "Anything"
}
Given the following substitution list:
val substitutionsList: List[mutable.Map[String, Any]] = List(
mutable.Map("field_b" -> "dummy string 1", "field_d" -> "2016-01-01", "field_f" -> "2011-01-01"),
mutable.Map("field_b" -> "dummy string 2", "nest" -> mutable.Map("nest_b" -> "90"))
)
Note that the second one updates a nested value. You need to create a nested Map to define nested values.
You can define a function to transform a Map[String, Any] (Any because we can have either a String or a Map as value) into a JsObject. This is a recursive function and will call itself in case the Value is a Map
def mapToJsObject(map: mutable.Map[String, Any]): JsObject =
JsObject(map.mapValues {
case v:String => JsString(v)
case v:mutable.Map[String, mutable.Map[String, Any]] => mapToJsObject(v.asInstanceOf[mutable.Map[String, Any]])
})
Then go through your list of substitutions and deep merge the JsObject made from each Map with the original JsObject using the deepMerge function defined on the JsObject class. It will merge the nested objects too. See API here
val substitutedJsObjects: List[JsObject] = substitutionsList
.map(mapToJsObject)
.map(baseSchemaInput.deepMerge)
This should give you a list of JsObjects, one per Map in your List
You can then write them to file. Here is an example to write one file per json string. Files will be named 0.json, 1.json, etc ..
def writeToFile(jsObject: JsObject, fileName: String): Unit = {
println("writing "+fileName)
val pw = new PrintWriter(new File(fileName))
pw.write(jsObject.toString())
pw.close()
}
substitutedJsObjects.zipWithIndex.foreach {
case (jsObject, index) => {
val fileName = index.toString + ".json"
writeToFile(jsObject, fileName)
}
}

Elm Json Decoder Pipeline error

I'm trying to decode some json coming in from an http request but I keep running into syntax issues. This is the error I get from the compiler:
-- TYPE MISMATCH ------------------------------------------------------ [7/1811$
The 2nd argument to function `send` is causing a mismatch.
65| Http.send CardFetch (Http.get url modelDecoder)
^^^^^^^^^^^^^^^^^^^^^^^^^
Function `send` is expecting the 2nd argument to be:
Http.Request String
But it is:
Http.Request Model
Here is my code:
module Main exposing (..)
import Html exposing (..)
import Html.Attributes exposing (..)
import Html.Events exposing (..)
import Http
import Json.Decode as Decode exposing (string, Decoder, at, index)
import Json.Decode.Pipeline exposing (..)
main : Program Never Model Msg
main =
program
{ init = init
, view = view
, update = update
, subscriptions = subscriptions
}
type alias Model =
{ boardName : String
, cardName : String
}
init =
( Model "Default Board" "Default Card"
, Cmd.none
)
-- UPDATE
type Msg
= CardFetch (Result Http.Error String)
| DataFetch
update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
case msg of
DataFetch ->
( model, getData )
CardFetch (Ok incomingName) ->
( Model model.cardName incomingName, Cmd.none )
CardFetch (Err errorMessage) ->
( model, Debug.log "Errors" Cmd.none )
-- HTTP
url : String
url =
"https://api.trello.com/1/members/user/actions?limit=3&key=..."
getData =
Http.send CardFetch (Http.get url modelDecoder)
{--decodeCard =
Decode.index 0
modelDecoder
(Decode.at
[ "data", "card", "name" ]
string
)
--}
modelDecoder : Decoder Model
modelDecoder =
decode Model
|> custom (index 0 (at [ "data", "card", "name" ] string))
|> custom (index 0 (at [ "data", "board", "name" ] string))
--UPDATE
-- VIEW
view : Model -> Html Msg
view model =
div []
[ div []
[ button [ onClick DataFetch ] [ text "Get Card" ] ]
, div [ class "card" ]
[ h3 [] [ text model.boardName ]
, div [ class "board" ] [ h4 [] [ text model.cardName ] ]
]
]
-- SUBSCRIPTIONS
subscriptions : Model -> Sub Msg
subscriptions model =
Sub.none
I'm am fairly new to Elm and I'm trying to figure out how API calls work. Elm documentation is excellent but the bit about API calls is kind of vague. I would appreciate any help I can get. Thanks a lot!
You declared in your messages:
CardFetch (Result Http.Error String)
which means that successful response will produce String. However, your modelDecoder is returning Model: modelDecoder : Decoder Model.
Changing your message declaration to:
CardFetch (Result Http.Error Model)
and in update function:
CardFetch (Ok incomingName) ->
( incomingName, Cmd.none )
should help.

Decode JSON Multiway Tree into an F# Multiway Tree Discriminated Union

I have the following JSON data in a documentdb and I would like to parse this into an F# multiway tree discriminated union
"commentTree": {
"commentModel": {
"commentId": "",
"userId": "",
"message": ""
},
"forest": []
}
F# multiway discriminated union
type public CommentMultiTreeDatabaseModel =
| CommentDatabaseModelNode of CommentDatabaseModel * list<CommentMultiTreeDatabaseModel>
where CommentMultiTreeDatabaseModel is defined as
type public CommentDatabaseModel =
{ commentId : string
userId : string
message : string
}
I am referencing Fold / Recursion over Multiway Tree in f# extensively. I am not sure where to begin to parse such a JSON structure into an F# multiway tree. Any suggestions will be much appreciated. Thanks
One way to think about this is by looking at what data you need in order to construct a CommentMultiTreeDatabaseModel. It needs a CommentDatabaseModel and a list of CommentMultiTreeDatabaseModel. So we need to write the following two functions:
let parseComment (input : JSON) : CommentDatabaseModel =
...
let parseTree (input : JSON) : CommentMultiTreeDatabaseModel =
...
But wait, the parseTree function is the one we're trying to write right now! So instead of writing a new function, we just mark our current function with the rec keyword and have it call itself where needed.
Below is a rough example of how it could be done. The key thing to look at is parseTree which builds up the data by recursively calling itself. I've represented the JSON input data with a simple DU. A library like Chiron can produce something like this.
Note that this code parses all of the JSON in one go. Also, it's not tail-recursive, so you'll have to be careful with how deep your tree structure is.
[<RequireQualifiedAccess>]
type JSON =
| String of string
| Object of (string * JSON) list
| Array of JSON list
type public CommentDatabaseModel = {
commentId : string
userId : string
message : string
}
type public CommentMultiTreeDatabaseModel =
| CommentDatabaseModelNode of CommentDatabaseModel * list<CommentMultiTreeDatabaseModel>
let parseComment = function
| JSON.Object [ "commentId", JSON.String commentId; "userId", JSON.String userId; "message", JSON.String message ] ->
{
commentId = commentId
userId = userId
message = message
}
| _ -> failwith "Bad data"
let rec parseTree (input : JSON) : CommentMultiTreeDatabaseModel =
match input with
| JSON.Object [ "commentModel", commentModel; "forest", JSON.Array forest ] ->
CommentDatabaseModelNode (parseComment commentModel, List.map parseTree forest)
| _ -> failwith "Bad data"
let parse (input : JSON) : CommentMultiTreeDatabaseModel =
match input with
| JSON.Object [ "commentTree", commentTree ] ->
parseTree commentTree
| _ -> failwith "Bad data"
let comment text =
JSON.Object [
"commentId", JSON.String ""
"userId", JSON.String ""
"message", JSON.String text
]
let sampleData =
JSON.Object [
"commentTree", JSON.Object [
"commentModel", comment "one"
"forest", JSON.Array [
JSON.Object [
"commentModel", comment "two"
"forest", JSON.Array []
]
JSON.Object [
"commentModel", comment "three"
"forest", JSON.Array []
]
]
]
]
parse sampleData
(*
val it : CommentMultiTreeDatabaseModel =
CommentDatabaseModelNode
({commentId = "";
userId = "";
message = "one";},
[CommentDatabaseModelNode ({commentId = "";
userId = "";
message = "two";},[]);
CommentDatabaseModelNode ({commentId = "";
userId = "";
message = "three";},[])])
*)

Conditional JSON decoding based on a field value

I have a need to decode JSON into an elm type like below:
Type
type User = Anonymous | LoggedIn String
type alias Model =
{ email_id : User
, id : Id
, status : Int
, message : String
, accessToken : AccessToken
}
JSON Message 1
{
"status": 0,
"message": "Error message explaining what happened in server"
}
into type value
Model {
"email_id": Anonymous
, id: 0
, status: 0
, message: json.message
, accessToken: ""
}
JSON Message 2
{
"status": 1,
"email_id": "asdfa#asdfa.com"
"token": "asdfaz.adfasggwegwegwe.g4514514ferf"
"id": 234
}
into type value
Model {
"email_id": LoggedIn json.email_id
, id: json.id
, status: json.status
, message: ""
, accessToken: json.token
}
Decoder information
Above, "message" is not always present and email_id/id/token are always not present.
How to do this type of conditional decoding in elm
Json.Decode.andThen lets you do conditional parsing based on the value of a field. In this case, it looks like you'll first want to pull out the value of the "status" field, andThen handle it separately based on whether it is a 1 or 0.
Edit 2016-12-15: Updated to elm-0.18
import Html as H
import Json.Decode exposing (..)
type User = Anonymous | LoggedIn String
type alias Id = Int
type alias AccessToken = String
type alias Model =
{ email_id : User
, id : Id
, status : Int
, message : String
, accessToken : AccessToken
}
modelDecoder : Decoder Model
modelDecoder =
(field "status" int) |> andThen modelDecoderByStatus
modelDecoderByStatus : Int -> Decoder Model
modelDecoderByStatus status =
case status of
0 ->
map5
Model
(succeed Anonymous)
(succeed 0)
(succeed status)
(field "message" string)
(succeed "")
1 ->
map5
Model
(map LoggedIn (field "email_id" string))
(field "id" int)
(succeed status)
(succeed "")
(field "token" string)
_ ->
fail <| "Unknown status: " ++ (toString status)
main = H.div []
[ H.div [] [ decodeString modelDecoder msg1 |> Result.toMaybe |> Maybe.withDefault emptyModel |> toString |> H.text ]
, H.div [] [ decodeString modelDecoder msg2 |> Result.toMaybe |> Maybe.withDefault emptyModel |> toString |> H.text ]
]
emptyModel = Model Anonymous 0 0 "" ""
msg1 = """
{
"status": 0,
"message": "Error message explaining what happened in server"
}
"""
msg2 = """
{
"status": 1,
"email_id": "asdfa#asdfa.com"
"token": "asdfaz.adfasggwegwegwe.g4514514ferf"
"id": 234
}
"""

How to remove List() and escape characters from response JSON in Scala

I have the following function that takes in a JSON input and validates it against a JSON-Schema using the "com.eclipsesource" %% "play-json-schema-validator" % "0.6.2" library. Everything works fine expect for when I get an invalid JSON, I try to collect all violations into a List and later return that list along with the response JSON. However my List is encoded with List() and also has escape characters. I want to have the response JSON look like this:
{
"transactionID": "123",
"status": "error",
"description": "Invalid Request Received",
"violations": ["Wrong type. Expected integer, was string.", "Property action missing"]
}
Instead of this: (This is what I am getting right now)
{
"transactionID": "\"123\"",
"status": "error",
"description": "Invalid Request Received",
"violations": "List(\"Wrong type. Expected integer, was string.\", \"Property action missing\")"
}
And here's the actual function for JSON validation
def validateRequest(json: JsValue): Result = {
{
val logger = LoggerFactory.getLogger("superman")
val jsonSchema = Source.fromFile(play.api.Play.getFile("conf/schema.json")).getLines.mkString
val transactionID = (json \ "transactionID").get
val result: VA[JsValue] = SchemaValidator.validate(Json.fromJson[SchemaType](
Json.parse(jsonSchema.stripMargin)).get, json)
result.fold(
invalid = { errors =>
var violatesList = List[String]()
var invalidError = Map("transactionID" -> transactionID.toString(), "status" -> "error", "description" -> "Invalid Request Received")
for (msg <- (errors.toJson \\ "msgs"))
violatesList = (msg(0).get).toString() :: violatesList
invalidError += ("violations" -> (violatesList.toString()))
//TODO: Make this parsable JSON list
val errorResponse = Json.toJson(invalidError)
logger.error("""Message="Invalid Request Received" for transactionID=""" + transactionID.toString() + "errorResponse:" + errorResponse)
BadRequest(errorResponse)
},
valid = {
post =>
db.writeDocument(json)
val successResponse = Json.obj("transactionID" -> transactionID.toString, "status" -> "OK", "message" -> ("Valid Request Received"))
logger.info("""Message="Valid Request Received" for transactionID=""" + transactionID.toString() + "jsonResponse:" + successResponse)
Ok(successResponse)
}
)
}
}
UPDATE 1
I get this after using Json.obj()
{
"transactionID": "\"123\"",
"status": "error",
"description": "Invalid Request Received",
"violations": [
"\"Wrong type. Expected integer, was string.\"",
"\"Property action missing\""
]
}
I got the escape characters removed by modifying this line:
violatesList = (msg(0).get).toString() :: violatesList
TO:
violatesList = (msg(0).get).as[String] :: violatesList
What you want is a JSON array, but by calling .toString() on your list, you're actually passing a string. Play has an implicit serializer for Lists to JSON arrays, so you actually just have to do less then what you already did - you can just remove the toString() part from violatesList.toString().
In addition, don't create a map for your JSON and then convert it to a JSON, you can use Json.obj with a very similar syntax instead:
val invalidError = Json.obj("transactionID" -> transactionID, "status" -> "error", "description" -> "Invalid Request Received")
for (msg <- (errors.toJson \\ "msgs"))
violatesList = (msg(0).get) :: violatesList
val errorResponse = invalidError ++ Json.obj("violations" -> violatesList)
Regarding your escaped quotes, I suppose it's because transactionID and msgs are JsStrings, so when you convert them with toString() the quotes are included. Just remove the toString everywhere and you'll be fine.