Map a nested JSON payload to a struct in Elixir - json

I am attempting to port the Golang tutorial geddit to Elixir. I have done so successfully with Dartlang, but Elixir's operations on maps & lists are confusing for me.
Using HTTPoison and JSEX, I have the following code:
defmodule Redditex do
use HTTPoison.Base
def process_url(url) do
"http://www.reddit.com/r/#{url}.json"
end
def process_response_body(body) do
json = JSEX.decode! body
json = Enum.map json, fn ({k, v}) -> {String.to_atom(k), v } end
json
end
end
My difficulty is parsing the JSON body into an appropriate struct where the JSON contains nested data. Jazz has some allusion to mapping to a structure but not with nested data.
Is there an example or a common practice to decode JSON in Elixir similar to Go's usage:
type Response struct {
Data struct {
Children []struct {
Data Item
}
}
}
type Item struct {
Title string
URL string
Comments int `json:"num_comments"` #mapping to another field label
}

Using the Poison JSON library I was able to get partly there for handling the nesting:
def handle_response(%{status_code: 200, body: body}) do
json = Poison.decode!(body, as: %{"data" => %{"children" => [%{"data" => Redditex.Item}]}})
items = Enum.map( json["data"]["children"], fn (x) -> x["data"] end )
end
Enumertion is necessary to remove the anonymous structs and the remapping of the field names has not shown as a native solution. A work path forward nonetheless.

Related

Convert serde_json Value keys to camelCase

I'm writing a CLI tool that reads JSON files and is supposed to convert the JSON object keys into camelCase.
Because this should work with any JSON file, I obviously can't just use strong typing and then #[serde(rename_all = "camelCase")].
I can't seem to find an obvious way in serde_json to make it use the already existing renaming code that serde clearly has and apply it to a serde_json::value::Value.
Am I missing something obvious?
You'll have to write a function that recurses through the serde_json::Value structure and replaces the keys of serde_json::Map whenever it encounters one. That's a bit awkward to implement, as there is no Map::drain.
fn rename_keys(json: &mut serde_json::Value) {
match json {
serde_json::Value::Array(a) => a.iter_mut().for_each(rename_keys),
serde_json::Value::Object(o) => {
let mut replace = serde_json::Map::with_capacity(o.len());
o.retain(|k, v| {
rename_keys(v);
replace.insert(
heck::ToLowerCamelCase::to_lower_camel_case(k.as_str()),
std::mem::replace(v, serde_json::Value::Null),
);
true
});
*o = replace;
}
_ => (),
}
}
use std::io::Read;
fn main() {
let mut stdin = vec![];
std::io::stdin()
.read_to_end(&mut stdin)
.expect("Read stdin");
let mut json = serde_json::from_slice::<serde_json::Value>(&stdin).expect("Parse Json");
rename_keys(&mut json);
println!("{}", serde_json::to_string_pretty(&json).unwrap());
}
(Note that rename_keys will produce a stack overflow on deep JSON structures, but serde_json only parses to a limited depth by default, so no need to worry. If you do need support for deeply nested structures, have a look at serde_stacker.)
If you're not interested in the serde_json::Value itself and just want to transform a JSON string, there's two more ways to go on about this:
You could do the renaming on serialization, by writing a custom serializer for a wrapper struct around serde_json::Value. An example of such a serializer is here, but you'd have to adopt it to be recursive. (Possibly, doing it at deserialization might be easier than at serialization)
Write a JSON tokenizer (or grab a crate that contains one) to skip creating the actual serde_json::Value structure and to the renaming on the token stream (no need to worry when working with GBs of JSON)

How to convert response body in Json using play framework

override def accessToken(): ServiceCall[RequestTokenLogIn, Done] = {
request=>
val a=request.oauth_token.get
val b=request.oauth_verifier.get
val url=s"https://api.twitter.com/oauth/access_token?oauth_token=$a&oauth_verifier=$b"
ws.url(url).withMethod("POST").get().map{
res=>
println(res.body)
}
The output which I am getting on terminal is
oauth_token=xxxxxxxxx&oauth_token_secret=xxxxxxx&user_id=xxxxxxxxx&screen_name=xxxxx
I want to convert this response in json format.like
{
oauth_token:"",
token_secret:"",
}
When Calling res.json.toString its not converting into jsValue.
Is there any other way or am I missing something?
According to the documentation twitter publishes, it seems that the response is not a valid json. Therefore you cannot convert it automagically.
As I see it you have 2 options, which you are not going to like. In both options you have to do string manipulations.
The first option, which I like less, is actually building the json:
print(s"""{ \n\t"${res.body.replace("=", "\": \"").replace("&", "\"\n\t\"")}" \n}""")
The second option, is to extract the variables into a case class, and let play-json build the json string for you:
case class TwitterAuthToken(oauth_token: String, oauth_token_secret: String, user_id: Long, screen_name: String)
object TwitterAuthToken {
implicit val format: OFormat[TwitterAuthToken] = Json.format[TwitterAuthToken]
}
val splitResponse = res.body.split('&').map(_.split('=')).map(pair => (pair(0), pair(1))).toMap
val twitterAuthToken = TwitterAuthToken(
oauth_token = splitResponse("oauth_token"),
oauth_token_secret = splitResponse("oauth_token_secret"),
user_id = splitResponse("user_id").toLong,
screen_name = splitResponse("screen_name")
)
print(Json.toJsObject(twitterAuthToken))
I'll note that Json.toJsObject(twitterAuthToken) returns JsObject, which you can serialize, and deserialize.
I am not familiar with any option to modify the delimiters of the json being parsed by play-json. Given an existing json you can manipulate the paths from the json into the case class. But that is not what you are seeking for.
I am not sure if it is requires, but in the second option you can define user_id as long, which is harder in the first option.

Play Framework Json Validation

I'm trying to parse a json structure, validate it and use the validated result.
def createEntry = Action(parse.tolerantJson) { request =>
request.body.validate[MyJson].map { myJson =>
// do something with the MyJson object
}.recoverTotal { err => BAD_REQUEST }
The MyJson object looks like this:
case class MyJson(
complexType: ComplexType,
strs: Seq[String],
maps: Map[String, ComplexType]
)
case class ComplexType(
str1: String,
bool1: Boolean,
cxType2: ComplexType2,
maps: Map[String, String]
)
case class ComplexType2(str: String, strs: Seq[String])
Would the validate method automatically try to box the JSON string into the object type? os Should I write an additional body parser?
Assuming you have the appropriate Format or Reads objects in scope for MyJson, yes, response.body.validate will automatically parse the JSON string into a MyJson instance, only entering the recoverTotal block if the JSON validation fails (i.e., the JSON in the request cannot be parsed into a MyJson, even if it is a valid JSON string).

Play Framework 2.2.1 / How should controller handle a queryString in JSON format

My client side executes a server call encompassing data (queryString) in a JSON object like this:
?q={"title":"Hello"} //non encoded for the sample but using JSON.stringify actually
What is an efficient way to retrieve the title and Hello String?
I tried this:
val params = request.queryString.map {case(k,v) => k->v.headOption}
that returns the Tuple: (q,Some({"title":"hello"}))
I could further extract to retrieve the values (although I would need to manually map the JSON object to a Scala object), but I wonder whether there is an easier and shorter way.
Any idea?
First, if you intend to pluck only the q parameter from a request and don't intend to do so via a route you could simply grab it directly:
val q: Option[String] = request.getQueryString("q")
Next you'd have to parse it as a JSON Object:
import play.api.libs.json._
val jsonObject: Option[JsValue] = q.map(raw: String => json.parse(raw))
with that you should be able to check for the components the jsonObject contains:
val title: Option[String] = jsonObject.flatMap { json: JsValue =>
(json \ "title").asOpt[String]
}
In short, omitting the types you could use a for comprehension for the whole thing like so:
val title = for {
q <- request.getQueryString("q")
json <- Try(Json.parse(q)).toOption
titleValue <- (json \ "title").asOpt[String]
} yield titleValue
Try is defined in scala.util and basically catches Exceptions and wraps it in a processable form.
I must admit that the last version simply ignores Exceptions during the parsing of the raw JSON String and treats them equally to "no title query has been set".
That's not the best way to know what actually went wrong.
In our productive code we're using implicit shortcuts that wraps a None and JsError as a Failure:
val title: Try[String] = for {
q <- request.getQueryString("q") orFailWithMessage "Query not defined"
json <- Try(Json.parse(q))
titleValue <- (json \ "title").validate[String].asTry
} yield titleValue
Staying in the Try monad we gain information about where it went wrong and can provide that to the User.
orFailWithMessage is basically an implicit wrapper for an Option that will transform it into Succcess or Failure with the specified message.
JsResult.asTry is also simply a pimped JsResult that will be Success or Failure as well.

JSON to Map[String,JsObject] with Play 2.0?

I'm new to both Play! & Scala, but I'm trying to make a service that will map the JSON request to a Map[String,JsObject] (or Map[String,JsValue], I'm not sure about the distinction), and then output a list of the keys recursively through the map (preferably as a tree).
But I'm having start issues:
def genericJSONResponse = Action(parse.json) {
request => request.body
var keys = request.keys
Ok("OK")
}
What I would expect here was for keys to be filled with the keys from the request, but of course, it doesn't compile. How should I approach this, given the description above?
Thanks in advance for helping out a Scala noob :-)
Nik
JsValue is the base class for all JSON values. JsObject is a subtype of JsValue (along with JsNull, JsUndefined, JsBoolean, JsNumber, JsString, and JsArray). Take a look at JSON spec if it's unclear: http://json.org/
If you know that the JSON in the body request is a JSON object (as opposed to other types listed above) you can pattern-match it:
def genericJSONResponse = Action(parse.json) { request =>
request.body match {
case JsObject(fields) => Ok("received object:" + fields.toMap + '\n')
case _ => Ok("received something else: " + request.body + '\n')
}
}
fields.toMap is of type you wanted: Map[(String, JsValue)] so you can use map or collect to process the object's keys recursively. (By the way, you can use fields directly, since it's a Seq[(String, JsValue)] and supports map and collect as well).