I'm trying to parse Json in Kotlin. I'm having a lot of trouble, it seems that a lot of people learn Kotlin after Java... Not me, I'm a Python guy. I got a Kotlin Jupyter Notebook running fairly quickly (https://github.com/ligee/kotlin-jupyter), after that I managed to pull information from the bittrex api like so:
import java.net.URL
val result = URL("https://bittrex.com/api/v1.1/public/getmarkets").readText()
It took me a long time to find that I needed to add import java.net.URL, this always seems to be implicit in all code examples. Anyway, this give me a response in json (the "result parameter"):
{"success":true,"message":"","result":[{"MarketCurrency":"LTC","BaseCurrency":"BTC","MarketCurrencyLong":"Litecoin","BaseCurrencyLong":"Bitcoin","MinTradeSize":0.01469482,"MarketName":"BTC-LTC","IsActive":true,"Created":"2014-02-13T00:00:00","Notice":null,"IsSponsored":null,"LogoUrl":"https://bittrexblobstorage.blob.core.windows.net/public/6defbc41-582d-47a6-bb2e-d0fa88663524.png"},{"MarketCurrency":"DOGE","BaseCurrency":"BTC","MarketCurrencyLong":"Dogecoin","BaseCurrencyLong":"Bitcoin","MinTradeSize":274.72527473,"MarketName":"BTC-DOGE","IsActive":true,"Created":"2014-02-13T00:00:00","Notice":null,"IsSponsored":null,"LogoUrl":"https://bittrexblobstorage.blob.core.windows.net/public/a2b8eaee-2905-4478-a7a0-246f212c64c6.png"},{"MarketCurrency ...
Now, in Python I'd just add .json() to the "result" parameter and I can then address the json fields as a dictionary with multiple levels, like
result["success"]
Would give me:
true
Is there something like that for Kotlin? I have tried Klaxon https://github.com/cbeust/klaxon, again it took me a lot of time to realize that I have to do import com.beust.klaxon.string, it is not mentioned on the website for example, so a side question is: How do you know what you need to import when you find code examples? It seems like everybody just knows... But I digress.
My main question is: How can I address the separate fields of the Json and get them into separate variables?
Highest regards.
There are many JSON parsers out there. Your example was a Kotlin explicit one and that is not mandatory for Kotlin because there are also many basic Java parsers, which you can use just as fine in Kotlin.
For your imports. Obviously you need to import the classes you want to use and IDE's like IntelliJ handle the imports for you automatically. That means that you won't have to type any import statements, but they are added automatically when referencing those classes.
I think that nowadays some libraries just expect that you do not handle the imports yourself and thus do not assist you on finding the right imports.
My suggestion for a parser is Fuel.
The library is optimized for Kotlin as well. Your problem would be solved with this simple code snippet with the help of Fuel:
"https://bittrex.com/api/v1.1/public/getmarkets".httpGet().responseJson { _, response, result ->
if (response.responseMessage == "OK" && response.statusCode == 200) {
val yourResult = result.get().obj().getBoolean("success")
}
}
Something you may or may not know is that Kotlin is 100% compatible with Java, thus all the Java json parsers work well with Kotlin. I highly recommend gson. It's small (~200kb), fast, and pretty simple to use.
If this code is running in a server, jackson is pretty standard. It's the most performant json parser for java at the moment, but it's very heavy. It will take some more complicated configuration though, and I think it might require some Kotlin specific modules.
I haven't tried it yet, as it hasn't officially been released, but Kotlin offers a plugin for generating json serialization code. That will probably eventually be the standard way for Kotlin to serialize / deserialize as it should theoretically be the most performant.
The best and quick practice is instead of manually checking for each key, generate native Kotlin "data classes" using tools e.g. https://json2kotlin.com
So your API response turns into the following couple of data classes corresponding to the JSON structure:
data class Json4Kotlin_Base (
val success : Boolean,
val message : String,
val result : List<Result>
)
and
data class Result (
val marketCurrency : String,
val baseCurrency : String,
val marketCurrencyLong : String,
val baseCurrencyLong : String,
val minTradeSize : Double,
val marketName : String,
val isActive : Boolean,
val isRestricted : Boolean,
val created : String,
val notice : String,
val isSponsored : String,
val logoUrl : String
)
When you get the result, you simple map the JSON response to these data classes. The video here shows how to do it step by step.
Related
I would like to capture JSON typos and other schema violations. I originally wanted to use circe-json-schema but the scala version provided in our execution environment is 2.11 which is not supported by said library. I turned to semi-automatic decoding but it, unfortunately, ignores any "additional" attributes in the incoming JSON. Example:
val json = json"""{ "pepe": "corre", "tito": "tira" }"""
case class Pepe(pepe: String)
implicit val pepeDecoder: Decoder[Pepe] = deriveDecoder
/* ideally, because "tito" is extra but the assertion nevertheless fails */
assert(json.as[Pepe] == Left("decoding failure"))
My other option would be to manually traverse the whole JSON w/ a Map of expectations but I think it pedestrian. If there are any other options w/in the circe world, including coexisting w/ a higher scala major version (in order to leverage circe-json-schema) w/in Maven, please advise.
I have a json-schema (https://json-schema.org) with recursive fields, and I would like to programmatically parse json that adheres to the schema in Scala.
One option is to use Argus (https://github.com/aishfenton/Argus), but the only issue is that it uses Scala macros, so a solution that uses this library isn't supported by IntelliJ.
What's the recommended way to perform a task like this in Scala, preferably something that plays well with IntelliJ?
Circe is a great library for working with JSON. The following example uses semi automatic decoding. Circe also has guides for automatic decoding and for using custom codecs.
import io.circe.Decoder
import io.circe.parser.decode
import io.circe.generic.semiauto.deriveDecoder
object Example {
case class MyClass(name: String, code: Int, sub: MySubClass)
case class MySubClass(value: Int)
implicit val myClassDecoder: Decoder[MyClass] = deriveDecoder
implicit val mySubClassDecoder: Decoder[MySubClass] = deriveDecoder
def main(args: Array[String]): Unit = {
val input = """{"name": "Bob", "code": 200, "sub": {"value": 42}}"""
println(decode[MyClass](input).fold(_ => "parse failed", _.toString))
}
}
Have you look at https://github.com/circe/circe , it is pretty good to parse Json with typed format.
I don't know what you mean with recursive fields. But there's lots of different libraries for parsing json. You could use lift-json
https://github.com/lift/framework/tree/master/core/json
Which seems popular, at least from what I've seen here on Stackoverflow. But I personally am very comfortable with and prefer play.json
https://www.playframework.com/documentation/2.6.x/ScalaJson#Json
(Also, I use IntelliJ and work in the Play-framework)
If you really don't want to use any special libraries, someone tried to do that here
How to parse JSON in Scala using standard Scala classes?
I found some code which can parse JSON document, convert it to BSON and then insert. But this code is implemented using Java classes in casbah. I could not find corresponding implementation in Scala.
Also casbah documentation says "In general, you should only need the org.mongodb.scala and org.bson namespaces in your code. You should not need to import from the com.mongodb namespace as there are equivalent type aliases and companion objects in the Scala driver. The main exception is for advanced configuration via the builders in MongoClientSettings which is considered to be for advanced users."
If you see below code and note imports, they are using com.mongodb classes. I can use below code in scala and make it work, but I want to know if there is Scala implementation out there to insert JSON into mongodb.
import com.mongodb.DBObject
import com.mongodb.casbah.MongoClient
import com.mongodb.casbah.MongoClientURI
import com.mongodb.util.JSON
val jsonString = """{"card_id" : 75893645814809,"cust_id": 1008,"card_info": {"card_type" : "Travel Card","credit_limit": 126839},"card_dates" : [{"date":"1997-09-09" },{"date":"2007-09-07" }]}"""
val dbObject: DBObject = JSON.parse(jsonString).asInstanceOf[DBObject]
val mongo = MongoClient(MongoClientURI("mongodb://127.0.0.1:27017"))
val buffer = new java.util.ArrayList[DBObject]()
buffer.add(dbObject)
mongo.getDB("yourDBName").getCollection("yourCollectionName").insert(buffer)
buffer.clear()
Reference : Scala code to insert JSON string to mongo DB
I found few link online which suggests to use different JSON parser libraries, but none of them seems straightforward even though above ~5 lines of code can insert JSON document in Java. I would like to achieve similar thing in Java.
I need to validate the schema of certain JSON input I receive. I am not clear about how to go about the whole thing. But this is what I have gathered so far :
I need to prepare a schema for all sorts of input using something like http://json-schema.org/implementations.html
Then I need a validator like https://github.com/fge/json-schema-validator
I need to give the json input to and the schema to the validator and get the result.
However my question is that I need to use a jar which I can import and use of the json-schema-validator https://github.com/fge/json-schema-validator . Also I am not clear on how to use it. I don't know the format it accepts , the classes and methods needed etc.
How good the validator's support is for Scala?
I would not go through the pain of manually collecting the jars of json schema validator (done that, not fun). It is better use a tool for that (like maven, sbt, gradle or ivy).
In case you want to use it in an OSGi environment, you might need to use a different (probably not up-to-date) version.
Usage:
val factory: JsonSchemaFactory = JsonSchemaFactory.getDefault
val validator: JsonValidator = factory.getValidator
val schemaJson: com.fasterxml.jackson.databind.JsonNode = yourJsonSchemaInJackson2Format
val report: ProcessingReport = validator.validate(schemaJson, yourJsonInJackson2Format)
//check your report.
PS.: In case you want to collect the dependencies manually, you can go through the dependencies transitively starting on this page.
There is Orderly, liftweb-json-based JSON validator implementation for Scala:
import com.nparry.orderly._
import net.liftweb.json.JsonAST._
val orderly = Orderly("integer {0,100};")
val noProblems = orderly.validate(JInt(50))
val notAllowed = orderly.validate(JInt(200))
Use net.liftweb.json.parse(s: String): JValue to obtain JValue from String.
I noticed that orderly4jvm does not support the latest JSON Schema version 4, which causes problems if you want to use it to generate the JSON schema.
I seem to be confused when using the play framework on how to deserialize json correctly. With jerkson it looks like you just have to define a case class which then automatically deserializes a json string (stolen from the jerkson docs).
case class Person(id: Long, name: String)
parse[Person]("""{"id":1,"name":"Coda"}""") //=> Person(1,"Coda")
But, with play framework you have to write a lot of boiler plate code to do the same thing. For instance from their documentation.
case class Foo(name: String, entry: Int)
object Foo {
implicit object FopReads extends Format[Foo] {
def reads(json: JsValue) = Foo(
(json \ "name").as[String],
(json \ "entry").as[Int])
def writes(ts: Foo) = JsObject(Seq(
"name" -> JsString(ts.name),
"entry" -> JsNumber(ts.entry)))
}
}
This seems like a lot more work, so i assume i'm either not using it correctly or don't quite understand the advantage of doing it this way. Is there a short cut so that I don't have to write all of this code? If not, should I just be using jerkson in my Action to parse a incoming json string? It seems as though asText is returning a blank string, even when asJson works just fine...which leads me to believe I am definitely doing something wrong.
Thanks
I think there are two answers to your question.
For somewhat less boiler plate, you can use the the Play support for handling case classes. Here's an example for a case class with three fields:
implicit val SampleSetFormat: Format[SampleSet] = productFormat3("sensorId", "times", "values")(SampleSet)(SampleSet.unapply)
I agree that there is more annoying boiler plate, the main reason the Play folks seem to use this approach so they can determine the correct serializer entirely at compile time. So no cost of reflecting as in Jerkson.
I am a total noob with Play and Jerkson, but wholeheartedly recommend doing the least boilerplate approach (using Jerkson lib within each Action). I find that it is philosophically more in line with Scala to do so and it works fine.