I am looking for help in reading/writing of JSON string in my DB in a backward compatible way.
The JSON names and objects are in case class FeatureList.
Old format:
"featureRequired":"Yes"
New format:
"featureRequired": {"thisfeatureRequired" :"Yes", "featureDetails": "Some text"}
case class FeatureList (
field1 : type,
.
.
.
featureRequired: Boolean //this is field22
)
The Issue : The legacy code reads featureRequired as a string. Then converts to boolean for decision making. For my new requirement, featureRequired is an object of {thisfeatureRequired, featureDetails}
We use scala play library for json manipulation.
The question : For my requirement, how do I READ the new format, which is backward compatible with the old one? Will custom formatter help? Can someone give an example please.
Related
I am using a service builder that is retrieving the form data fine from mysql db. I have a field that has the json data and I tried to map it using object mapper and using com.fasterxml.jackson.databind.ObjectMapper to display the json content. However, the Blob Data is shown as: com.mysql.cj.jdbc.Blob#4ca74f7f
How do I actually get/extract the data from the storing link above? Here is my code snippet:
for (ddmcontent MyTemp : myList) {
System.out.println("Content ID : "+myList.getContentId());
System.out.println("User Blob Data : "+myList.getData());
Blob responseBody =myList.getData();
ObjectMapper objectMapper = new ObjectMapper();
List<ModelData> myDat = objectMapper.readValue((DataInput)
responseBody,objectMapper.getTypeFactory().constructCollectionType
(List.class,ModelData.class));
for (ModelData dt : myDat) {
System.out.println("User Name : "+dt.Name);
System.out.println("Users Email : "+dt.Email);
}
}
Please note, I have defined my ModelData elements as all String.
Any suggestion? What am I missing?
Thanks in advance!
The toString() representation hints at the object's type com.mysql.cj.jdbc.Blob. If you look up its javadoc (or the interface it implements) you'll see the options that you have to decode the contents of the Blob, namely getting an InputStream or a byte[] representation, which you'd have to subject to the correct character set decoding to turn it into a String.
Make sure you nail the character set by testing it with all kinds of Unicode content, so that you don't have to fix badly encoded database content later when your table contains a lot of data in unknown encodings.
As you're using Liferay's Service Builder, you might want to share the relevant parts of your service.xml (optional model-hints.xml) to check for an easier implementation.
I finally got this working by changing the field in question to String and addining a max length to certain number of Char
Thanks!
I have a dataset with a bunch of BigDecimal values. I would like to output these records to a JSON file, but when I do the BigDecimal values will often be written with trailing zeros (123.4000000000000), but the spec we are must conform to does not allow this (for reasons I don't understand).
I am trying to see if there is a way to override how the data is printed to JSON.
Currently, my best idea is to convert each record to a string using JACKSON and then writing the data using df.write().text(..) rather than JSON.
I suggest to convert Decimal type to String before writing to JSON.
Below code is in Scala, but you can use it in Java easily
import org.apache.spark.sql.types.StringType
# COLUMN_NAME is your DataFrame column name.
val new_df = df.withColumn('COLUMN_NAME_TMP', df.COLUMN_NAME.cast(StringType)).drop('COLUMN_NAME').withColumnRenamed('COLUMN_NAME_TMP', 'COLUMN_NAME')
I'm trying to parse Json in Kotlin. I'm having a lot of trouble, it seems that a lot of people learn Kotlin after Java... Not me, I'm a Python guy. I got a Kotlin Jupyter Notebook running fairly quickly (https://github.com/ligee/kotlin-jupyter), after that I managed to pull information from the bittrex api like so:
import java.net.URL
val result = URL("https://bittrex.com/api/v1.1/public/getmarkets").readText()
It took me a long time to find that I needed to add import java.net.URL, this always seems to be implicit in all code examples. Anyway, this give me a response in json (the "result parameter"):
{"success":true,"message":"","result":[{"MarketCurrency":"LTC","BaseCurrency":"BTC","MarketCurrencyLong":"Litecoin","BaseCurrencyLong":"Bitcoin","MinTradeSize":0.01469482,"MarketName":"BTC-LTC","IsActive":true,"Created":"2014-02-13T00:00:00","Notice":null,"IsSponsored":null,"LogoUrl":"https://bittrexblobstorage.blob.core.windows.net/public/6defbc41-582d-47a6-bb2e-d0fa88663524.png"},{"MarketCurrency":"DOGE","BaseCurrency":"BTC","MarketCurrencyLong":"Dogecoin","BaseCurrencyLong":"Bitcoin","MinTradeSize":274.72527473,"MarketName":"BTC-DOGE","IsActive":true,"Created":"2014-02-13T00:00:00","Notice":null,"IsSponsored":null,"LogoUrl":"https://bittrexblobstorage.blob.core.windows.net/public/a2b8eaee-2905-4478-a7a0-246f212c64c6.png"},{"MarketCurrency ...
Now, in Python I'd just add .json() to the "result" parameter and I can then address the json fields as a dictionary with multiple levels, like
result["success"]
Would give me:
true
Is there something like that for Kotlin? I have tried Klaxon https://github.com/cbeust/klaxon, again it took me a lot of time to realize that I have to do import com.beust.klaxon.string, it is not mentioned on the website for example, so a side question is: How do you know what you need to import when you find code examples? It seems like everybody just knows... But I digress.
My main question is: How can I address the separate fields of the Json and get them into separate variables?
Highest regards.
There are many JSON parsers out there. Your example was a Kotlin explicit one and that is not mandatory for Kotlin because there are also many basic Java parsers, which you can use just as fine in Kotlin.
For your imports. Obviously you need to import the classes you want to use and IDE's like IntelliJ handle the imports for you automatically. That means that you won't have to type any import statements, but they are added automatically when referencing those classes.
I think that nowadays some libraries just expect that you do not handle the imports yourself and thus do not assist you on finding the right imports.
My suggestion for a parser is Fuel.
The library is optimized for Kotlin as well. Your problem would be solved with this simple code snippet with the help of Fuel:
"https://bittrex.com/api/v1.1/public/getmarkets".httpGet().responseJson { _, response, result ->
if (response.responseMessage == "OK" && response.statusCode == 200) {
val yourResult = result.get().obj().getBoolean("success")
}
}
Something you may or may not know is that Kotlin is 100% compatible with Java, thus all the Java json parsers work well with Kotlin. I highly recommend gson. It's small (~200kb), fast, and pretty simple to use.
If this code is running in a server, jackson is pretty standard. It's the most performant json parser for java at the moment, but it's very heavy. It will take some more complicated configuration though, and I think it might require some Kotlin specific modules.
I haven't tried it yet, as it hasn't officially been released, but Kotlin offers a plugin for generating json serialization code. That will probably eventually be the standard way for Kotlin to serialize / deserialize as it should theoretically be the most performant.
The best and quick practice is instead of manually checking for each key, generate native Kotlin "data classes" using tools e.g. https://json2kotlin.com
So your API response turns into the following couple of data classes corresponding to the JSON structure:
data class Json4Kotlin_Base (
val success : Boolean,
val message : String,
val result : List<Result>
)
and
data class Result (
val marketCurrency : String,
val baseCurrency : String,
val marketCurrencyLong : String,
val baseCurrencyLong : String,
val minTradeSize : Double,
val marketName : String,
val isActive : Boolean,
val isRestricted : Boolean,
val created : String,
val notice : String,
val isSponsored : String,
val logoUrl : String
)
When you get the result, you simple map the JSON response to these data classes. The video here shows how to do it step by step.
I have to convert a string to json format in scala. The string is like this:
"classification" : "Map(Metals -> List(Cu, Co, Ni), Nonmetals -> List(N,O,C), Noblegases -> List(Ar, Kr))"
The desired json format is like this:
"classification" : {"Metals": [Cu, Co, Ni],
"Nonmetals":[N,O,C],
"Noblegases":[Ar, Kr]
}
Any quick suggestions would be appreciated.
Your question is not very specific so my answer is a bit vague as well.
First you will have to parse the input string and extract the values. I would use a combination of regular expressions and simple String operations like searching for the first occurrence of a certain character (e.g. colon) and splitting the string there.
In the next step you create the JSON object. There are several libraries out there that you can use. I suggest JSON-Java/org.json or if you like to use a scala library you can use play-json.
I need to validate the schema of certain JSON input I receive. I am not clear about how to go about the whole thing. But this is what I have gathered so far :
I need to prepare a schema for all sorts of input using something like http://json-schema.org/implementations.html
Then I need a validator like https://github.com/fge/json-schema-validator
I need to give the json input to and the schema to the validator and get the result.
However my question is that I need to use a jar which I can import and use of the json-schema-validator https://github.com/fge/json-schema-validator . Also I am not clear on how to use it. I don't know the format it accepts , the classes and methods needed etc.
How good the validator's support is for Scala?
I would not go through the pain of manually collecting the jars of json schema validator (done that, not fun). It is better use a tool for that (like maven, sbt, gradle or ivy).
In case you want to use it in an OSGi environment, you might need to use a different (probably not up-to-date) version.
Usage:
val factory: JsonSchemaFactory = JsonSchemaFactory.getDefault
val validator: JsonValidator = factory.getValidator
val schemaJson: com.fasterxml.jackson.databind.JsonNode = yourJsonSchemaInJackson2Format
val report: ProcessingReport = validator.validate(schemaJson, yourJsonInJackson2Format)
//check your report.
PS.: In case you want to collect the dependencies manually, you can go through the dependencies transitively starting on this page.
There is Orderly, liftweb-json-based JSON validator implementation for Scala:
import com.nparry.orderly._
import net.liftweb.json.JsonAST._
val orderly = Orderly("integer {0,100};")
val noProblems = orderly.validate(JInt(50))
val notAllowed = orderly.validate(JInt(200))
Use net.liftweb.json.parse(s: String): JValue to obtain JValue from String.
I noticed that orderly4jvm does not support the latest JSON Schema version 4, which causes problems if you want to use it to generate the JSON schema.