Generate Data class on dynamic value inside a JSON payload - json

I have a payload which look like this :
{
"data": {
"12345": {
"is_indexable": true,
"description": "Lorem description",
"items": {
"id": 2644,
"name": "Naming here"
}
},
"678910": {
"is_indexable": false,
"description": "Lorem description 2",
"items": {
"id": 29844,
"name": "Naming here again"
}
}
}
}
I wanted to generate a specific data class for that payload using tools like https://transform.tools/json-to-kotlin but it's impossible since the array inside the "data" object is an ID (so a dynamic data)
data class Root(
val data_field: Data
)
data class Data(
val payload: List<Payload>, // Something to represent the dynamic ids
)
data class Payload(
val isIndexable: Boolean,
val description: String,
val items: Items
)
data class Items(
val id: Long,
val name: String
)
I don't know if I'm clear, did you have an idea to make this in a clean way ?
Thank you all !

You need to array of map to achieve this. You are building map in map right now. It should be like following if you need array;
{
"data": [
{ id: "12345",
"is_indexable": true,
"description": "Lorem description",
"items": {
"id": 2644,
"name": "Naming here"
}
},
{"id" : "678910"
"is_indexable": false,
"description": "Lorem description 2",
"items": {
"id": 29844,
"name": "Naming here again"
}
}
]
}

There is an amazing plugin add to the android studio where it converts this payload to suitable classes
for example, the JSON has one object and inside the object, there are a lot of objects and array this tool will determine all this as classes
you can load it from this link:
https://plugins.jetbrains.com/plugin/9960-json-to-kotlin-class-jsontokotlinclass
From this link, you can also know how you can use this tool
go to android studio and from file->setting->plugin->Install Plugin from disk
enter image description here
and convert it by easy and without problems
good luck

Related

How to traverse list of nested maps in scala

I have been given a json string that looks like the following one:
{
"dataflows": [
{
"name": "test",
"sources": [
{
"name": "person_inputs",
"path": "/data/input/events/person/*",
"format": "JSON"
}
],
"transformations": [
{
"name": "validation",
"type": "validate_fields",
"params": {
"input": "person_inputs",
"validations": [
{
"field": "office",
"validations": [
"notEmpty"
]
},
{
"field": "age",
"validations": [
"notNull"
]
}
]
}
},
{
"name": "ok_with_date",
"type": "add_fields",
"params": {
"input": "validation_ok",
"addFields": [
{
"name": "dt",
"function": "current_timestamp"
}
]
}
}
],
"sinks": [
{
"input": "ok_with_date",
"name": "raw-ok",
"paths": [
"/data/output/events/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
},
{
"input": "validation_ko",
"name": "raw-ko",
"paths": [
"/data/output/discards/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
}
And I have been asked to use it as some kind of recipe for an ETL pipeline, i.e., the data must be extracted from the "path" specifid in the "sources" key, the transformations to be carried out are specified within the "transformations" key and, finally, the transformed data must saved to one of the two specified "sink" keys.
I have decided to convert the json string into a scala map, as follows:
val json = Source.fromFile("path/to/json")
//parse
val parsedJson = jsonStrToMap(json.mkString)
implicit val formats = org.json4s.DefaultFormats
val parsedJson = parse(jsonStr).extract[Map[String, Any]]
so, with that, I get a structure like this one:
which is a map whose first value is a list of maps. I can evaluate parsedJson("dataflows") to get:
which is a list, as expected, but, then I cannot traverse such list, even though I need to in order to get to the sources, transformations and sinks. I have tried using the index of the listto, for example, get its first element, like this: parsedJson("dataflows")(0), but to no avail.
Can anyone please help me traverse this structure? Any help would be much appreciated.
Cheers,
When you evaluate parsedJson("dataflows") a Tuple2 is returned aka a Tuple which has two elements that are accessed with ._1 and ._2
So for dataflows(1)._1 the value returned is "sources" and dataflows(1)._2 is list of maps (List[Map[K,V]) which can be traversed like you would normally traverse elements of a List where each element is Map
Let's deconstruct this for example:
val dataFlowsZero = ("sources", List(Map(42 -> "foo"), Map(42 -> "bar")))
The first element in the Tuple
scala> dataFlowsZero._1
String = sources
The second element in the Tuple
scala> dataFlowsZero._2
List[Map[Int, String]] = List(Map(42 -> foo), Map(42 -> bar))`
Map the keys in each Map in List to a new List
scala> dataFlowsZero._2.map(m => m.keys)
List[Iterable[Int]] = List(Set(42), Set(42))
Map the values in each Map in the List to a new List
scala> dataFlowsZero._2.map(m => m.values)
List[Iterable[String]] = List(Iterable(foo), Iterable(bar))
The best solution is to convert the JSON to the full data structure that you have been provided rather than just Map[String, Any]. This makes it trivial to pick out the data that you want. For example,
val dataFlows = parse(jsonStr).extract[DataFlows]
case class DataFlows(dataflows: List[DataFlow])
case class DataFlow(name: String, sources: List[Source], transformations: List[Transformation], sinks: List[Sink])
case class Source(name: String, path: String, format: String)
case class Transformation(name: String, `type`: String, params: List[Param])
case class Param(input: String, validations: List[Validation])
case class Validation(field: String, validations: List[String])
case class Sink(input: String, name: String, paths: List[String], format: String, saveMode: String)
The idea is to make the JSON handler do most of the work to create a type-safe version of the original data.

Map JSON in Angular 9

I designed an app using Angular which is linked to a DB using a backEnd made with ASP.NET
My problem is that my JSON in the Front is a little different that the JSON in the backEnd.
And I need help on how to map the frontEnd JSON to send it to the backEnd.
psd: i created all the interfaces on the frontend, and I'm usign formControls and FormArrays to store the data in the front, the problem is that de JSON are a bit different.
I cannot change the JSON on the backEnd because is linked with the DB and it has PK and FK so that's the reasson because it's a bit different.
I was thinking about doing something like this.myform.value and map to the other interface, but i have no idea on how to do that because i have array of arrays...
Maybe a for loop?
Here is my frontEnd JSON:
{
"non-array fields....................." : "bla bla",
"arraySustancias": [
{
"sustanciaActiva": "hello",
"contenido": "2",
"suministro": ["hello2","hello3"],
"unidades": "3"
}
],
"arrayProcedimientos": [
{
"procedimiento": "",
"fechaInicioProcedimiento": "",
"fechaFinProcedimiento": "",
"notasProcedimiento": "",
"arraySubprocedimientos": [
{
"subprocedimiento": "",
"fechaInicioSubProcedimiento": "",
"fechaFinSubProcedimiento": "",
"notasSubProcedimiento": "",
"exp": ""
}
]
}
]
}
And here is my backEnd JSON:
{
"non-array fields.....................": "bla bla",
"registros_arraySustancias": [
{
"registros_arraySustancias_suministro": [
{
"registros_arraySustancias_id": "",
"id": "",
"value": [
"ABBOTT LABORATORIES",
"ADAMA Agan Ltd."
]
}
],
"registros_id": "",
"id": "",
"sustanciaActiva": "hola",
"contenido": 2,
"unidades": 3
}
],
"registros_arrayProcedimientos": [
{
"registros_id": "",
"id": "",
"procedimiento": "text",
"fechaInicioProcedimiento": "18/06/2020",
"fechaFinProcedimiento": "18/06/2020",
"notasProcedimiento": "notes",
"registros_arrayProcedimientos_arraySubprocedimientos": [
{
"registros_arrayProcedimientos_id": "",
"id": "",
"subprocedimiento": "text",
"fechaInicioSubProcedimiento": "19/06/2020",
"fechaFinSubProcedimiento": "19/06/2020",
"notasSubProcedimiento": "notes"
}
]
}
]
}
Thanks for your help :)
In that case I think you have to work a little bit with angular forms so that data from frontend should be from the same structure. FormGroups and FormArayys will help you alot.
Try looking at this article as well.
http://www.howilearnt.com/web-development/angular/make-a-json-object-with-angular-forms/
You simply create an object from your original object. There is no magic or automatic way to do that. Example:
interface OriginalData {
foo: string;
baz: {
test: number[]
};
}
interface MyData {
foobar: string;
bazTest: number[];
}
function getData(): Observable<OriginalData> {
return this.http.get<OriginalData>(url);
}
function convertData(original: OriginalData): MyData {
const res: MyData = {
foobar: original.foo,
bazTest: original.baz.test
};
return res;
}
// somewhere in your code
getData().subscribe(
v => sendData(convertData(v))
);
I'm using the Angular http client here, because you added the tag "angular". It will give you an object of the type you specified.
That was what I was looking for:
mapeado = {
pais: "",
registros_arraySustancias: [{
unidades: "",
contenido: 2,
sustanciaActiva: "",
registros_arraySustancias_suministro: [{value: [""]}]
}
]
}
Thanks a lot for your help

Ruby: How to parse json to specific types

I have a JSON that I want to parse in Ruby. Ruby is completely new to me, but I have to work with it :-)
Here is my litte snippet, that should do the parsing:
response = File.read("app/helpers/example_announcement.json")
JSON.parse(response)
this works pretty fine. The only downside is, I do not know the properties at the point where I use it, it is not typesafe. So I created the objects for it
class Announcements
##announcements = Hash # a map key => value where key is string and value is type of Announcement
end
class Announcement
##name = ""
##status = ""
##rewards = Array
end
And this is how the json looks like
{
"announcements": {
"id1" : {
"name": "The Diamond Announcement",
"status": "published",
"reward": [
{
"id": "hardCurrency",
"amount": 100
}
]
},
"id2": {
"name": "The Normal Announcement",
"players": [],
"status": "published",
"reward": []
}
}
}
So I tried JSON parsing like this
response = File.read("app/helpers/example_announcement.json")
JSON.parse(response, Announcements)
But this is not how it works^^can anybody help me with this?

Kotlin Jackson generation objects from JSON

Please help! I'm trying to generate object from JSON with jackson kotlin module. Here is json source:
{
"name": "row",
"type": "layout",
"subviews": [{
"type": "horizontal",
"subviews": [{
"type": "image",
"icon": "ic_no_photo",
"styles": {
"view": {
"gravity": "center"
}
}
}, {
"type": "vertical",
"subviews": [{
"type": "text",
"fields": {
"text": "Some text 1"
}
}, {
"type": "text",
"fields": {
"text": "Some text 2"
}
}]
}, {
"type": "vertical",
"subviews": [{
"type": "text",
"fields": {
"text": "Some text 3"
}
}, {
"type": "text",
"fields": {
"text": "Some text 4"
}
}]
}, {
"type": "vertical",
"subviews": [{
"type": "image",
"icon": "ic_no_photo"
}, {
"type": "text",
"fields": {
"text": "Some text 5"
}
}]
}]
}]
}
I'm trying to generate instance of Skeleton class.
data class Skeleton (val type : String,
val name: String,
val icon: String,
val fields: List<Field>,
val styles: Map<String, Map<String, Any>>,
val subviews : List<Skeleton>)
data class Field (val type: String, val value: Any)
As you can see, Skeleton object can have other Skeleton objects inside (and these objects can have other Skeleton objects inside too), also Skeleton can have List of Field objects
val mapper = jacksonObjectMapper()
val skeleton: Skeleton = mapper.readValue(File(file))
This code ends with exception:
com.fasterxml.jackson.databind.JsonMappingException: Instantiation of [simple type, class com.uibuilder.controllers.parser.Skeleton] value failed (java.lang.IllegalArgumentException): Parameter specified as non-null is null: method com.uibuilder.controllers.parser.Skeleton.<init>, parameter name
at [Source: docs\layout.txt; line: 14, column: 3] (through reference chain: com.uibuilder.controllers.parser.Skeleton["subviews"]->java.util.ArrayList[0]->com.uibuilder.controllers.parser.Skeleton["subviews"]->java.util.ArrayList[0])
There are several issues I found about your mapping that prevent Jackson from reading the value from JSON:
Skeleton class has not-null constructor parameters (e.g. val type: String, not String?), and Jackson passes null to them if the value for those parameters is missing in JSON. This is what causes the exception you mentioned:
Parameter specified as non-null is null: method com.uibuilder.controllers.parser.Skeleton.<init>, parameter name
To avoid it, you should mark the parameters that might might have values missing as nullable (all of the parameters in your case):
data class Skeleton(val type: String?,
val name: String?,
val icon: String?,
val fields: List<Field>?,
val styles: Map<String, Map<String, Any>>?,
val subviews : List<Skeleton>?)
fields in Skeleton has type List<Field>, but in JSON it's represented by a single object, not by an array. The fix would be to change the fields parameter type to Field?:
data class Skeleton(...
val fields: Field?,
...)
Also, Field class in your code doesn't match the objects in JSON:
"fields": {
"text": "Some text 1"
}
You should change Field class as well, so that it has text property:
data class Field(val text: String)
After I made the changes I listed, Jackson could successfully read the JSON in question.
See also: "Null Safety" in Kotlin reference.

Mongoose - How can I make this data more usable?

I have the following data strucutre outputting form my Schema in a node/express app. I'd like to have the feeds array simply an array of name:key pairs. I don't like the sort of weird numbered object structure going on between "feeds" and the actual feeds data. But i can't figure out how to manually define that in mongoose. any help would be awesome. thanks!
outputted JSON
{
"title": "Testing",
"created_at": "2011-10-05T16:23:26.217Z",
"feeds": [{
"0": {
"name": "twitter",
"key": "person1"
},
"1": {
"name": "twitter",
"key": "person2"
},
"_id": "4e8c847e02edc10035000003"
}]
}
i want this:
{
"title": "Testing",
"created_at": "2011-10-05T16:23:26.217Z",
"feeds": [
{
"name": "twitter",
"key": "person1"
},
{
"name": "twitter",
"key": "person2"
}
],
"_id": "4e8c847e02edc10035000003"
}
this is my schema:
var Feed = new Schema({
name : { type: String }
, key : { type: String }
});
var Page = new Schema({
title : { type: String, required: true, index: { unique: true } }
, feeds : [Feed]
, created_at : { type: Date, required: true, default: Date.now }
});
Ok, a colleague was able to answer this for me. My bad for not posting the relevant code, I didn't realize where the problem actually was. But for those who may encounter this problem:
If you push your embedded docs into the model when saving, you may need to do a forEach loop rather than pushing the embedded docs (in this case Feeds) together. Using forEach, the database saved the feeds directly to the feeds array rather than creating those weird groupings.
This pushed the feeds in properly:
req.body.feed.forEach(function(feed){
page.feeds.push(feed);
});
Let me know if you have the same problem and need more explanation.