I want to do something weird... :)
I have a json file that will have either 1 key name OR another keyname. I want to detect which one is actually there, and assign to kotlin variable in class. Only one OR the other will be there. So,
data class CED(
#JsonProperty("rev_env")
var revision: Any? = null,
#JsonProperty("rev")
var revision: Any? = null,
.
.
.
So, either rev_env OR rev will be in the json file, but whichever is there, their mapping always goes to "revision".
Any way to do that?
You can use #JsonAlias.
For example:
data class A(
#JsonAlias(value = ["rev", "rev_env"])
val rev: Any? = null
)
Related
I kind of feel stupid for asking this, but haven't been able to find a way to get the key of a JSON value. I know how to retrieve the key if I have an iterator of the object. I also know of operator[].
In my case the key is not a known value, so can't use get(const char *key) or operator[]. Also can't find a getKey() method.
My JSON looks like this:
{Obj_Array: [{"122":{"Member_Array":["241", "642"]}}]}
For the piece of code to parse {"122":{"Member_Array":["241", "642"]}} I want to use get_key()-like function just to retrieve "122" but seems like I have to use an iterator which to me seems to be overkill.
I might have a fundamental lack of understanding of how jsoncpp is representing a JSON file.
First, what you have won't parse in JsonCPP. Keys must always be enclosed in double quotes:
{"Obj_Array": [{"122":{"Member_Array":["241", "642"]}}]}
Assuming that was just an oversight, if we add whitespace and tag the elements:
{
root-> "Obj_Array" : [
elem0-> {
key0-> "122":
val0-> {
key0.1-> "Member_Array" :
val0.1-> [
elem0.1.0-> "241",
elem0.1.1-> "642" ]
}
}
]
}
Assuming you have managed to read your data into a Json::Value (let's call it root), each of the tagged values can be accessed like this:
elem0 = root[0];
val0 = elem0["122"]
val0_1 = val0["Member_Array"];
elem0_1_0 = val0_1[0];
elem0_1_1 = val0_1[1];
You notice that this only retrieves values; the keys were known a priori. This is not unusual; the keys define the schema of the data; you have to know them to directly access the values.
In your question, you state that this is not an option, because the keys are not known. Applying semantic meaning to unknown keys could be challenging, but you already came to the answer. If you want to get the key values, then you do have to iterate over the elements of the enclosing Json::Value.
So, to get to key0, you need something like this (untested):
elem0_members = elem0.getMemberNames();
key0 = elem0_members[0];
This isn't production quality, by any means, but I hope it points in the right direction.
My model has a json field. I can access jsonfield['key1'] with the following query
from django.contrib.postgres.fields.jsonb import KeyTextTransform
MyModel.objects.annotate(val=KeyTextTransform('key1', 'jsonfield')).order_by('val')
But how can I access a key like jsonfield['key1']['key2'] or even more nested ones?
This can't be the only solution, right?
MyModel.objects.annotate(val=KeyTextTransform('key2', (KeyTextTransform('key1', 'jsonfield'))).order_by('val')
The hard part is already done, thankfully. KeyTextTransform is composable. All we have to do is compose it.
class NestableKeyTextTransform:
def __new__(cls, field, *path):
if not path:
raise ValueError("Path must contain at least one key.")
head, *tail = path
field = KeyTextTransform(head, field)
for head in tail:
field = KeyTextTransform(head, field)
return field
MyModel.objects.annotate(
single_nested_value=NestableKeyTextTransform(
"json_field", "query", "name"
),
array_access=NestableKeyTextTransform(
"json_field", "query", "address_line", 1
),
)
Though I would like to point out that this may be a better way to do it:
from django.db.models import F
MyModel.objects.annotate(
single_nested_value=F("json_field__query__name"),
array_access=F("json_field__query__address_line__1"),
)
NB as of the start of 2023 (in django's development version) you can now also do:
from django.db.models.fields.json import KT
Dogs.objects.annotate(
first_breed=KT("data__breed__1"),
owner_name=KT("data__owner__name")
)
See docs and feature request
I have a CSV file of data in the form
21.06.2016 23:00:00.349, 153.461, 153.427
21.06.2016 23:00:00.400, 153.460, 153.423
etc
The initial step of creating a frame involves the optional inclusion of a 'schema' to specify or rename column heads and specify types:
let df = Frame.ReadCsv(__SOURCE_DIRECTORY__ + "/data/GBPJPY.csv", hasHeaders=true, inferTypes=false, schema="TS (DateTimeOffset), Bid (float(3)), Ask (float(3))")
I would like to specify the first column of string values to be ParseExact'ed to DateTimeOffset of the format
"dd.mm.yyyy HH:mm:ss.fff"
(I'm assuming the use of the setting System.Globalization.CultureInfo.InvariantCulture).
How do I express the schema such that it will parse the datetime string in that first Frame.ReadCsv("file.csv", schema = ........ )? Or is this not possible to accomplish within the schema statement?
I have this string:
var test = "toAdd";
I want to use it to extract data from JSON, like:
console.log(value.stats.test);
As you see, test is not correct, as it is just a string and can't be used, it is just not recognized at all. How do I make it recognize?
What you are attempting to do is this:
var someVar;
someVar.test = 'Sample';
someVar.test.attribute = 'Another sample';
// this:
console.log(someVar['test']['attribute']);
// will produce the same as this:
console.log(someVar['test'].attribute);
// as well as the same as this:
console.log(someVar.test['attribute']);
This will print "Another sample".
This has nothing to do with JSON. This is just javascript, and like anything where the . notation won't work switch to array notation:
foo.bar.baz = 'qux';
alert(foo['bar'].baz); // popup with 'qux'
^-----^-- note these
In your case, value.stats[test]. Now "test" isn't an array key, it's a variable whose value gets used as the key.
I want to parse a string of complex JSON in Pig. Specifically, I want Pig to understand my JSON array as a bag instead of as a single chararray. I found that complex JSON can be parsed by using Twitter's Elephant Bird or Mozilla's Akela library. (I found some additional libraries, but I cannot use 'Loader' based approach since I use HCatalog Loader to load data from Hive.)
But, the problem is the structure of my data; each value of Map structure contains value part of complex JSON. For example,
1. My table looks like (WARNING: type of 'complex_data' is not STRING, a MAP of <STRING, STRING>!)
TABLE temp_table
(
user_id BIGINT COMMENT 'user ID.',
complex_data MAP <STRING, STRING> COMMENT 'complex json data'
)
COMMENT 'temp data.'
PARTITIONED BY(created_date STRING)
STORED AS RCFILE;
2. And 'complex_data' contains (a value that I want to get is marked with two *s, so basically #'d'#'f' from each PARSED_STRING(complex_data#'c') )
{ "a": "[]",
"b": "\"sdf\"",
"**c**":"[{\"**d**\":{\"e\":\"sdfsdf\"
,\"**f**\":\"sdfs\"
,\"g\":\"qweqweqwe\"},
\"c\":[{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"}]
},
{\"**d**\":{\"e\":\"sdfsdf\"
,\"**f**\":\"sdfs\"
,\"g\":\"qweqweqwe\"},
\"c\":[{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"}]
},]"
}
3. So, I tried... (same approach for Elephant Bird)
REGISTER '/path/to/akela-0.6-SNAPSHOT.jar';
DEFINE JsonTupleMap com.mozilla.pig.eval.json.JsonTupleMap();
data = LOAD temp_table USING org.apache.hive.hcatalog.pig.HCatLoader();
values_of_map = FOREACH data GENERATE complex_data#'c' AS attr:chararray; -- IT WORKS
-- dump values_of_map shows correct chararray data per each row
-- eg) ([{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... }])
([{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... }]) ...
attempt1 = FOREACH data GENERATE JsonTupleMap(complex_data#'c'); -- THIS LINE CAUSE AN ERROR
attempt2 = FOREACH data GENERATE JsonTupleMap(CONCAT(CONCAT('{\\"key\\":', complex_data#'c'), '}'); -- IT ALSO DOSE NOT WORK
I guessed that "attempt1" was failed because the value doesn't contain full JSON. However, when I CONCAT like "attempt2", I generate additional \ mark with. (so each line starts with {\"key\": ) I'm not sure that this additional marks breaks the parsing rule or not. In any case, I want to parse the given JSON string so that Pig can understand. If you have any method or solution, please Feel free to let me know.
I finally solved my problem by using jyson library with jython UDF.
I know that I can solve it by using JAVA or other languages.
But, I think that jython with jyson is the most simplist answer to this issue.