How to update json file sorted by number? - Python - json

For example I have a JSON file with a mess number
{
"data": {
"31": {
...
},
"52": {
...
},
"1": {
...
}
}
}
I wanted to make It like sorted by number so the json data will not be messed up
{
"data": {
"52": {
...
},
"31": {
...
},
"52": {
...
}
}
}
I tried a code that uses:
with open ("file.json", "r", encoding="utf-8") as f:
file = json.load(f)
file["data"].update(
{num: {"question": question, "answer": answer, "options": options}}
)
My errors code: TypeError: cannot convert dictionary update sequence element #0 to a sequence

Option #1
Use sorted with key argument:
import json
my_dict = {
"data": {
"31": {
"k": "..."
},
"52": {
"k": "..."
},
"1": {
"k": "..."
}
}
}
my_dict['data'] = dict(sorted(my_dict['data'].items(), key=lambda t: int(t[0])))
print(my_dict['data'])
Result:
{'1': {'k': '...'}, '31': {'k': '...'}, '52': {'k': '...'}}
Option #2
Use json.dumps with sort_keys argument:
print(json.dumps(my_dict, indent=2, sort_keys=True))
Result:
{
"data": {
"1": {
"k": "..."
},
"31": {
"k": "..."
},
"52": {
"k": "..."
}
}
}

Related

How do i validate a dynamic JSON key?

{
"id": "aghysfgagaw365",
"MetricLevelId": "4890718305",
"level": "BUB",
"type": "Mash",
"summary": {
"counts": [
{},
{
"label": {},
"value": 2674,
"labelLoc": {
"192706": {
"ADD": 8977,
"DEL": 3257,
"Count": 59
},
"543419": {
"ADD": 0,
"DEL": 0,
"Count": 1
}
}
}
]
}
}
I read the documentation but i'm still unclear to validate a complex API's like this. a demo to validate this API would help me to solve other API validations...especially this one has a dynamic JSON key.how do i validate ADD,DEL and Cont with "192706" being dynamic.
From the docs, please refer to the documentation on JsonPath. Also recommended is the section on "JSON Transforms". Here is one way:
* def response =
"""
{
"id": "aghysfgagaw365",
"MetricLevelId": "4890718305",
"level": "BUB",
"type": "Mash",
"summary": {
"counts": [
{
},
{
"value": 2674,
"labelLoc": {
"192706": {
"ADD": 8977,
"DEL": 3257,
"Count": 59
},
"543419": {
"ADD": 0,
"DEL": 0,
"Count": 1
}
}
}
]
}
}
"""
* def label = get[0] response..labelLoc
* def vals = get label.*
* match each vals == { ADD: '#number', DEL: '#number', Count: '#number' }
EDIT also this would work:
* json keys = label.keySet()
* match keys == ['192706', '543419']

convert JavaScript nested array for priming tree format

i have following JAVASCRIPT OBJECT and i need to convert it to primeng tree format , please help
INPUT
{
"com": {
"ups": {
"demo": {
"a": 9
}
}
}
}
OUTPUT expected
[
{
"label": "COM",
"data": "COM",
"children": [{
"label": "ABC",
"data": "abc",
"children": [ "label": "x" data": "x" ,children:[]]
}]
}]
Working Example
validate(a) {
let newArr = [];
for (const key in a) {
if (key) {
newArr.push({data: key, label: key, childern: this.validate(a[key])});
}
}
return newArr;
}
const a = {
"com": {
"ups": {
"demo": {
"a": 9
}
}
}
};
console.log(this.validate(a));

Put Items using Json File in AWS DynamoDB using AWS CLI

While putting below JSON in dynamo DB using AWS CLI with below command:
aws dynamodb put-item --table-name ScreenList --item file://tableName.json
I am getting Parameter validation failed Exception.I have gone rigorously through AWS docs but failed to find example to insert a complicated json.Every small help is welcome.
The updated Json :
{
"itemName": {
"S": "SCREEN_LIST"
},
"productName": {
"S": "P2P_MOBITEL"
},
"screenList": {
"L": [
{
"menu": {
"L": [
{
"M": {
"menuId": {
"N": "1"
},
"menuText": {
"S": "ENG_HEADING"
},
"menuType": {
"S": "Dynamic"
}
}
}
]
},
"M": {
"screenFooter": {
"S": "F_LANGUAGE_CHANGE"
},
"screenHeader": {
"S": "H_LANGUAGE_CHANGE"
},
"screenId": {
"S": "LANGUAGE_CHANGE"
},
"screenType": {
"S": ""
}
}
}
]
}
}
It seems that you are defining complex types incorrectly. According to AWS documentation you should define a list like this:
"L": ["Cookies", "Coffee", 3.14159]
and a map should be defined like this:
"M": {"Name": {"S": "Joe"}, "Age": {"N": "35"}}
which means that a menu map should be defined like this:
"menu": {
"L": [
{
"M": {
"menuId": {"N" :"1"},
"menuText": {"S" :"PACKS_SCREEN"},
"menuType": {"S" :"Dynamic"}
}
}
]
}
Notice the "M" and "L" attributes.
You should change the rest of your JSON in a similar fashion.
You can find full JSON definition here in the Options section.
UPDATE
Now your list definition is incorrect. You have:
"screenList":{
"L":[
{
"menu":{ ... },
"M":{ ... }
}
]
}
While it should be:
"screenList":{
"L":[
{
"M":{ ... }
},
{
"M":{ ... }
},
]
}

How to access Dynamodb's original JSON elements?

I am trying to test my lambda manually with the following dynamodb event input configured in tests -
Let's call this Json-1
{
"Records": [
{
"eventID": "1",
"eventVersion": "1.0",
"dynamodb": {
"Keys": {
"Id": {
"N": "101"
}
},
"NewImage": {
"Message": {
"S": "New item!"
},
"Id": {
"N": "101"
}
},
"StreamViewType": "NEW_AND_OLD_IMAGES",
"SequenceNumber": "111",
"SizeBytes": 26
},
"awsRegion": "us-west-2",
"eventName": "INSERT",
"eventSourceARN": eventsourcearn,
"eventSource": "aws:dynamodb"
},
{
"eventID": "2",
"eventVersion": "1.0",
"dynamodb": {
"OldImage": {
"Message": {
"S": "New item!"
},
"Id": {
"N": "101"
}
},
"SequenceNumber": "222",
"Keys": {
"Id": {
"N": "101"
}
},
"SizeBytes": 59,
"NewImage": {
"Message": {
"S": "This item has changed"
},
"Id": {
"N": "101"
}
},
"StreamViewType": "NEW_AND_OLD_IMAGES"
},
"awsRegion": "us-west-2",
"eventName": "MODIFY",
"eventSourceARN": sourcearn,
"eventSource": "aws:dynamodb"
},
{
"eventID": "3",
"eventVersion": "1.0",
"dynamodb": {
"Keys": {
"Id": {
"N": "101"
}
},
"SizeBytes": 38,
"SequenceNumber": "333",
"OldImage": {
"Message": {
"S": "This item has changed"
},
"Id": {
"N": "101"
}
},
"StreamViewType": "NEW_AND_OLD_IMAGES"
},
"awsRegion": "us-west-2",
"eventName": "REMOVE",
"eventSourceARN": sourcearn,
"eventSource": "aws:dynamodb"
}
]
}
However, the json of dynamodb items look like this -
Let's call this Json-2
{
"id": {
"S": "RIGHT-aa465568-f4c8-4822-9c38-7563ae0cd37b-1131286033464633.jpg"
},
"lines": {
"L": [
{
"M": {
"points": {
"L": [
{
"L": [
{
"N": "0"
},
{
"N": "874.5625"
}
]
},
{
"L": [
{
"N": "1765.320601851852"
},
{
"N": "809.7800925925926"
}
]
},
{
"L": [
{
"N": "3264"
},
{
"N": "740.3703703703704"
}
]
}
]
},
"type": {
"S": "guard"
}
}
}
]
},
"modified": {
"N": "1483483932472"
},
"qastatus": {
"S": "reviewed"
}
}
Using the lambda function below, I can connect to my table. My goal is create a json which elastic search will accept.
#Override
public Object handleRequest(DynamodbEvent dynamodbEvent, Context context) {
List<DynamodbEvent.DynamodbStreamRecord> dynamodbStreamRecordlist = dynamodbEvent.getRecords();
DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient());
log.info("Whole event - "+dynamodbEvent.toString());
dynamodbStreamRecordlist.stream().forEach(dynamodbStreamRecord -> {
if(dynamodbStreamRecord.getEventSource().equalsIgnoreCase("aws:dynamodb")){
log.info("one record - "+dynamodbStreamRecord.getDynamodb().toString());
log.info(" getting N from new image "+dynamodbStreamRecord.getDynamodb().getNewImage().toString());
String tableName = getTableNameFromARN(dynamodbStreamRecord.getEventSourceARN());
log.info("Table name :"+tableName);
Map<String, AttributeValue> keys = dynamodbStreamRecord.getDynamodb().getKeys();
log.info(keys.toString());
AttributeValue attributeValue = keys.get("Id");
log.info("Value of N: "+attributeValue.getN());
Table table = dynamoDB.getTable(tableName);
}
});
return dynamodbEvent;
}
The format of a JSON item that elastic search expects is this and this is what I want to map the test input json to-
Let's call this Json-3
{
_index: "bar-guard",
_type: "bar-guard_type",
_id: "LEFT-b1939610-442f-4d8d-9991-3ca54685b206-1147042497459511.jpg",
_score: 1,
_source: {
#SequenceNumber: "4901800000000019495704485",
#timestamp: "2017-01-04T02:24:20.560358",
lines: [{
points: [[0,
1222.7129629629628],
[2242.8252314814818,
1254.702546296296],
[4000.0000000000005,
1276.028935185185]],
type: "barr"
}],
modified: 1483483934697,
qastatus: "reviewed",
id: "LEFT-b1939610-442f-4d8d-9991-3ca54685b206-1147042497459511.jpg"
}
},
So what I need is read Json-1 and map it to Json-3.
However, Json-1 does not seem to be complete i.e. it does not have information that a dynamodb json has - like points and lines in Json-2.
And so, I was trying to get a connection to the original table and then read this additional information of lines and points by using the ID.
I am not sure if this is the right approach. Basically, want to figure out a way to get the actual JSON that dynamodb has and not the one that has attribute types
How can I get lines and points from json-2 using java? I know we have DocumentClient in javascript but I am looking for something in java.
Also, came across a converter here but doesn't help me- https://github.com/aws/aws-sdk-js/blob/master/lib/dynamodb/converter.js
Is this something that I should use DynamoDBMapper or ScanJavaDocumentAPI for ?
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/dynamodbv2/datamodeling/DynamoDBMapper.html#marshallIntoObjects-java.lang.Class-java.util.List-com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig-
If yes, I am a little lost how to do that in the code below -
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = dynamoDBClient.scan(scanRequest);
for(Map<String, AttributeValue> item : result.getItems()){
AttributeValue value = item.get("lines");
if(value != null){
List<AttributeValue> values = value.getL();
for(AttributeValue value2 : values){
//what next?
}
}
}
Ok, this seems to work for me.
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = dynamoDBClient.scan(scanRequest);
for(Map<String, AttributeValue> item : result.getItems()){
AttributeValue value = item.get("lines");
if(value != null){
List<AttributeValue> values = value.getL();
for(AttributeValue value2 : values){
if(value2.getM() != null)
{
Map<String, AttributeValue> map = value2.getM();
AttributeValue points = map.get("points");
List<AttributeValue> pointsvalues = points.getL();
if(!pointsvalues.isEmpty()){
for(AttributeValue valueOfPoint : pointsvalues){
List<AttributeValue> pointList = valueOfPoint.getL();
for(AttributeValue valueOfPoint2 : pointList){
}
}
}
}
}
}
}

Reading JSON file from R

I try reading a JSON file from R using rjson but keep getting errors. I validated the JSON file using various online validators. Here is the content of the JSON file:
{
"scenarios": [
{
"files": {
"type1": "/home/blah/Desktop/temp/scen_0.type1",
"type2": "/home/blah/Desktop/temp/scen_0.type2"
},
"ID": "scen_0",
"arr": [],
"TypeToElementStatsFilename": {
"type1": "/home/blah/Desktop/temp/scen_0.type1.elements",
"type2": "/home/blah/Desktop/temp/scen_0.type2.elements"
}
}
],
"randomSeed": "39327314969888",
"zone": {
"length": 1000000,
"start": 1
},
"instanceFilename": "/home/blah/bloo/data/XY112.zip",
"txtFilename": "/home/blah/bloo/data/XY112.txt",
"nSimulations": 2,
"TypeTodbFilename": {
"type1": "/home/blah/bloo/data/map.type1.oneAmb.XY112.out"
},
"arr": {
"seg11": {
"length": 1000,
"start": 147000
},
"seg12": {
"length": 1000,
"start": 153000
},
"seg5": {
"length": 1000,
"start": 145000
},
"seg6": {
"length": 1000,
"start": 146000
},
"seg1": {
"length": 100,
"start": 20000
}
},
"outPath": "/home/blah/Desktop/temp",
"instanceID": "XY112",
"arrIds": [
"seg5",
"seg6",
"seg1",
"seg11",
"seg12"
],
"truth": {
"files": {
"type1": "/home/blah/Desktop/temp/truth.type1",
"type2": "/home/blah/Desktop/temp/truth.type2"
},
"ID": "truth",
"TypeToElementStatsFilename": {
"type1": "/home/blah/Desktop/temp/truth.type1.elements",
"type2": "/home/blah/Desktop/temp/truth.type2.elements"
}
}
}
And the error:
> json_file <- "~/json"
> json_data <- fromJSON(paste(readLines(json_file), collapse=""))
Error in fromJSON(paste(readLines(json_file), collapse = "")) :
unexpected character: :
RJSON freaks out about empty arrays.
fromJSON( '{ "arr": [ ] }')
Error in fromJSON("{ \"arr\": [ ] }") : unexpected character: :
You can try the fromJSON function in the RJSONIO package hosted at http://www.omegahat.org. It seems to read the file fine.
There's a fix for this.
Create a new function to replace the existing getURL function used in RCurl and you should have your solution.
myGetURL <- function(...) {
rcurlEnv <- getNamespace("RCurl")
mapUnicodeEscapes <- get("mapUnicodeEscapes", rcurlEnv)
unlockBinding("mapUnicodeEscapes", rcurlEnv)
assign("mapUnicodeEscapes", function(str) str, rcurlEnv)
on.exit({
assign("mapUnicodeEscapes", mapUnicodeEscapes, rcurlEnv)
lockBinding("mapUnicodeEscapes", rcurlEnv)
}, add = TRUE)
return(getURL(...))
}
Test:
> json <- myGetURL("http://abicky.net/hatena/rcurl/a.json")
> cat(json, fill = TRUE)
{"a":"\\\"\u0030\\\""}
> fromJSON(json)
$a
[1] "\\\"0\\\""