I am trying to get data from DynamoDB using GetDynamoDB processor in nifi, I have provided all the mandatory fields except Json Document attribute, I don't know what to set in that field.
Input Data :
{
"ProductCatalog": [
{
"PutRequest": {
"Item": {
"Id": {
"N": "101"
},
"Title": {
"S": "Book 101 Title"
},
"ISBN": {
"S": "111-1111111111"
},
"Authors": {
"L": [
{
"S": "Author1"
}
]
},
"Price": {
"N": "2"
},
"Dimensions": {
"S": "8.5 x 11.0 x 0.5"
},
"PageCount": {
"N": "500"
},
"InPublication": {
"BOOL": true
},
"ProductCategory": {
"S": "Book"
}
}
}
}]
}
Any one can be appreciated.
When you store a JSON document in DynamoDB, the JSON document you store is placed as a nested object inside a wrapper object known as a DynamoDB Item, which stores metadata about the document. The Json Document attribute lets you specify the key in the DynamoDB Item for the nested JSON document so that it can be extracted from the top-level DynamoDB Item.
Looking at the source code for the GetDynamoDB NiFi processor shows how the value (jsonDocument) is used to call the AWS DynamoDB SDK, extract the JSON for the specified DynamoDB Item attribute, and create a NiFi FlowFile for the contents:
BatchGetItemOutcome result = dynamoDB.batchGetItem(tableKeysAndAttributes);
// Handle processed items and get the json document
List<Item> items = result.getTableItems().get(table);
for (Item item : items) {
ItemKeys itemKeys = new ItemKeys(item.get(hashKeyName), item.get(rangeKeyName));
FlowFile flowFile = keysToFlowFileMap.get(itemKeys);
if ( item.get(jsonDocument) != null ) {
ByteArrayInputStream bais = new ByteArrayInputStream(item.getJSON(jsonDocument).getBytes());
flowFile = session.importFrom(bais, flowFile);
}
session.transfer(flowFile,REL_SUCCESS);
keysToFlowFileMap.remove(itemKeys);
}
Hope this helps!
Json Document attribute is which key inside your DynamodDB item do you want to become the content for the downstream flowfile.
Current, there is no way to only return the entire 'root' DynamoDB item, only inside attributes.
Below an example flow:
DynamoDB Table:
GetDynamoDB:
Inside the GenerateFlowFile I'm creating a FlowFile with the attributes hash_key and range_key.
If I configure the Json Document attribute as json the output FlowFile content would be {"key1":"name","key2":2}.
If I configure as attribute the output would be "attribute".
Related
I have a JSON without a key but just a value. Is it possible to add a key to it using React?
Here I have the and hanging without a key. How do I add a key to it, so that it's easier to read? I looked at JSON.Stringify and checked the replacer but it does not let me do it. delete also does not let me do it.
{
"sample": [
{
"id": "r-1",
"name": "sam"
},
"and",
{
"id": "r-2",
"name": "jerry"
}
]
}
You're confusing objects and arrays. The "key" with the string and as a value isn't a key. It's an array element (the value of the sample property is an array). It can't have a key unless you convert it from a string to an object, like the array elements before and after it, which are objects.
The structure of the array on the sample object property looks like this:
object | string | object
If you're ok changing your data from a string to an object, just replace the second array element (index 1) in the sample property with a new object:
const myObj = {
"sample": [{
"id": "r-1",
"name": "sam"
},
"and",
{
"id": "r-2",
"name": "jerry"
}
]
}
myObj.sample[1] = {
newKey: myObj.sample[1]
};
console.log(myObj);
I'm trying to validate json files which have an element that has a property which contains a value that should exist in another part of the json. I'm using jsonschema Draft 07.
This is a simple little example that shows the scenario I'm trying to validate in my data.
{
"objects": {
"object1": {
"colorKey": "orange"
}
},
"colors": {
"orange": {
"red": "FF",
"green": "AF",
"blue": "00"
}
}
}
How can I validate that the 'value' of colorKey (in this case 'orange') actually exists as a property of the 'colors' object? The data isn't stored in arrays, just defined properties.
For official JSON Schema...
You cannot check that a key in the data is the same as a value of the data.
You cannot extract the value of data from your JSON instance to use in your JSON Schema.
That being said, ajv, the most popular validator, implements some unofficial extensions. One of which is $data.
Example taken from: https://github.com/epoberezkin/ajv#data-reference
var ajv = new Ajv({$data: true});
var schema = {
"properties": {
"smaller": {
"type": "number",
"maximum": { "$data": "1/larger" }
},
"larger": { "type": "number" }
}
};
var validData = {
smaller: 5,
larger: 7
};
ajv.validate(schema, validData); // true
This would not work for anyone else using your schemas.
I am trying to reformat JSON to remove child nodes from some JSON objects whilst keeping whats inside the child node. (unsure how to explain, like removing the title node). In this example i am trying to have a JSONarray of groups without the UserGroup child node (whilst keeping each UserGroups content intact). Example JSON
{
"groups":[
{
"UserGroup":{
"integrationKey":"0000073807",
"uid":"0000073807"
},
"UserGroup":{
"integrationKey":"0000073810",
"uid":"0000073810"
}
}
]
}
What I'd like after processing
{
"groups":[
{
"integrationKey":"0000073807",
"uid":"0000073807"
},
{
"integrationKey":"0000073810",
"uid":"0000073810"
}
]
}
For the JSON with no duplicated keys, which formats the data, removing child keys may look like
import groovy.json.JsonSlurper
import groovy.json.JsonOutput
def inputJson = '''
{
"groups": [{
"UserGroup1": {
"integrationKey": "0000073807",
"uid": "0000073807"
},
"UserGroup2": {
"integrationKey": "0000073810",
"uid": "0000073810"
}
}
]
}
'''
Map parsed = new JsonSlurper().parseText(inputJson) as Map
parsed.each {
parsed[it.key] = it.value*.values().flatten()
}
println JsonOutput.toJson(parsed)
This code will print
{"groups":[{"integrationKey":"0000073807","uid":"0000073807"},{"integrationKey":"0000073810","uid":"0000073810"}]}
Handle duplicated key
But your provided input example contains 2 items with UserGroup name. It's not common for JSON and most parsers just take the last mentioned value for the key. To handle this you can use below dependency
<dependency>
<groupId>net.sf.json-lib</groupId>
<artifactId>json-lib</artifactId>
<version>2.4</version>
<classifier>jdk15</classifier>
</dependency>
And modify the above script to
import net.sf.json.JSONObject
// ...
ipuptJson = JSONObject.fromObject(inputJson).toString()
Map parsed = new JsonSlurper().parseText(inputJson) as Map
parsed.each {
parsed[it.key] = it.value*.values().flatten()
}
println JsonOutput.toJson(parsed)
Starting from terminology, you are not really modifiyng the json itself. In fact you can modify the in-memory representation of it exposed as a mixture of Lists, Maps and primitive object types.
The simplest way to do what you want would be:
import groovy.json.*
def json = new JsonSlurper().parseText '''
{
"groups": [{
"UserGroup1": {
"integrationKey": "0000073807",
"uid": "0000073807"
},
"UserGroup2": {
"integrationKey": "0000073810",
"uid": "0000073810"
}
}
]
}
'''
def modified = [ groups:json.groups*.values().flatten() ]
println JsonOutput.prettyPrint( JsonOutput.toJson( modified ) )
prints:
{
"groups": [
{
"integrationKey": "0000073807",
"uid": "0000073807"
},
{
"integrationKey": "0000073810",
"uid": "0000073810"
}
]
}
I'm trying to create a test API in mockable.
What am I trying to create?
I'm trying to build an Json object with a Nested object which holds another nested object.
Example for use: store object => Store info => product list
What I expect to create
{
"Object": {
"id": 0,
"name": "Nova",
"nestedObject": {
{
"id": 1,
"name": "NestedNestedObject1",
},
{
"id": 2,
"name": "NestedNestedObject2",
},
}
Result I'm getting:
Error: Parse error on line 11:
...: { {
----------------------^
Expecting 'STRING', '}'
At NestedNestedObject2
How do I create a nested, nested object? If I'm correct mockable accepts pure Json
It depends on what you want to create and that depends on your API. The actual problem is that your JSON is not valid.
After your nestedObject there is just a { and that is wrong. In this case I assume you want to have an array of nestedObject (and perhaps also name should be nestedObjects) so fix would be (see the array []):
{
"Object": {
"id": 0,
"name": "Nova",
"nestedObject": [
{
"id": 1,
"name": "NestedNestedObject1"
},
{
"id": 2,
"name": "NestedNestedObject2"
}
]
}
}
I am writing my first api (express/node) and one of the endpoints receives json data in the body like:
{
"text": "some comment here...",
"tags": [
{"id": 0, "tag": "some tag 1"},
{"id": 123, "tag": "some tag 2"}
],
"date": "1452305028289",
}
Is there some way you can check that all the properties exist on the object and that they have values? Or do you have to write a custom function checking for each required property and values?
You can use one of these packages for validating data with NodeJS:
https://github.com/hapijs/joi
https://github.com/mafintosh/is-my-json-valid
https://github.com/ctavan/express-validator
A simple solution would be this function that takes an object and a list of strings as the properties of that object:
var checkProperties = function (obj, props) {
return props
.map(function(prop) { return obj.hasOwnProperty(prop); })
.reduce(function (p, q) { return p && q; });
}
use like this
checkProperties({ prop1: someValue, prop2: someOtherValue }, ["prop1", "prop2"]); // true