So when I parse a file with EvaluateJSON the JSON looks like this:
{ "TEST_DATE": "", "T_DATE": "" }
When I do EvaluateJSON ($.TEST_DATE) to "test-date" on attributes... I get:
test-date: ""
Then when I do AttributesToJSON (flowfile-content destination) I get:
{ "test-date": "", "t-date": "" }
HOWEVER... I want it to be:
{ "test-date": null, "t-date": null }
I have tried every possible option. There is no way aside from "ReplaceText"-style dangerous regex to put NULL into the JSON.
Any updateAttribute fails to put "null" into it. I tried "replaceEmpty(null)", replaceEmpty("null") (which puts a string "null" instead). I tried "replaceEmpty(literal("null")) doesn't work.
It's like Nifi doesn't recognize null.
DExter,
You can replace the double quotes("") by null value in ReplaceText processor.
Afterwards you get below value.
{ "test-date": "", "t-date": "" }
Use ReplaceText processor to be search for empty double quotes and replace it with null.
search value:""
Replacement value:null
For your reference check this https://regexr.com/3kctp.
It will end like your required result;
{ "test-date": null, "t-date": null }
Please let me know, if you face any issues.
Related
I'm trying to create a variable from an aggregate query in MongoDB and then use it to make another query.
var test_var = db.d_snapshot4.aggregate([{$group : {_id: null,
max_snapshot_date: {$max:"$snapshot_date"},
max_snapshot_date_str:
{$max:"$snapshot_date_str"}
}
}
]);
The content of test_var is as follows:
{
"_id": null,
"max_snapshot_date": ISODate("2020-05-31T00:00:00.000Z"),
"max_snapshot_date_str": "20200531"
}
But when I try to see the result of test_var.max_snapshot_date I get nothing back.
I need to use the variable as follows:
db.d_snapshot4.aggregate([{$match: {"snapshot_date": {$gte: test_var.max_snapshot_date } }}]);
Any suggestions?
Thanks,
Dina
Did you try with test_var["max_snapshot_date"]? Or do you get any error?
I have JSON such as:
{ "message": "hi" }
But it can also be of the format:
{ "message": { "action": "foo" } }
I want to filter out any records where the message.action == "foo" IF message.action even exists.
If I use the command:
jq 'select(.message.action? == null or .message.action? != "foo" )'
Then I get zero results. This appears to be because once you check for action, it then filters out any messages that are not objects, but I still want to be able to display { message: "hi" }
Check if message points to an object which has the key action whose value is foo instead, and take logical complement of the result. There's no harm in typing a few more letters.
select(.message | type == "object" and has("action") and .action == "foo" | not)
You can use the alternative operator // to provide a default value for action when it does not exist (whether because it is missing or because the value is not an object in the first place):
jq 'select((.message.action? // "foo") != "foo")'
This will accept either {"message": "hi"} or {"message": {"action": "not foo"}}, but not {"message": {"action": "foo"}}.
I'm creating an AWS Step Function definition in Dhall. However, I don't know how to create a common structure they use for Choice states such as the example below:
{
"Not": {
"Variable": "$.type",
"StringEquals": "Private"
},
"Next": "Public"
}
The Not is pretty straightforward using mapKey and mapValue. If I define a basic Comparison:
{ Type =
{ Variable : Text
, StringEquals : Optional Text
}
, default =
{ Variable = "foo"
, StringEquals = None Text
}
}
And the types:
let ComparisonType = < And | Or | Not >
And adding a helper function to render the type as Text for the mapKey:
let renderComparisonType = \(comparisonType : ComparisonType )
-> merge
{ And = "And"
, Or = "Or"
, Not = "Not"
}
comparisonType
Then I can use them in a function to generate the record halfway:
let renderRuleComparisons =
\( comparisonType : ComparisonType ) ->
\( comparisons : List ComparisonOperator.Type ) ->
let keyName = renderComparisonType comparisonType
let compare = [ { mapKey = keyName, mapValue = comparisons } ]
in compare
If I run that using:
let rando = ComparisonOperator::{ Variable = "$.name", StringEquals = Some "Cow" }
let comparisons = renderRuleComparisons ComparisonType.Not [ rando ]
in comparisons
Using dhall-to-json, she'll output the first part:
{
"Not": {
"Variable": "$.name",
"StringEquals": "Cow"
}
}
... but I've been struggling to merge that with "Next": "Sup". I've used all the record merges like /\, //, etc. and it keeps giving me various type errors I don't truly understand yet.
First, I'll include an approach that does not type-check as a starting point to motivate the solution:
let rando = ComparisonOperator::{ Variable = "$.name", StringEquals = Some "Cow" }
let comparisons = renderRuleComparisons ComparisonType.Not [ rando ]
in comparisons # toMap { Next = "Public" }
toMap is a keyword that converts records to key-value lists, and # is the list concatenation operator. The Dhall CheatSheet has a few examples of how to use both of them.
The above solution doesn't work because # cannot merge lists with different element types. The left-hand side of the # operator has this type:
comparisons : List { mapKey : Text, mapValue : Comparison.Type }
... whereas the right-hand side of the # operator has this type:
toMap { Next = "Public" } : List { mapKey : Text, mapValue : Text }
... so the two Lists cannot be merged as-is due to the different types for the mapValue field.
There are two ways to resolve this:
Approach 1: Use a union whenever there is a type conflict
Approach 2: Use a weakly-typed JSON representation that can hold arbitrary values
Approach 1 is the simpler solution for this particular example and Approach 2 is the more general solution that can handle really weird JSON schemas.
For Approach 1, dhall-to-json will automatically strip non-empty union constructors (leaving behind the value they were wrapping) when translating to JSON. This means that you can transform both arguments of the # operator to agree on this common type:
List { mapKey : Text, mapValue : < State : Text | Comparison : Comparison.Type > }
... and then you should be able to concatenate the two lists of key-value pairs and dhall-to-json will render them correctly.
There is a second solution for dealing with weakly-typed JSON schemas that you can learn more about here:
Dhall Manual - How to convert an existing YAML configuration file to Dhall
The basic idea is that all of the JSON/YAML integrations recognize and support a weakly-typed JSON representation that can hold arbitrary JSON data, including dictionaries with keys of different shapes (like in your example). You don't even need to convert the entire the expression to this weakly-typed representation; you only need to use this representation for the subset of your configuration where you run into schema issues.
What this means for your example, is that you would change both arguments to the # operator to have this type:
let Prelude = https://prelude.dhall-lang.org/v12.0.0/package.dhall
in List { mapKey : Text, mapValue : Prelude.JSON.Type }
The documentation for Prelude.JSON.Type also has more details on how to use this type.
In PHP when I want to give a name to an array of json objects I use this line.
$json = json_encode(array("users" => $output));
And this variable would be printed like this.
{"users":[{"user_id":"1"}
But now, I'm building a project in Node.js and I need to give a name to an array, and I don't know how to do it, I'm printing data using this line:
res.send(JSON.stringify(users, null, 4));
[ { "user_id": "1" }, { "user_id": "2" } ]
Thank you.
Just build the object with "root" key users before stringify
res.send( JSON.stringify( { 'users': users } , null, 4) );
You can send json directly specifying data points
res.json({ variablename : thevalue (in your case array)})
if more than one value, just follow the syntax
res.json({ variablename : thevalue1, variablename : value2)})
You can something like this:
{'users': JSON.stringify(users, null, 4)}
Consider I have following typed JSON objects:
Parent: {
"field1" : "Value of field1"
"fieldC" : {Child}
}
Child: {
"field2" : "Value of field2"
}
Q: What do I call field1 and field2?
Just Strings?
Q: What to i call the "path" fieldC.field2?
Accessor path?
Field path?
Member hierarcy path?
field1 and field2 are just strings.
[anything, ..., ... ] is just an array, so the elements of an object.
and then you have 0-9 (with decimals, negative, positive or with e), true/false and null, as numeric values, boolean and nullvalue
{Child} is an object. I don't think it's called path (I'd say that's opinion-based). maybe field-path, but it's rather a child-object. the key is a string and the value is an object/array/string/bool/null/numeric or decimal
all the possibilities e.g.:
{
"string": "string-value",
"nulltype": null,
"child_object": {
"boolean": true,
"any_decimal_int": -1.5e3
},
"array_values":[
{
"any_value": true
},
{
"any_value": false
}
]
}
of course you can combine more and have unlimited child-objects and lists :)
jsonapi.org seems to refer field1,fieldC,and field2 as member names, which I find much more descriptive than just 'Strings'.
As mentioned in my comment to first answer, I guess I'll personally be using (hierarchical) property path or just (object) member hierarchy while referring to 'writing open' the object-hierarchical property/attribute/member 'path' such as fieldC.field2. Seems to be alot of room for interpretation in that. : ]