I have a collection that looks something like this like this (each array element is 1 document):
[
{
"_id": 1,
"title": "title",
"data": [
{
"_id": 41,
"name": "test 1"
},
{
"_id": 42,
"name": "test 2"
},
{
"_id": 43,
"name": "test 3"
}
]
},
{
"_id": 1,
"title": "title 2",
"data": [
{
"_id": 44,
"name": "test 4"
},
{
"_id": 45,
"name": "test 5"
}
]
},
]
What I'd like to do is filter out all documents using $redact where data._id is $in an array of [41, 42]. So, if data._id is ever equal to 41 or 42, the whole document should be kept. Else, it should be pruned.
{
$cond: {
if: {
$in: [
"$data._id",
[ 41, 42 ]
]
},
then: "$$KEEP",
else: "$$PRUNE"
}
}
However, the query above doesn't return anything when I use the $redact stage in Compass.
I think you are missing the $redact aggregation, apart from that the query looks fine https://www.mongodb.com/docs/manual/reference/operator/aggregation/in/
{
$redact: {
$cond: {
if: {
$in: [
"$data._id",
[ 41, 42 ]
]
},
then: "$$KEEP",
else: "$$PRUNE"
}
}
}
Related
I met a problem in flattening the json into a relational table.
for example I have a json file like the below,
How can I flatten the table content in both sheets:'sheet:1':'section 1':table
and sheets:'sheet:2':'section 1':table ??
the number of sheet and section changes in each json file.
is there any way to use regular expression in the json path?
each json path of each file is following the same pattern, but the keys are not the same...
{
"extraction date": {
"month": "OCTOBER",
"monthValue": 10,
"year": 2020
},
"fileName": "test_1.xls",
"number of sheets": 2,
"sheets": {
"sheet:1": {
"content": {
"conversion state": "Success",
"section 1": {
"meta": {
"Remark": "This is the remark",
"Row: 4": "this is the title"
},
"table": [
{
"col1": null,
"col2": "2020-07-14"
"Row": 9
},
{
"col1": null,
"col2": "2020-07-14"
"Row": 10
}
]
}
},
"name": "Sheet1",
"sections": 1
},
"sheet:2": {
"content": {
"conversion state": "Success",
"section 1": {
"meta": {
"Remark": " null",
"Row: 4": "title a"
},
"table": [
{
"col1": null,
"col2": "2020-07-14",
"Row": 8
},
{
"col1": null,
"col2": "2020-07-14",
"Row": 9
}
]
}
},
"name": "mySheetName",
"sections": 1
}
}
}
I want to read hyperopt parameters from a JSON file.
My JSON file would be like:
[
{
"id": "121",
"model": [
{
"model_name": "power",
"estimator_type": [
{
"type": "Polynomial",
"degree": [2, 3, 4]
},
{
"type": "svm",
"C": [0, 1],
"kernel": [
{
"ktype": "linear"
},
{
"ktype": "RBF",
"width": [0, 1]
}
]
}
],
"cut_values": {
"qids": ["1234"]
}
},
{
"model_name": "speed",
"estimator_type": [
{
"type": "Polynomial",
"degree": ["quniform", 2, 3]
}
],
"cut_values": null
}
]
},
{
"id": "123",
"model": [
{
"model_name": "power",
"estimator_type": [
{
"type": "LinearRegression"
}
],
"cut_values": null
}
]
}
]
I have checked this post but with no success for more complex JSON like the one above.
I want to be able to create a space like 2.2 A Search Space Example: scikit-learn.
I have an array of JSON objects formatted as follows:
[
{
"id": 1,
"names": [
{
"name": "Bulbasaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 1,
"types": [
{
"slot": 1,
"type": {
"name": "grass",
"url": "http://myserver.com:8000/api/v2/type/12/"
}
},
{
"slot": 2,
"type": {
"name": "poison",
"url": "http://myserver.com:8000/api/v2/type/4/"
}
}
]
},
{
"id": 2,
"names": [
{
"name": "Ivysaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 2,
"types": [
{
"slot": 1,
"type": {
"name": "ice",
"url": "http://myserver.com:8000/api/v2/type/10/"
}
},
{
"slot": 2,
"type": {
"name": "electric",
"url": "http://myserver.com:8000/api/v2/type/8/"
}
}
]
},
{
"id": 3,
"names": [
{
"name": "Venusaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
},
{
"id": 3,
"types": [
{
"slot": 1,
"type": {
"name": "ground",
"url": "http://myserver.com:8000/api/v2/type/2/"
}
},
{
"slot": 2,
"type": {
"name": "rock",
"url": "http://myserver.com:8000/api/v2/type/3/"
}
}
]
}
]
Note that these are pairs of separate objects that appear sequentially in a JSON array, with each pair sharing an id field. This pattern repeats several hundred times in the array. What I need to accomplish is to "merge" each id-sharing pair into one object. So, the resultant output would be
[
{
"id": 1,
"names": [
{
"name": "Bulbasaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "grass",
"url": "http://myserver.com:8000/api/v2/type/12/"
}
},
{
"slot": 2,
"type": {
"name": "poison",
"url": "http://myserver.com:8000/api/v2/type/4/"
}
}
]
},
{
"id": 2,
"names": [
{
"name": "Ivysaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "ice",
"url": "http://myserver.com:8000/api/v2/type/10/"
}
},
{
"slot": 2,
"type": {
"name": "electric",
"url": "http://myserver.com:8000/api/v2/type/8/"
}
}
]
},
{
"id": 3,
"names": [
{
"name": "Venusaur",
"language": {
"name": "en",
"url": "http://myserver.com:8000/api/v2/language/9/"
}
},
],
"types": [
{
"slot": 1,
"type": {
"name": "ground",
"url": "http://myserver.com:8000/api/v2/type/2/"
}
},
{
"slot": 2,
"type": {
"name": "rock",
"url": "http://myserver.com:8000/api/v2/type/3/"
}
}
]
}
]
I've gotten these objects to appear next to each other via the group_by(.id) command, but I'm at a loss as to how I should actually combine them. I'm very much still a novice with jq so I'm a bit overwhelmed with the amount of possible solutions.
[Note: The following assumes that the data shown in the Q have been corrected so that they are valid JSON.]
The merging you want can be achieved by object addition (x + y). For example, given the two JSON objects as shown in the question (i.e., as a stream), you could write:
jq -s '.[0] + .[1]'
However, since the question also indicates these objects are actually in an array, let's next consider the case of an array with two objects. In that case, you could simply write:
jq add
Finally, if you have an array of arrays each of which is an array of objects, you could use map(add). Since you don't have a very large array, you could simply write:
group_by(.id) | map(add)
Please note that jq defines object addition in a non-commutative way. Specifically, there is a bias towards the right-most key.
I am new to python, using python3. I have json data like:
{
"message": {
"count": 46,
"limit": 1000,
"schools": [
{
"class": "1",
"class_id": "1c8***",
"charges": [
{
"cost": 10,
"breakdown": [
{
"books": "1",
"unitQuantity": "10"
}
]
}
],
"area": "maccau"
},
{
"class": "2",
"class_id": "1c3***",
"charges": [
{
"cost": 100,
"breakdown": [
{
"books": "1",
"unitQuantity": "100"
}
]
}
],
"area": "maccau"
},
{
"class": "1",
"class_id": "1c3***",
"charges": [
{
"cost": 10,
"breakdown": [
{
"books": "1",
"unitQuantity": "10"
}
]
}
],
"area": "maccau"
},
{
"class": "2",
"class_id": "1c8***",
"charges": [
{
"cost": 50,
"breakdown": [
{
"books": "1",
"unitQuantity": "50"
}
]
}
],
"area": "maccau"
}
],
"url": {
"link": "/"
}
}
}
I was able to use json.loads to load data and I am trying to get results like:
class Cost
1 20
2 150
I tried converting json to a dictionary:
item_dict = json.load(json_data)
Tried to get data out using for loop and checking if class = 1 and then summing up the cost. But I feel like that is not the best approach. Can someone please tell me what would be the best way of doing this?
This is the JSON response I am trying to parse:
{
"data": {
"Content": {
"id": 26,
"name": "Dashboard1"
},
"List": [
{
"ListContent": {
"id": 178,
"name": "Card-144"
},
"cards": [
{
"id": 1780,
"configuration": {
"id": 7178,
"name": "Emp"
}
}
]
},
{
"ListContent": {
"id": 179,
"name": "Card-14"
},
"cards": [
{
"id": 1798,
"configuration": {
"id": 1789,
"name": "RandomColumns"
}
}
]
},
{
"ListContent": {
"id": 180,
"name": "Card-1"
},
"cards": [
{
"id": 18080,
"configuration": {
"id": 1080,
"allow": true
}
}
]
},
{
"ListContent": {
"id": 181,
"name": "Card-14"
},
"cards": [
{
"id": 18081,
"configuration": {
"id": 1881,
"name": "Functions"
}
}
]
},
{
"ListContent": {
"id": 182,
"name": "Card-1443"
},
"cards": [
{
"id": 1782,
"configuration": {
"id": 1802,
"name": "Emp-O"
}
}
]
}
]
}
}
From the Json, I need to extract "id"s under the "ListContent" nodes and store it in an array. Also, will need to ignore "id"s under the child nodes.
Here is a groovy script I am trying to achieve this with,
def CList = ""
import groovy.json.JsonSlurper
def jsonRespData = context.expand( '${TestStep#Response#$.data.List}' )
def outputResp = new JsonSlurper().parseText(jsonRespData)
outputResp.id.each()
{log.info( ":"+ it)
CList=CList.concat(it.toString()).concat(',')}
log.info (CList)
So, the array that I am expecting is CList [178,179,180,181,182]
but I am currently getting null.
What should be the correct groovy to only read "id" from "ListContent" and write it to an array?
Any help would be really appreciated.
Thanks in advance.
You can just use the (implicit) spread operator like this:
def json = new groovy.json.JsonSlurper().parse('/tmp/x.json' as File)
//
def i = json.data.List.ListContent.id
assert i == [178, 179, 180, 181, 182]
// with explicit spread operator
def e = json.data.List*.ListContent*.id
assert e == [178, 179, 180, 181, 182]
def str = '''
{
"data": {
"Content": {
"id": 26,
"name": "Dashboard1"
},
"List": [
{
"ListContent": {
"id": 178,
"name": "Card-144"
},
"cards": [
{
"id": 1780,
"configuration": {
"id": 7178,
"name": "Emp"
}
}
]
},
{
"ListContent": {
"id": 179,
"name": "Card-14"
},
"cards": [
{
"id": 1798,
"configuration": {
"id": 1789,
"name": "RandomColumns"
}
}
]
},
{
"ListContent": {
"id": 180,
"name": "Card-1"
},
"cards": [
{
"id": 18080,
"configuration": {
"id": 1080,
"allow": true
}
}
]
},
{
"ListContent": {
"id": 181,
"name": "Card-14"
},
"cards": [
{
"id": 18081,
"configuration": {
"id": 1881,
"name": "Functions"
}
}
]
},
{
"ListContent": {
"id": 182,
"name": "Card-1443"
},
"cards": [
{
"id": 1782,
"configuration": {
"id": 1802,
"name": "Emp-O"
}
}
]
}
]
}
}
'''
def outputResp = new groovy.json.JsonSlurper().parseText(str)
outputResp.data.List.collect { it.ListContent.id }
As you already have List from (context.expand( '${TestStep#Response#$.data.List}' )) , you can simply do:
outputResp.collect { it.ListContent.id }
Above returns an ArrayList.