This question already has answers here:
Retrieve only the queried element in an object array in MongoDB collection
(18 answers)
Closed 4 years ago.
I have the followng json file
{
"movies": [{
"title" : "Star Wars: Episode I - The Phantom Menace",
"episode_number" : "1",
"main_characters" : ["Qui-Gon Jinn", "Obi-Wan Kenobi"],
"description" : "The evil Trade Federation, led by Nute Gunray is planning to take over the peaceful world of Naboo. ",
"poster" : "star_wars_episode_1_poster.png",
"hero_image" : "star_wars_episode_1_hero.jpg"
},
{
"title" : "Star Wars: Episode II - Attack of the Clones",
"episode_number" : "2",
"main_characters" : ["Obi-Wan Kenobi", "Anakin Skywalker"],
"description" : "Ten years after the 'Phantom Menace' threatened the planet Naboo, Padmé Amidala is now a Senator representing her homeworld.",
"poster" : "star_wars_episode_2_poster.png",
"hero_image" : "star_wars_episode_2_hero.jpg"
}
]
}
The collection name is fruits
I am trying to extract only the data for episode_number = 2. I have tried following queries
db.fruits.find({"movies": {$elemMatch:{"episode_number": "2"}}}).pretty();
which returns the entire document instead of just the data for episode_number 2.
and
db.fruits.find({"movies.episode_number": "2"}}}).pretty();
which also returns the entire document including that for episode_number 1
I am trying to figure out how to query this document so that only the data for episode_number = 2 is returned.
Please help.
find method takes two parameters: first one represents filtering condition and second one is for projection. To get one element from your array you can use $elemMatch in projection part:
db.fruits.find(
{"movies.episode_number": "2"},
{"movies": {$elemMatch:{"episode_number": "2"}}})
.pretty();
So first parameter filters out your collection and second one filters inner array
If you want all documents in the collection with movies array but it should contain only episode 2 details in it you need to filter the array.
https://docs.mongodb.com/manual/reference/operator/aggregation/filter/
or use $unwind, $match.
db.fruits.aggregate
([
{
$unwind: '$movies'
},
{
$match:{"movies.episode_number":'2'}
}
])
Related
I am using MySQL 5.7 and one of the columns in my table contains multiple JSON documents. Some thing like:
'[ {
"animal" : "dog",
"data" : {
"body" : "This sentence does not contain any thing about grooming",
}
},
{
"animal" : "cat",
"data" : {
"body" : "No grooming needed"
}
},
{
"animal" : "horse",
"data" : {
"body" : "He is grooming his horse after the ride."
}
}
]'
I want to return all rows where $.data.body contains grooming more than once, but only if $.animal == horse. So in the example given above it should not return the row since grooming is used only once in the section $.data.body where $.animal == horse.
Is there a good way to query this in MySql/SQL? I can do it in python but interested in knowing if there's a way to do this in SQL/MySQL. Thanks!
Searching JSON requires complex queries, and it is hard to optimize:
SELECT ...
FROM mytable
CROSS JOIN JSON_TABLE(myjsoncolumn, '$[*]' COLUMNS(
animal varchar(20) PATH '$.animal',
body text PATH '$.data.body'
)) AS j
WHERE j.animal = 'horse' AND j.body LIKE '%grooming%';
The JSON_TABLE() function is available in MySQL 8.0.4, but not earlier versions of MySQL.
The bottom line is that if you are trying to search the content of JSON documents, your use of SQL is going to be a lot more difficult and less efficient.
This would be far easier if you did not store the data in JSON, but instead stored data in normal rows and columns. From the example you show, there's no reason it needs to be JSON.
I'm having a problem with querying a MongoDB dataset ("On Street Crime in Camden" from data.gov.uk)
The database name is Crime_Data_in_Camden and the collection name is Street_Crime_Camden. The query to find all records, db.Street_Crime_Camden.find(), works fine but anything else returns nothing at
all. Here is a portion of the metadata:
{
"id" : 509935,
"name" : "Ward Name",
"dataTypeName" : "text",
"fieldName" : "ward_name",
"position" : 13,
"renderTypeName" : "text",
"tableColumnId" : 258836,
"width" : 100,
"cachedContents" : {
"largest" : "West Hampstead",
"non_null" : 79813,
"null" : 0,
"top" : [ {
"item" : "Regent's Park",
"count" : 20
}, {
"item" : "Swiss Cottage",
"count" : 19
}, {
"item" : "Holborn and Covent Garden",
"count" : 18
}
}
}
I've tried 3 attempts at a basic query:
db.Street_Crime_Camden.find({"ward_name":"West Hampstead"});
db.Street_Crime_Camden.find({'meta.ward_name':'West Hampstead'});
db.Street_Crime_Camden.find({meta:{ward_name:"West Hampstead"} });
According to any documentation or tutorial that I've seen any of these approaches should be valid. And I know that there are hundreds of rows (or documents) that match those terms, so why are these queries returning nothing? Advice would be appreciated.
The common theme in the three aproaches you tried is some form of ward_name = West Hampstead but there is no attribute named ward_name in the document you shared with us.
Based on the document you show in your question the only way of addressing an attribute with the value West Hampstead is:
db.Street_Crime_Camden.find({"cachedContents.largest": "West Hampstead"});
For background; you address attributes in your documents by using dot notation so the document you included in your question could be found by any of the following find commands:
db.Street_Crime_Camden.find({"name": "Ward Name"});
db.Street_Crime_Camden.find({"position": 13});
db.Street_Crime_Camden.find({"cachedContents.top.item": "Swiss Cottage"});
db.Street_Crime_Camden.find({"cachedContents.top.1.count": 20});
... etc
These examples might help you to understand how to form find criteria. The MongoDB docs are also useful.
I have a JSON file containing an array of objects (test.json):
[
{
"name": "Test 1",
"id": 1
},
{
"name": "Test 2",
"id": 2
},
{
"name": "Test 3",
"id": 3
}
]
I want to extract all objects, that have a certain ID. I managed to get an object if I want just one specific ID: jq 'map(select(.id == 2 ))' test.json.
Thing is, I have a list of IDs, say 1 and 3. How do I get a list containing only those object? So in this example a list containing the objects with ID 1 and 3?
You can check the example here: https://jqplay.org/s/xQgpA4yJAz
jq 'map(select(.id | contains(1,3)))'
Man, jq is so great
https://github.com/stedolan/jq/wiki/Cookbook#filter-objects-based-on-the-contents-of-a-key
The solution using contains/1 as presented on this page could just as well be written using ==:
map(select(.id == (1,3)))
The main reason for mentioning this is that contains is full of potential surprises. (Consider, for example, what would happen if .id were string-valued.)
Unfortunately, using either == or contains as above is computationally inefficient (it is O(m*n)), though in practice it is quite fast.
Let's say my JSON looks like this (example provided here) -
{
"year" : 2013,
"title" : "Turn It Down, Or Else!",
"info" : {
"directors" : [
"Alice Smith",
"Bob Jones"
],
"release_date" : "2013-01-18T00:00:00Z",
"rating" : 6.2,
"genres" : [
"Comedy",
"Drama"
],
"image_url" : "http://ia.media-imdb.com/images/N/O9ERWAU7FS797AJ7LU8HN09AMUP908RLlo5JF90EWR7LJKQ7##._V1_SX400_.jpg",
"plot" : "A rock band plays their music at high volumes, annoying the neighbors.",
"rank" : 11,
"running_time_secs" : 5215,
"actors" : [
"David Matthewman",
"Ann Thomas",
"Jonathan G. Neff"
]
}
}
I would like to query all movies where genres contains Drama.
I went through all of the examples but it seems that I can query only on hash key and sort key. I can't have JSON document as key itself as that is not supported.
You cannot. DynamoDB requires that all attributes you are filtering for have an index.
As you want to query independently of your main index, you are limited to Global Secondary Indexes.
The documentation lists on what kind of attributes indexes are supported:
The index key attributes can consist of any top-level String, Number, or Binary attributes from the base table; other scalar types, document types, and set types are not allowed.
Your type would be an array of Strings. So this query operation isn't supported by DynamoDB at this time.
You might want to consider other NoSQL document based databases which are more flexible like MongoDB Atlas, if you need this kind of querying functionality.
String filterExpression = "coloumnname.info.genres= :param";
Map valueMap = new HashMap();
valueMap.put(":param", "Drama");
ItemCollection scanResult = table
.scan(new ScanSpec().
withFilterExpression(filterExpression).
withValueMap(valueMap));
One example that I took from AWS Developer Forums is as follows.
We got some hints for you from our team. Filter/condition expressions for maps have to have key names at each level of the map specified separately in the expression attributeNames map.
Your expression should look like this:
{
"TableName": "genericPodcast",
"FilterExpression": "#keyone.#keytwo.#keythree = :keyone",
"ExpressionAttributeNames": {
"#keyone": "attributes",
"#keytwo": "playbackInfo",
"#keythree": "episodeGuid"
},
"ExpressionAttributeValues": {
":keyone": {
"S": "podlove-2018-05-02t19:06:11+00:00-964957ce3b62a02"
}
}
}
I would like to store ticket details in an array in a mongo document. it works fine till the document size reaches 16MB, after that I get the exception (Resulting document after update is larger than 16777216) and program terminates abruptly. I cannot split this document coz it stores all the ticket details falls under that year 2016.
here goes my document structure.
{
year:2016,
purpose: ticketdetail,
tickets:[
{ticketid:001, desc:"xyz", created:20161231},
{ticketid:002, desc:"xyz", created:20161231},
{ticketid:003, desc:"xyz", created:20161231},
.......
.......
{ticketid:00N, desc:"xyz", created:20161231},
}]
}
You will need to split your document into separate documents, perhaps in a different collection. I would not recommend GridFS, because you cannot query data within a GridFS blob.
Here is a suggested document structure:
{
_id: ObjectId("85bf0ef0b9692c0010978359"),
"ticketid" : "001",
"desc" : "xyz",
"created" : ISODate("2016-12-31T00:00:00.000Z")
}
,
{
_id: ObjectId("85bed4257726f90010d4e21f"),
"ticketid" : "002",
"desc" : "xyz",
"created" : ISODate("2016-12-31T00:00:00.000Z")
}
Notes on this structure:
Each ticket is in a different document - this makes it scalable, because there is no limit on the number of documents in a collection.
The "created" field is now in a proper date field. This gives you more accurate queryability.
You said that your original document was needed to store all tickets for 2016. A suitable query on this new collection will return you all tickets for 2016, so you don't need to store them all in a single document:
db.tickets.find({
"created" : {
$gte: ISODate("2016-01-01"),
$lt: ISODate("2017-01-01")
}
}
});