I'm having a problem with querying a MongoDB dataset ("On Street Crime in Camden" from data.gov.uk)
The database name is Crime_Data_in_Camden and the collection name is Street_Crime_Camden. The query to find all records, db.Street_Crime_Camden.find(), works fine but anything else returns nothing at
all. Here is a portion of the metadata:
{
"id" : 509935,
"name" : "Ward Name",
"dataTypeName" : "text",
"fieldName" : "ward_name",
"position" : 13,
"renderTypeName" : "text",
"tableColumnId" : 258836,
"width" : 100,
"cachedContents" : {
"largest" : "West Hampstead",
"non_null" : 79813,
"null" : 0,
"top" : [ {
"item" : "Regent's Park",
"count" : 20
}, {
"item" : "Swiss Cottage",
"count" : 19
}, {
"item" : "Holborn and Covent Garden",
"count" : 18
}
}
}
I've tried 3 attempts at a basic query:
db.Street_Crime_Camden.find({"ward_name":"West Hampstead"});
db.Street_Crime_Camden.find({'meta.ward_name':'West Hampstead'});
db.Street_Crime_Camden.find({meta:{ward_name:"West Hampstead"} });
According to any documentation or tutorial that I've seen any of these approaches should be valid. And I know that there are hundreds of rows (or documents) that match those terms, so why are these queries returning nothing? Advice would be appreciated.
The common theme in the three aproaches you tried is some form of ward_name = West Hampstead but there is no attribute named ward_name in the document you shared with us.
Based on the document you show in your question the only way of addressing an attribute with the value West Hampstead is:
db.Street_Crime_Camden.find({"cachedContents.largest": "West Hampstead"});
For background; you address attributes in your documents by using dot notation so the document you included in your question could be found by any of the following find commands:
db.Street_Crime_Camden.find({"name": "Ward Name"});
db.Street_Crime_Camden.find({"position": 13});
db.Street_Crime_Camden.find({"cachedContents.top.item": "Swiss Cottage"});
db.Street_Crime_Camden.find({"cachedContents.top.1.count": 20});
... etc
These examples might help you to understand how to form find criteria. The MongoDB docs are also useful.
Related
How do I access the JSON array to display the output of "AdjustedScheduleTime" from the Trip section?
I got it working for StopLabel as shown below, but I'm struggling to access AdjustedScheduleTime.
I tried the following:
["GetNextTripsForStopResponse"]["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"]["AdjustedScheduleTime"]
but doesn't work.
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view, typically from a nib.
let parameters = [
"appID": "5rt5rydg", //incorrect appID
"apiKey": "3b5fb15rdgy5454hdrfhr", //incorrect apiKey
"routeNo": "14",
"stopNo": "8600",
"format": "JSON"
]
AF.request("https://api.octranspo1.com/v1.2/GetNextTripsForStop?", method: .post, parameters: parameters,encoding:
URLEncoding.httpBody, headers: nil).responseJSON{ response in
let swiftyJsonVar = JSON(response.result.value!)
print(swiftyJsonVar)
if let busInfo = swiftyJsonVar["GetNextTripsForStopResult"]["StopLabel"].string {
print(": ",busInfo)
print("Label1: ", self.label1.text = busInfo)
}
}
}
This is the results:
{
"GetNextTripsForStopResult" : {
"Error" : "",
"Route" : {
"RouteDirection" : {
"RouteLabel" : "St-Laurent",
"Error" : "",
"RequestProcessingTime" : "20190112151425",
"Trips" : {
"Trip" : [
{
"AdjustmentAge" : "0.38",
"GPSSpeed" : "0.5",
"Latitude" : "45.429457",
"Longitude" : "-75.684117",
"TripDestination" : "St-Laurent",
"LastTripOfSchedule" : false,
"TripStartTime" : "14:31",
"BusType" : "4LB - IN",
"AdjustedScheduleTime" : "11"
},
{
"AdjustmentAge" : "4.32",
"GPSSpeed" : "0.5",
"Latitude" : "45.413749",
"Longitude" : "-75.689748",
"TripDestination" : "St-Laurent",
"LastTripOfSchedule" : false,
"TripStartTime" : "14:46",
"BusType" : "4LB - IN",
"AdjustedScheduleTime" : "22"
},
{
"AdjustmentAge" : "0.55",
"GPSSpeed" : "31.3",
"Latitude" : "45.399587",
"Longitude" : "-75.727631",
"TripDestination" : "St-Laurent",
"LastTripOfSchedule" : false,
"TripStartTime" : "15:01",
"BusType" : "4L - IN",
"AdjustedScheduleTime" : "37"
}
]
},
"RouteNo" : 14,
"Direction" : "Eastbound"
}
},
"StopLabel" : "MCARTHUR \/ IRWIN MILLER",
"StopNo" : "8600"
}
}
: MCARTHUR / IRWIN MILLER //This is the desired output for StopLabel
Ok, so do you explain JSON. Here's a shot.
First some rules:
When you see opening { it means dictionary, you have to pick a key next
When you see opening [ it means array. you have to pick an index
When you see "SomeString": its a key in an array.
Dictionaries have keys, arrays have index. Pick accordingly..
So when we walk through this response:
We see that we start with {. We have a dictionary! We're expecting to see some keys next.
So lets pick a key: We only have one and it's "GetNextTripsForStopResult". so far we have: swiftyJsonVar["GetNextTripsForStopResult"]
We now look at the content of "GetNextTripsForStopResult". We see it's also a dictionary. Again we should have some keys. We do. We have Error, Route, StopLabel and more. Let's pick a key. Since we're trying to get to a "AdjustedScheduleTime", lets pick Route. so far we have ["GetNextTripsForStopResult"]["Route"]
Now lets look at the contents of Route. Its a dictionary again.
Again we pick a key and keep repeating till we hit Trip. You should have ["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"]
Lets look at what we have in Trip Whats this?..its an array!
We have to pick an index now. We need to chose somehow. Thats the tricky part. In order to do that we need some more information. So lets just ARBITRARILY chose one. Lets take the last one. so we have: ["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"][2]
Now we can get our final key AdjustedScheduleTime. So let's pick it!
["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"][2]["AdjustedScheduleTime"]
Keep in mind:
These hard coded indexes are almost NEVER what you want. Maybe you need to show all the AdjustedScheduleTime to the user or let the user chose one, or add all of them up. That really depends on your application and what you're trying to accomplish. I chose the last index (2) arbitrarily without having any knowledge of your application, the api you're calling and what you're trying to achieve. Its VERY possible that you don't want the last index.
Let's say my JSON looks like this (example provided here) -
{
"year" : 2013,
"title" : "Turn It Down, Or Else!",
"info" : {
"directors" : [
"Alice Smith",
"Bob Jones"
],
"release_date" : "2013-01-18T00:00:00Z",
"rating" : 6.2,
"genres" : [
"Comedy",
"Drama"
],
"image_url" : "http://ia.media-imdb.com/images/N/O9ERWAU7FS797AJ7LU8HN09AMUP908RLlo5JF90EWR7LJKQ7##._V1_SX400_.jpg",
"plot" : "A rock band plays their music at high volumes, annoying the neighbors.",
"rank" : 11,
"running_time_secs" : 5215,
"actors" : [
"David Matthewman",
"Ann Thomas",
"Jonathan G. Neff"
]
}
}
I would like to query all movies where genres contains Drama.
I went through all of the examples but it seems that I can query only on hash key and sort key. I can't have JSON document as key itself as that is not supported.
You cannot. DynamoDB requires that all attributes you are filtering for have an index.
As you want to query independently of your main index, you are limited to Global Secondary Indexes.
The documentation lists on what kind of attributes indexes are supported:
The index key attributes can consist of any top-level String, Number, or Binary attributes from the base table; other scalar types, document types, and set types are not allowed.
Your type would be an array of Strings. So this query operation isn't supported by DynamoDB at this time.
You might want to consider other NoSQL document based databases which are more flexible like MongoDB Atlas, if you need this kind of querying functionality.
String filterExpression = "coloumnname.info.genres= :param";
Map valueMap = new HashMap();
valueMap.put(":param", "Drama");
ItemCollection scanResult = table
.scan(new ScanSpec().
withFilterExpression(filterExpression).
withValueMap(valueMap));
One example that I took from AWS Developer Forums is as follows.
We got some hints for you from our team. Filter/condition expressions for maps have to have key names at each level of the map specified separately in the expression attributeNames map.
Your expression should look like this:
{
"TableName": "genericPodcast",
"FilterExpression": "#keyone.#keytwo.#keythree = :keyone",
"ExpressionAttributeNames": {
"#keyone": "attributes",
"#keytwo": "playbackInfo",
"#keythree": "episodeGuid"
},
"ExpressionAttributeValues": {
":keyone": {
"S": "podlove-2018-05-02t19:06:11+00:00-964957ce3b62a02"
}
}
}
This is the structure of a document I have in one monggodb collection.
I wanted to understand how one can do a mongo aggregate of grouped count over key "code" and the index position in the nested json (not the priority as it can be any number but within schedules nested there can be just 5 values):
{
"_id" : ObjectId("5749e9fde4b0064e7362b560"),
"_class" : "com.weirdcompanyname.core.collectionname",
"rfId" : 1,
"scheduleds" : [
{
"code" : "556e4835f1eae40bdfa2f2001f2afc76",
"type" : "HT",
"priority" : 0
},
{
"code" : "8b2ab67af4f60e42f7ea64813b5795cf",
"type" : "HT",
"priority" : 1
},
{
"code" : "ed17101eb918b4d8c7c598e4884523ea",
"type" : "HT",
"priority" : 2
},
{
"code" : "7e0ffb4db",
"type" : "QZ",
"priority" : 3
},
{
"code" : "1453dfa1794f39b05f0259ad04699073",
"type" : "HT",
"priority" : 4
}
],
"created" : ISODate("2016-05-28T18:57:00.878Z")
}
The result I'm trying to find is:
code index_position count
556e4835f1eae40bdfa2f2001f2afc76 0 100
8b2ab67af4f60e42f7ea64813b5795cf 1 100
ed17101eb918b4d8c7c598e4884523ea 2 100
7e0ffb4db 3 100
1453dfa1794f39b05f0259ad04699073 4 100
I could get my head around unwinding the nested json in single arrays and then grouping the code over code and maybe other column, let's say priority and have the count but the problem is to get the index position.
Is this even doable on mongo, I've read around a lot of stuff about it and I figured if I have value for which I need a position then it can be doable but I don't really have a value to look for, what I'm looking for is each code and its index position in the "scheduleds" and count.
This is what I could do with my limited mongo querying skills:
db.collectionname.aggregate([{'$match':{'date_key':{'$gte': yesterday_beginning, '$lte': yesterday_end}}}, {'$unwind':'$scheduleds'}, {'$group':{'_id':{'code':'$scheduleds.code','priority':'$scheduleds.priority'}, 'rfid':{'$addToSet':'$rfId'}}}, {'$project':{'_id':0, 'code':'$_id.code', 'priority':'$_id.priority', 'totalRfid':{'$size':'$rfid'}}}, { $limit : 1000 }],{ allowDiskUse:true})
Alain1405 says here that MongoDB 3.2 supports unwinding of the array index.
Instead of passing a path the $unwind operator, you can pass an
object with the field path and the field includeArrayIndex which
will hold the array index.
From MongoDB official documentation:
{
$unwind:
{
path: <field path>,
includeArrayIndex: <string>,
preserveNullAndEmptyArrays: <boolean>
}
}
I have a rather complex structure on my json and I cannot find how to query it to get the rows I am interested in. Here is a sample of my data:
{
"_id" : ObjectId("5282bf9ce4b05216ca1b68f8"),
"authorID" : ObjectId("5282a8c3e4b0d7f4f4d07b9a"),
"blogID" : "7180831558698033600",
"blogs" : {
"$" : {
"posts" : [
[
{
"author" : {
"displayName" : "mms",
...
...
...
}}}
So, I am interested in finding all json entries that have the author displayName equal to "mms".
My collection name is bz so, a find all query would be: db.dz.find()
What criteria do I have to put inside the find() to only get json document with author displayName equal to mms?
Any ideas?
Thank you in advance!
Suppose you have replaced field name "$" with "dollarSign".
Then db.dz.find({"blogs.dollarSign.posts.author.displayName": "mms"}) will fetch whole documents according to your requirements.
Ahoy! I'm having a very funny issue with MongoDB and, possibly more in general, with JSON. Basically, I accidentally created some MongoDB documents whose subdocuments contain an empty key, e.g. (I stripped ObjectIDs to make the code look nicer):
{
"_id" : ObjectId("..."),
"stats" :
{
"violations" : 0,
"cost" : 170,
},
"parameters" :
{
"" : "../instances/comp/comp20.ectt",
"repetition" : 29,
"time" : 600000
},
"batch" : ObjectId("..."),
"system" : "Linux 3.5.0-27-generic",
"host" : "host3",
"date_started" : ISODate("2013-05-14T16:46:46.788Z"),
"date_stopped" : ISODate("2013-05-14T16:56:48.483Z"),
"copy" : false
}
Of course the problem is line:
"" : "../instances/comp/comp20.ectt"
since I cannot get back the value of the field. If I query using:
db.experiments.find({"batch": ObjectId("...")}, { "parameters.": 1 })
what I get is the full content of the parameters subdocument. My guess is that . is probably ignored if followed by an empty selector. From the JSON specification (15.12.*) it looks like empty keys are allowed. Do you have any ideas about how to solve that?
Is that a known behavior? Is there a use for that?
Update I tried to $rename the field, but that won't work, for the same reasons. Keys that end with . are not allowed.
Update filed issue on MongoDB issue tracker.
Thanks,
Tommaso
I have this same problem. You can select your sub-documents with something like this:
db.foo.find({"parameters.":{$exists:true}})
The dot at the end of "parameters" tells Mongo to look for an empty key in that sub-document. This works for me with Mongo 2.4.x.
Empty keys are not well supported by Mongo, I don't think they are officially supported, but you can insert data with them. So you shouldn't be using them and should find the place in your system where these keys are inserted and eliminate it.
I just checked the code and this does not currently seem possible for the reasons you mention. Since it is allowed to create documents with zero length field names I would consider this a bug. You can report it here : https://jira.mongodb.org
By the way, ironically you can query on it :
> db.c.save({a:{"":1}})
> db.c.save({a:{"":2}})
> db.c.find({"a.":1})
{ "_id" : ObjectId("519349da6bd8a34a4985520a"), "a" : { "" : 1 } }