I am new to this site and this is my first question. I am trying to query the "Seattle Real Time Fire 911 Calls" database from the Socrata Seattle Open Data site: https://data.seattle.gov/Public-Safety/Seattle-Real-Time-Fire-911-Calls/kzjm-xkqj. I'm not an expert at using the SODA API, and I'm having difficulty figuring out how to query the most recent entries in the database. All attempts to use the "order" or "where" SoQL statements give me data from 2010 or 2011, and I cannot figure out how to query the most recent 300 entries. Querying the "top" rows yields the oldest entries. Using a full OData feed pull yields data as recent as today, but I need to use a fast json or csv SODA API query.
Note: The datetime field does not respond to any "where" statements that I use.
Thank you!
OK, a few tips to get started:
The $order parameter sorts by default in ascending (ASC) order, so you'll want to actually order by datetime DESC to get the latest records first
Unfortunately Seattle has a number of crimes that are listed with no datetime, so you'll also want to filter with a $where query to only retrieve results in a date range. $where=datetime > '2014-07-01' works for me, for example
To only get the top 300 results, you'll want to pass a $limit=300 parameter as well.
Here's a sample request in Runscope for you to try out.
Related
I'm trying to pull data from a variety of Socrata datasets in to a analytic architecture using the REST json API. I would like to find a way to get the new data dropping into the data set since the last request.
My plan at the moment is to use $order and $where with one of the date fields, and then pull a filtered set since the last day every 24-hours.
Are there any examples of ways to use some data math, or is there a better way that I'm missing to get the newest data since the last query?
Your help is appreciated.
I've looked on a couple different forums and I'm unable to find what I need they all have it listed in SQL View, and that is NOT what I want. I would like to do this in the Query Design as it is much easier for me.
I simply would like to make a query to display certain parameters of the database I maintain. And those parameters would equal last month's data.
I previously was able to successfully make a query displaying all information for the past year but can't figure out how to display just this past months.
The past year Criteria:
>DateAdd("yyyy",-1,Date())
Since that worked I tried doing this but it would not work for me:
>DateAdd("mmmm",-1,Date())
I'm sure it's something simple that I'm just not seeing here. Any help or recommendations are welcome.
Referencing the link provided by Fionnuala I have come up with
>DateAdd("m',-30,Date())
The problem being is that it queries all results for the past 30 days from Today's date. I wish to display only data from October ! While this can be done easily. I don't wish to manually go in this query every month and change certain parameters. I would like it to be automatic so or next month December I click on the query and it displays all 30 days of November's data. And there is no manual process of going back in and changing any of the criteria.
In the Query Designer you can use a Criteria: like this
>=DateSerial(Year(Date()),Month(Date())-1,1) And <DateSerial(Year(Date()),Month(Date()),1)
The corresponding SQL statement is
SELECT Donations.*
FROM Donations
WHERE (((Donations.DonationDate)>=DateSerial(Year(Date()),Month(Date())-1,1)
And (Donations.DonationDate)<DateSerial(Year(Date()),Month(Date()),1)));
If it was run on November 18, 2014 then it would effectively be
SELECT Donations.*
FROM Donations
WHERE (((Donations.DonationDate)>=DateSerial(2014,10,1)
And (Donations.DonationDate)<DateSerial(2014,11,1)));
Notes:
This query should be sargable and take advantage of an index on the date field (if one exists).
In case anyone is concerned about "month wrap-around", the DateSerial() function takes care of that for us. If the query was run in January 2015 then the first WHERE condition would indeed be ... >=DateSerial(2015,0,1) but that's okay because it returns 2014-12-01.
I am trying to apply group active record command in rails rest api, however my database is in MySql.
When I query without group by I get correct data but when I use group on the same query I get strange data collection. I am using group to decrease query time coz in original it takes alot of time to retrieve data from database
Here is my original query
Records.owned_by(User.find_by_email(params[:user].to_s).id).where(device_id: params[:did]).includes(:record_students, :record_employees, :record_admins, :record_others)
but when I use group to increase the efficiency the returned data set is not valid
here is my new query with group
Records.owned_by(User.find_by_email(params[:user].to_s).id).where(device_id: params[:did]).includes(:record_students, :record_employees, :record_admins, :record_others).group("date(created_at)")
any idea what is wrong. Thanks
How do I format a query with multiple order by columns. The data I'm working with has a date column and a time column and I want to order by both of them. I know how to do this query in regular SQL but I can't make it work in SoQL. Here is what I've tried:
This works ('date DESC') but isn't what I'm trying to do:
http://data.sfgov.org/resource/tmnf-yvry.json?$order=date+DESC
This fails ('date DESC, time DESC') with a 403 error:
http://data.sfgov.org/resource/tmnf-yvry.json?$order=date+DESC%2Ctime+ASC
This fails ("'date DESC, time DESC'") with a 403 error:
http://data.sfgov.org/resource/tmnf-yvry.json?$order=%27date+DESC%2C+time+DESC%27
Currently, sorting on multiple columns at the same time is something you unfortunately can't do with the SODA API. It'll respond with a "query.execution.queryTooComplex" error like you're seeing.
However, this is something that'll be fixed in the future as we migrate datasets to our new backend. Details on this process and how to tell when/if a dataset has been migrated will be available soon.
Note: You also need to use the $order parameter in your query, not just order. I'll edit your URLs above to match.
I started building a search engine monitor. I'm pulling data from the google rest api into a mysql database with the following fields: date, search-keyword, domain, url, position.
Now I got into trouble querying and outputting the data for charting. The results go up and down, new results from google come into the list which haven't been there on the first day. However for charting I have to assign the first days at least blank values to output a chart.
What I do right now: First I select every domain showing up in the period. Lets say the for the keyword searchengine I get the domains wikipedia.org, ixquick.com, yahoo.com, searchenginewatch.com When I make another request for ever domain to query an array of rankings grouped by day. leading to the ...
Problem: Is where any query (mysql/nosql) which returns for each day an average and if where is no row a default value e.g. blank?
Result should look like:
dates={01/01/2014,02,03,04,05,06,07,08,...,31}
wikipedie={1,1,1,1,1,1,1,1,...,1}
yahoo = {"","",7,5,3,3,3,...,3}
You can create a date table, select the date range you'd like, and outer join your data to it, filling in 0s for values that do not exist for a given term/date.
Edit:
Some more details.
1) Create a table that has a row for every date +- 10 years (or whatever is appropriate). You can make this one column if you'd like, or many columns (date, month, year, etc.). The second approach makes this extensible if you want to summarize by various rollups in the future.
2) Outer join your table to the date table and use a NVL statement to coerce any null averages to 0.
3) Profit!
If your results are grouped by date, how can MySQL know there's (for example) 31 days in that month?
On the other hand, you can somehow fill the holes in PHP by loop through the array and fill a zero if the value does not exist.