I am working with the NYC crime dataset and I notice that the offense type for homicides is MURDER & NON-NEGL. MANSLAUGHTE. (Yes, that’s “manslaughte” without the “r”.) However, when I try to filter for this using the SODA API, I get errors:
https://data.cityofnewyork.us/resource/hyij-8hr7.json?$where=offense='MURDER & NON-NEGL. MANSLAUGHTE'
{
"error" : true,
"message" : "Unrecognized arguments [ NON-NEGL. MANSLAUGHTE']"
}
It seems to me that the ampersand in the string is escaping.
Note that I tried to use the like=‘MURDER’ command to avoid needing to type the ampersand, so for instance querying https://data.cityofnewyork.us/resource/hyij-8hr7.json?$where=offense like 'MURDER', but this gives me the error
{
"error" : true,
"message" : "Error: function #LIKE is not defined in SoQL.: Error: function #LIKE is not defined in SoQL.”
}
This must mean that the NYC dataset is using SoQL. v. 2.0 and not 2.1.
So in conclusion, how can I query for homicides? Is there a way to either (a) properly escape the ampersand, or (b) search for substrings?
Thanks for any assistance on either of these requests.
You're correct, hyij-8hr7 is the SODA 2.0 endpoint for that dataset. However, a 2.1 endpoint also exists: https://dev.socrata.com/foundry/data.cityofnewyork.us/e4qk-cpnv
Using the 2.1 endpoint, you can escape the ampersand as a URL-encoded %26: $where=offense='MURDER %26 NON-NEGL. MANSLAUGHTE'. Full URL, fully-escaped:
GET https://data.cityofnewyork.us/resource/e4qk-cpnv.json?$where=offense%3D%27MURDER%20%26%20NON-NEGL.%20MANSLAUGHTE%27
With the 2.1 endpoint, you can also use LIKE to do SQL-style substring searches or $q to use the full-text search index.
Related
I'm trying to fetch results via API using the module askargs. I have no problems getting results when I have just one condition or more conditions aggregated with the operator AND where I make use of the pipe character to separate them (like written in documentation).
E.g.
[[Category:+]] AND [[Jurisdiction::A]] AND [[Type::B]]
Category:+ | Jurisdiction::A | Type::B
But the pipe character doesn't work with OR.
I need to be able to use both logical conditions with several arguments within the same query.
Am I missing something?
Am I missing something?
No. The API doesn't handle OR condition, due to simplistic code in the query parameters formatter.
See file SemanticMediaWiki/src/MediaWiki/Api/ApiRequestParameterFormatter.php
at line 132:
protected function formatConditions( $condition ) {
return "[[$condition]]";
}
Every condition in the query is formatted with surrounding brackets, leading OR to be interpreted as a page title.
An alternative is to use Special:Ask with URL encoded query and json format:
https://www.semantic-mediawiki.org/wiki/Special:Ask/-5B-5BHas-20keyword::askargs-5D-5DOR-5B-5BHas-20keyword::ask-5D-5D/-3F%3Dhelp-20page/-3FHas-20description%3Ddescription/format%3Djson
Since I came here from a website search i'm going to add another neat possibility:
If you use the Alternative separator you can use a double pipe as logical OR conjunction.
Example:
%1FCategory:+%1FJurisdiction::A%1FType::B||C
Which should be read as following
Category:+ AND Jurisdiction::A AND (Type::B OR Type::C)
For a while I have been trying to access data from The Danish Company register, which is stores in elastic searchs rest API. I have tried using httr and The elastic package for R with no luck.
I think i have succeded connecting using
elastic::connect("distribution.virk.dk/CVR-permanent/", es_port=" ", es_user="username", es_pwd="pass")
I can use the count function to see the index of companies, participant and production units
Count(index="virksomhed")
[1] 1617466
The issue comes when I use Search(). Below is my input
Search(index=virksomhed, q="422900")
where 422900 is nacecode. I have tried various type of test with no luck though. Also tried using body copying search JSON search strings.
My result is always the same.
Error in if (gsub("\.","",ping()$version$number) <500) {: argument is of length zero
Is there Any help out there? Httr jsonlite, anything??
code here
using Socrata to access Data,
the issue am having is, when I DO NOT use my app_token key (String1 See Below), it works fine, with current data information, but when I do use my app_token with the String (String2), i get the follow error. (See below), And if I use (String1) with just my app_token without no extra data fields like draw_data (draw_date=2016-06-24T00:00:00.000) it works. So i know its not my key, any reason why? How do i get this to work correctly?
String1 (WORKS): https://data.ny.gov/resource/h6w8-42p9.json?draw_date=2016-06-24T00:00:00.000
String2 (DON'T WORK): https://data.ny.gov/resource/h6w8-42p9.json?$$app_token=MY-TOKEN?draw_date=2016-06-24T00:00:00.000
Getting Error (With String2):
SODA code": "permission_denied", "error": true, "message": "Invalid app_token specified"
First, I'm pretty confident MY-TOKEN isn't your app token, but just in case, make sure you've signed up for a real app token.
Second:
https://data.ny.gov/resource/h6w8-42p9.json?$$app_token=MY-TOKEN?draw_date=2016-06-24T00:00:00.000
...should instead be:
https://data.ny.gov/resource/h6w8-42p9.json?$$app_token=MY-TOKEN&draw_date=2016-06-24T00:00:00.000.
There should be an ampersand (&) between your $$app_token and draw_date parameters. The question mark (?) is only used to separate the URL from the parameter set. To our query parser, it looks like your app token is MY-TOKEN?draw_date=2016-06-24T00:00:00.000.
Using the HATEOAS links functionality which is great I am trying to output a templated url to highlight the filter params available to a user
Example controller method
#RequestMapping(value = "/persons", method = RequestMethod.GET, produces = "application/hal+json")
public PersonsResource getPersons (#RequestParam(required = false, value = "name") String name, #RequestParam(required = false, value = "age") Integer age) {
...
personsResource.add(ControllerLinkBuilder.linkTo(ControllerLinkBuilder.methodOn(PersonController.class).getPersons(name, age)).withSelfRel());
}
When this method is invoked with no parameters links appears
_links: {
self: {
href: "http://myserver:8080/persons"
}
}
But I'd like
href: "http://myserver:8080/persons?name={name}&age={age}
Even better if one param was supplied then
href: "http://myserver:8080/persons?name={name}&age=21
Icing on the cake would be query parameters of {...] to be ignored ?
Does anyone know if this is possible using the Spring HATEOAS api ? I have managed to code around this but it seems like a reasonable suggestion for the API ?
You could try AffordanceBuilder from spring-hateoas-ext as a drop-in replacement for ControllerLinkBuilder. It creates template variables for parameters you leave undefined in the linkTo-methodOn idiom.
It not only allows to create templates, but also gives you the full capabilities of a RFC 5988 Link and has knowledge about request bodies, so that one can render Hydra or Html or Siren Responses with form-style request descriptors from it.
Disclaimer: I'm the author of spring-hateoas-ext.
This has been addressed in the latest spring-hateoas version. You can check the following issue:
https://github.com/spring-projects/spring-hateoas/issues/169
You should be able to get the required templated URL using something like:
resource.add(linkTo(methodOn(Controller.class).method(null)).withSelfRel());
I guess, the framework is still pretty immature.
I have v.0.11.0.RELEASE and have the same issue.
When you don't supply parameter values you don't have template URL as a result of the ControllerLinkBuilder.linkTo(methodOn) invocation. It's just the way you said, base path from the method annotation.
But when you supply parameter values it's exactly like you say:
https://stackoverflow.com/some/service/path?name=SomeName&age=11
(in my case parameters are different, but the effect is the one you see here)
The 'conceptually correct' URL should be
https://stackoverflow.com/some/service/path{?name,age}
But Spring HATEOAS doesn't support this. Unless you want to append it yourself in the code. Which is really undesirable.
I checked the UriBuilder from JavaEE, it works the same way, no templating for query parameters supported.
I've recently added the Sphider crawler to my site in order to add search functionality. But the default search.php that comes with the distribution of Sphider that I downloaded is too plain and doesn't integrate well with the rest of my site. I have a little navigation bar at the top of the site which has a search box in it, and I'd like to be able to access Sphider's search results through that search field using Ajax. To do this, I figure I need to get Sphider to return its results in JSON format.
The way I did that is I used a "theme" that outputs JSON (Sphider supposts "theming" its output). I found that theme on this thread on Sphider's site. It seems to work, but more strict JSON parsers will not parse it. Here's some example JSON output:
{"result_report":"Displaying results 1 - 1 of 1 match (0 seconds) ", "results":[ { "idented":"false", "num":"1", "weight":"[100.00%]", "link":"http://www.avtainsys.com/articles/Triple_Contraints", "title":"Triple Contraints", "description":" on 01/06/12 Project triple constraints are time, cost, and quality. These are the three constraints that control the performance of the project. Think about this triple-constraint as a three-leg tripod. If one of the legs is elongated or", "link2":"http://www.avtainsys.com/articles/Triple_Contraints", "size":"3.3kb" }, { "num":"-1" } ], "other_pages":[ { "title":"1", "link":"search.php?query=constraints&start=1&search=1&results=10&type=and&domain=", "active":"true" }, ] }
The issue is that there is a trailing comma near the end. According to this, "trailing commas are not allowed" when using PHP's json_decode() function. This JSON also failed to parse using this online formatter. But when I took the comma out, it worked and I got this better-formatted JSON:
{
"result_report":"Displaying results 1 - 1 of 1 match (0 seconds) ",
"results":[
{
"idented":"false",
"num":"1",
"weight":"[100.00%]",
"link":"http://www.avtainsys.com/articles/Triple_Contraints",
"title":"Triple Contraints",
"description":" on 01/06/12 Project triple constraints are time, cost, and quality. These are the three constraints that control the performance of the project. Think about this triple-constraint as a three-leg tripod. If one of the legs is elongated or",
"link2":"http://www.avtainsys.com/articles/Triple_Contraints",
"size":"3.3kb"
},
{
"num":"-1"
}
],
"other_pages":[
{
"title":"1",
"link":"search.php?query=constraints&start=1&search=1&results=10&type=and&domain=",
"active":"true"
}
]
}
Now, how would I do this programmatically? And (perhaps more importantly), is there a more elegant way of accomplishing this? And you should know that PHP is the only language I can run on my shared hosting account, so a Java solution for example would not work for me.
In search_result.html, you can surround the , at the end of the foreach loop with condition to only print if the index is strictly less than the number of pages - 1.