I have a data-set of 16 Millions for which I will send the address etc to Google Geocoding API and get the lat-longs. Right now we are making an API request for POI by POI, I'm wondering if we can do bulk /batch request, where in which I send around 500 POIs or so, so that we can save some round-trip time. Please advise.
You can use the Web-Service Geocoding API and call the request via Curl, you'll just need to create an interval for each batch request to reaching the request/100s limit. If the maximum request/100s limit is not enough for your use case, you may file a support case via https://console.cloud.google.com/google/maps-apis/support so the support team can help you give your account more capability to send requests more than the limit.
Related
Google references this:
https://developers.google.com/maps/documentation/javascript/geocoding
And states:
The per-session rate limit prevents the use of client-side services for batch requests, such as batch geocoding. For batch requests, use the Geocoding API web service.
However, when you go to the Geocoding API web services page, I see no reference to batch processing. The above sentence infers that you can do batch processing. I need to send a large number of addresses to get lat and longitude, but doing individual calls for each address is taking extremely long periods of time and need a more efficient method. Hopefully, a single batch call to send all the addresses.
Any ideas of how to batch process addresses on google to get lat and longitude?
I have seen this Google Batch Geocoding API
However, it states you can not which is not what the above google statement infers.
Per the Geocoding API web service documentation:
Other Usage Limits
While you are no longer limited to a maximum number of requests per day (QPD), the following usage limits are still in place for the Geocoding API:
50 requests per second (QPS), calculated as the sum of client-side and server-side queries.
You are just limited to 50 requests per second and have to pay for them (after you use up the $200 credit)
The best way I found to solve this problem is to use the Directions API with up to 27 destinations (origin, destination and 25 waypoints) and get your geolocation for the response legs. The position accuracy is slightly lower than in the geocode case from what I observed, but it is still a great tradeoff.
In the worst case you will have to call the Directions API twice when one or more addresses are not found in your call. The good thing in this case is that the Directions API will give you a response with the geocoded_waypoints which will specify the NOT_FOUND locations with a geocoder_status. After that, you can eliminate the bad ones and call again.
There are currently no available feature for a Geocoding API to handle multiple address at a single call, however, you may implement the batch process via cURL, by doing this, you can call multiple requests at once automatically. Implementation will be up to your use case as well.
I have enabled pay as you go pricing on my Google account and I am sending an API key with all my Google Maps Geocoding requests however I am still receiving this error message
You have exceeded your daily request quota for this API. We recommend registering for a key at the Google Developers Console
I have checked the quota page on the API dashboard and I am well below the limit of 100,000 per day. My key is only used for this geocoding script and this is not the same as the rate limit error which I sometimes receive.
Is there another limit I am unaware of? Can the quota page of the console be relied upon?
Unless you are doing 50 requests per second, calculated as the sum of client-side and server-side, you shouldn't have this problem.
" My key is only used for this geocoding script and this is not the same as the rate limit error which I sometimes receive."
what do you mean with this? are you using more than one API key?
which APIs are you using? only Geocoding?
I am having latitudinal and longitudinal position of around 500,000 bridges and I want the elevation of those bridges using the latitude and longitude coordinates of this bridges. I wrote a script in c# to parse elevation for all the 500000 bridges using the elevation API from the google. I generated google key with my account. Also, I enabled the billing for this account , but after few 100 request, I get an error
We're sorry...
... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.
See Google Help for more information.
I read the google api limit usage and it says that I can make upto 100000 request a day with billing enabled.
How to parse the elevation for 100000 bridges in a day?
You have hit a TOS issue..10 e to be exact:
No mass downloading. You will not use the Service in a manner that gives you or a third party access to mass downloads or bulk feeds of any Content. For example, you are not permitted to offer a batch geocoding service that uses Content contained in the Maps API(s)
https://developers.google.com/maps/terms?hl=en
We have a page that accepts a city, state combination or a zip code. Upon submit, it passes that information to the Geocode Webservice for Google Maps and returns a list of store location results based on the information passed. We sporadically have issues with bots hitting that page multiple times and then Google shuts down our usage of the geocode webservice for the day. Is there a way to ask Google to restore this more quickly? How should we handle this?
Use the client-side geocoder, the limits there will be based on the client not the server.
Geocoding Strategies
As specified on in the documentation, "Use of the Google Geocoding API is subject to a query limit of 2,500 requests per day (User of Google Maps API for Business may perform up to 100,000 requests per day.)"
There's a nice article on Geocoding Strategies which discusses caching and other options to optimize geocoding. The "Quota Considerations" section should be useful.
Remember to use client-side geocoding if at all possible, so that the quota limit will be counted per client, instead of per service/server. If you're still hitting the limits, you might need to go for the Business version and pay for a larger limit.
The following are the usage limits as specified by Google on their Developer Guide pages:
Google Maps JavaScript API v3 => For-profit web sites are permitted to generate up to 25,000 map loads per day
Google Geocoding API => subject to a query limit of 2,500 geolocation requests per day
Google Maps API for Business => may perform up to 100,000 requests per day
Am trying to evaluate using any one of the above for use with Visualforce on Salesforce.com (SFDC) platform [*]
I understand for a public website the requests are per IP. Now for SFDC, there could be many different Organizations on a particular server (say, NA1). So, two different companies using SFDC and Google Maps API could have an URL at https://na1.salesforce.com/something_here and their requests should be counted separately.
Will it be so? What will happen in case of each API?
[*]SFDC is a SaaS cloud for the purpose of our discussion. All users login through the same page but they could be logged into different "orgs"/"organizations" but their URLs might look similar
It's important to differentiate between the server-side and client-side limits here. The server-side geocoding api would have have the 2500 limit enforced across the shared Salesforce instance based on how many machines the requests come from (I assume NA1 isn't 1 huge server). Multiple organization using the free geocoding API would all share the same server-side geocoding limit. I've actually run into the same limits using Google's own App Engine platform, where a bunch of applications share the same outbound IP address.
For any sort of guaranteed performance you'll need to send the queries from your own server or go the Maps for Business route which lets you authenticate your queries to get those higher limits.
Client-side geocoding via the JavaScript API doesn't have these server-limits, so if users do any sort of action to trigger a geocode or two using the JS API is the best route.
You can already create your own "bucket" to track your 25K map loads per day by signing up for an API Key.
This question on SO addresses the geocoding API specifically being run from a visualforce page directly, Salesforce: Google maps query status 620 G_GEO_TOO_MANY_QUERIES and it does seem to mean that without a key the limits are shared. I would suspect that unless you plan on giving the app away that you are working on, you will pretty much be forced to pick up an upgraded API key. One thing you may want to look at to work around this is hosting the maps portion in another location, and iframing it into Salesforce.