we have several google oauth2 refresh tokens for Google Drive for which we always get the following error when trying to request a new access token:
POST /o/oauth2/token HTTP/1.1
Connection: close
accept-encoding: gzip, deflate
content-type: application/x-www-form-urlencoded
Content-Length: 208
Host: accounts.google.com
refresh_token=1%2FY5_2XY8uGujYa222rxXnsjR<snipped>&client_id=<clientid>&grant_type=refresh_token&client_secret=<clientsecret>
Response:
HTTP/1.1 400 Error processing OAuth 2 request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Tue, 20 Aug 2013 14:55:24 GMT
<HTML>
<HEAD>
<TITLE>Error processing OAuth 2 request</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Error processing OAuth 2 request</H1>
<H2>Error 400</H2>
</BODY>
</HTML>
This only happens for some accounts, others work fine. The broken accounts fail reproducibly across days and weeks. Is there something wrong with the data we send? Any hint on what fails the validation?
I could provide you the tokens that are failing if necessary.
As far as I can tell all these refresh tokens fail because of invalid user accounts (deleted, disabled, etc). In these cases for all intents and purposes the token itself is invalid.
The HTML error response is a bug, and it should be fixed by now. Please report if you still see errors like this. The proper OAuth 2 error code in this case is "invalid_token".
Related
I'm trying to scrape products from a site (e.g. https://www.violetgrey.com/en-us/shopping/the-rich-cream-18105401). Whilst on browser it loads normally, when I copy the initial curl request for the site, it gives me access denied. This is all done in local environment. So far, before copying the curl request from browser dev tools I have:
Disabled JS for the site
Cleared all my cache, cookies
Tried different browsers
Still, it's the same result - blocked via curl. When the exact same request worked in my browser. Could anyone please point me to the right direction?
If you look at the response header you can see it comes from Cloudflare.
Cloudflare is evil. IMHO.
The HTTP status is 403. HTTP/2 403 Which means Forbidden.
The body is the text:
error code: 1020
Error 1020 can be roughly translated to "take your curl and go elsewhere. You and your curl are not wanted here."
Cloudflare profiles and fingerprints Browsers. For example they monitor the SSL/TLS handshaking and if your curl handshaking is not do the handshaking exactly like the Browser in your User Agent, they give you a 403 Forbidden and Error code 1020.
And your request does not reach violetgrey.com. They do not even know you tried.
Cloudflare is political and blocks whatever traffic they want to. If it is in their best interest not allow you through, they block you. For example Cloudflare blocked me from accessing the US Patent and Trademark site. Not only that but they sent out 3 XHR beacon requests to YouTube and Google Play. My Firefox blocked those requests. Cloudflare and Google are closely related. I do not trust either one of them.
There is no shortage of articles about your problem and possible fixes. Just search "Cloudflare 403 forbidden 1020 error". And maybe not use Google to do the search.
Here is my effort to scrape your URL. I tried a few things like trying various User Agents. I tried wget.
Request header
GET /en-us/shopping/the-rich-cream-18105401 HTTP/2
Host: www.violetgrey.com
mozilla/5.0 (x11; netbsd amd64; rv:16.0) Gecko/20121102 Firefox/16.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
accept-language: en-US,en;q=0.5
accept-encoding: gzip, deflate, br
dnt: 1
alt-used: www.violetgrey.com
connection: keep-alive
upgrade-insecure-requests: 1
sec-fetch-dest: document
sec-fetch-mode: navigate
sec-fetch-site: cross-site
sec-fetch-user: ?1
te: trailers
Response header:
HTTP/2 403
date: Thu, 27 Oct 2022 23:56:19 GMT
content-type: text/plain; charset=UTF-8
content-length: 16
x-frame-options: SAMEORIGIN
referrer-policy: same-origin
cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
expires: Thu, 01 Jan 1970 00:00:01 GMT
server-timing: cf-q-config;dur=4.9999998736894e-06
vary: Accept-Encoding
server: cloudflare
cf-ray: 760f5e1ced6e8dcc-MIA
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
Response Body:
error code: 1020
I have been able to run several jobs on Forge, but I am running into a new situation today. It seems like the workitem stays idle for about 70 minutes even when it reports back "In Progress". Finally when the workitem starts, it fails immediately on downloading resources. I am assuming because the signed URL doesn't last that long:
[06/10/2022 01:07:09] Starting work item accb10182b02420f925ccd4d113dffbd
[06/10/2022 01:07:09] Start download phase.
[06/10/2022 01:07:09] Start downloading input: verb - 'GET', url - 'https://developer.api.autodesk.com/oss/v2/signedresources/b70c5565-0ff5-48cb-af7b-b3f57113786e?region=US'
[06/10/2022 01:07:10] Embedded resource [{"ViewName": "VR_EXPORT"}] is saved as file: T:\Aces\Jobs\accb10182b02420f925ccd4d113dffbd\params.json.
[06/10/2022 01:07:10] Error: Failed - 404 (Not Found)
Request: GET https://developer.api.autodesk.com/oss/v2/signedresources/b70c5565-0ff5-48cb-af7b-b3f57113786e?region=US
Response Headers:
Access-Control-Allow-Headers: Authorization, Accept-Encoding, Range, Content-Type
Access-Control-Allow-Methods: GET
Access-Control-Allow-Origin: *
Date: Fri, 10 Jun 2022 01:07:10 GMT
Strict-Transport-Security: max-age=31536000; includeSubDomains
x-ads-region: US
X-Content-Type-Options: nosniff
Connection: keep-alive
Response Content Headers:
Content-Type: application/json; charset=utf-8
Content-Length: 38
Response Body:
{"reason":"Signed Resource not found"}
Not sure if this is an internal issue with Design Automation or I am doing something wrong.
I'm experiencing an issue with Chrome that I can't seem to fully understand, I'm curious if folks here have dealt with it before. This doesn't reproduce in Firefox. The steps are as follows:
Start incognito Chrome, navigate to https://foo.mysite.com and have the JS on the page make a GET ajax request to S3 for https://s3.amazonaws.com/mystuff/file.json . You get back a 200 response with:
HTTP/1.1 200 OK
x-amz-id-2: somestuffhere
x-amz-request-id: somestuffhere
Date: Tue, 14 Oct 2014 03:06:41 GMT
Access-Control-Allow-Origin: https://foo.mysite.com
Access-Control-Allow-Methods: GET
Access-Control-Max-Age: 3000
Access-Control-Allow-Credentials: true
Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
Cache-Control: max-age=86400
Content-Encoding: gzip
Last-Modified: Sun, 05 Oct 2014 00:29:53 GMT
ETag: "fe76607baa40a793eb3b3cbd373a3fb8"
Accept-Ranges: bytes
Content-Type: application/json
Content-Length: 5609
Server: AmazonS3
Open a second tab, navigate to https://bar.mysite.com and have its JS make a GET ajax request to S3 for the same file https://s3.amazonaws.com/mystuff/file.json . Get back the following 304 response:
HTTP/1.1 304 Not Modified
x-amz-id-2: somestuffhere
x-amz-request-id: somestuffhere
Date: Tue, 14 Oct 2014 03:06:58 GMT
Access-Control-Allow-Origin: https://bar.mysite.com
Access-Control-Allow-Methods: GET
Access-Control-Max-Age: 3000
Access-Control-Allow-Credentials: true
Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
Cache-Control: max-age=86400
Last-Modified: Sun, 05 Oct 2014 00:29:53 GMT
ETag: "fe76607baa40a793eb3b3cbd373a3fb8"
Server: AmazonS3
Open a third tab, navigate to https://foo.mysite.com (the first site) and repeat the same steps as in 1. Chrome kills the response for CORS reasons and reports the following:
XMLHttpRequest cannot load https://s3.amazonaws.com/mystuff/file.json. The 'Access-Control-Allow-Origin' header has a value 'https://bar.mysite.com' that is not equal to the supplied origin. Origin 'https://foo.mysite.com' is therefore not allowed access.
What's the story here? This doesn't reproduce in Firefox. In Firefox I'm happily getting a 304 in both steps 2 and 3, which I would expect to see in Chrome as well.
A temporary workaround for this issue in Chrome is to set Cache-Control: no-cache on the file in S3, but then I'm forcing our clients to be re-downloading that file for no good reason, so it's not a real solution.
Is this intended and documented behavior? Is this a bug with Chrome? Any other thoughts?
Looks like this is caused by Chromium issue 260239
I found this blog that help: Add Vary headers to S3
It helped by adding Vary headers to all XHR request.
I did run into a problem with html request (i.e. ) but I was able to overcome that by using hackround#2 described here:https://serverfault.com/a/856948
TL;DR of hack#2 is to use a "dummy" query string parameter that differs for HTML and XHR or is absent from one or the other. Example:
<img src="https://s3.png?x-request=html">
I just add a timestamp in request URL to force load the asset from S3 again, not from cache, such as xxxx?timestamp=yyyy
Having had repeatedly no joy with accessing groups settings via UrlFetch in GoogleAppsScript I looked to test the basic query I was using according to the docs in the APIs Explorer
UrlFetchApp.fetch('https://www.googleapis.com/groups/v1/groups/exampleGroupId#example.com?key={YOUR_API_KEY}', fetchArgs);
Never completes a round tring
The fact that the APIs explorer also fails with similar errors at least gives me some comfort, but where to report the failing?
GET https://www.googleapis.com/groups/v1/groups/exampleGroupId#example.com?key={YOUR_API_KEY}
Authorization: Bearer ya29....[snip]
X-JavaScript-User-Agent: Google APIs Explorer
gives
200 OK
cache-control: private, max-age=0, must-revalidate, no-transform
content-encoding: gzip
content-length: 731
content-type: application/atom+xml; charset=UTF-8
date: Tue, 12 Nov 2013 11:31:09 GMT
etag: "N…[snip]…I"
expires: Tue, 12 Nov 2013 11:31:09 GMT
server: GSE
[application/atom+xml; charset=UTF-8 data]
adding any field to the query
GET https://www.googleapis.com/groups/v1/groups/exampleGroupId#example.com?fields=archiveOnly&key={YOUR_API_KEY}
Authorization: Bearer ya29....[snip]
X-JavaScript-User-Agent: Google APIs Explorer
always results in error
400 Bad Request
cache-control: private, max-age=0
content-encoding: gzip
content-length: 209
content-type: application/vnd.google.gdata.error+xml; charset=UTF-8
date: Tue, 12 Nov 2013 11:33:34 GMT
expires: Tue, 12 Nov 2013 11:33:34 GMT
server: GSE
[application/vnd.google.gdata.error+xml; charset=UTF-8 data]
<?xml version="1.0" encoding="UTF-8"?>
<errors xmlns="http://schemas.google.com/g/2005">
<error>
<domain>GData</domain>
<code>invalidParameter</code>
<location type="parameter">fields</location>
<internalReason>Invalid field selection archiveOnly</internalReason>
</error>
</errors>
Is the API borked?
Try using the Google OAuth 2.0 Playground instead. I think the Groups Settings API Explorer is broken.
I am also having trouble with implementing with in Apps Script. I am stuck in an authorization loop. Similar to this reported bug. https://code.google.com/p/google-apps-script-issues/issues/detail?id=3046
I'm trying to do HEAD requests to follow 302 links, however this link: http://news.google.com/news/url?sa=t&fd=R&usg=AFQjCNGrJk-F7Dmshmtze2yhifxRsv8sRg&url=http://www.mtv.com/news/articles/1647243/20100907/story.jhtml
is troublesome because a HEAD request returns a 200 OK and a GET request returns the expected 302 Status code.
So I'll need to do a GET request but I'd rather not have to pay for the extra bandwidth times that will come from getting the entire HTML document. Anyone know a hack to do a GET without getting the body returned?
UPDATE: took David's advice to do a Range header but they seem to still be ignoring it
GET /news/url?sa=t&fd=R&usg=AFQjCNGrJk-F7Dmshmtze2yhifxRsv8sRg&url=http://www.mtv.com/news/articles/1647243/20100907/story.jhtml HTTP/1.1
Range: bytes=0-10
x-ms-range: 0-600
Host: news.google.com
Connection: Keep-Alive
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=UTF-8
Location: http://www.mtv.com/news/articles/1647243/20100907/story.jhtml
Content-Length: 258
Date: Wed, 08 Sep 2010 20:28:16 GMT
Expires: Wed, 08 Sep 2010 20:28:16 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Set-Cookie: PREF=ID=ef5f1bc768645c5e:TM=1283977696:LM=1283977696:S=5n26IrEDpcQTJIb1; expires=Fri, 07-Sep-2012 20:28:16 GMT; path=/; domain=.google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
here.
</BODY></HTML>
File a bug with the web server's owner.
Try using the Range header in your request.
If that doesn't work, can you just hang up the connection after you get the headers you want?
In the specific example you cite, you could just pull it out of the original URL's "url" parameter. But for a more generic approach, I'd stick to David M.'s suggestions