We are trying to upload a 'bigger' model to a forge bucket via the model derivative API.
This model has a file size of 956MB and the extension .nwd
const uploadPromise = http(uploadUrl(socketId), {
method: 'PUT',
data: Body,
headers: {
'Content-Type': 'application/octet-stream',
Authorization: `Bearer ${Forge.urns[socketId].autoDeskToken}`,
},
maxContentLength: 1024 * 1024 * 1024,
timeout: 600000,
})
{ Error: Request failed with status code 408
at createError (/nodejs/node_modules/axios/lib/core/createError.js:16:15)
at settle (/nodejs/node_modules/axios/lib/core/settle.js:18:12)
at IncomingMessage.handleStreamEnd (/nodejs/node_modules/axios/lib/adapters/http.js:202:11)
at IncomingMessage.emit (events.js:203:15)
at IncomingMessage.EventEmitter.emit (domain.js:448:20)
at endReadableNT (_stream_readable.js:1145:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
The url seems to be correct:
https://developer.api.autodesk.com/oss/v2/buckets/-BUCKETNAME-/objects/-filename-.nwd
What could be the reason of this error?
This is a problem a customer is experiencing.
kind regards,
Thomas
That's interesting, the error code 408 is not listed in the documentation of the endpoint: https://forge.autodesk.com/en/docs/data/v2/reference/http/buckets-:bucketKey-objects-:objectName-PUT. The error code is most likely coming from some middle-layer (Apigee), and not from the Forge servers directly.
However, to upload a file this big, I'd strongly suggest using the "resumable upload" feature that's available in the Data Management service. You can find an example of how the resumable upload works in my TypeScript code here: https://github.com/petrbroz/vscode-forge-tools/blob/develop/src/commands/data-management.ts#L230-L272.
Related
I'm trying to get some JSON data from this API - https://ped.uspto.gov/api/queries
This cURL request works fine and returns what is expected:
curl -X POST "https://ped.uspto.gov/api/queries" -H "accept: application/json" -H "Content-Type: application/json" -d "{ \"searchText\":\"*:*\", \"fq\":[ \"totalPtoDays:[1 TO 99999]\", \"appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]\" ], \"fl\":\"*\", \"mm\":\"100%\", \"df\":\"patentTitle\", \"facet\":\"true\", \"sort\":\"applId asc\", \"start\":\"0\"}"
I have this python script to do the same thing:
from requests.structures import CaseInsensitiveDict
import json
url = "https://ped.uspto.gov/api/queries"
headers = CaseInsensitiveDict()
headers["accept"] = "application/json"
headers["Content-Type"] = "application/json"
data = json.dumps({
"searchText":"*:*",
"fq":[
"totalPtoDays:[1 TO 99999]",
"appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]"
],
"fl":"*",
"mm":"100%",
"df":"patentTitle",
"facet":"true",
"sort":"applId asc",
"start":"0"
})
resp = requests.post(url, headers=headers, data=data)
print(resp.status_code)
but it returns a 403 error code and the following response header:
"Date":"Mon, 24 Oct 2022 16:13:58 GMT",
"Content-Type":"text/html",
"Content-Length":"919",
"Connection":"keep-alive",
"X-Cache":"Error from cloudfront",
"Via":"1.1 d387fec28536c5aa92926c56363afe9a.cloudfront.net (CloudFront)",
"X-Amz-Cf-Pop":"LHR50-P8",
"X-Amz-Cf-Id":"RMd69prehvXNAl97mo0qyFtuBIiY8r9liIxcQEmbdoBV1zwXLhirXA=="
I'm at quite a loss at what to do, because I really don't understand what my Python is missing to replicate the cURL request.
Thanks very much.
I was interested in this. I got an account with uspto.gov and acquired an access key. Their other API's work well. But the PEDS API? I kept getting the Cloudflare Gateway Timeout 503 error. While I was on their website, I looked into the PEDS API, I could not load any link to a https://ped.uspto.gov page.
I called them and they gave me an email address. I got this reply:
The PEDS API was taken down, because repeated data mining was bringing the entire PEDS System down.
The PEDS Team is working on a solution to fix the PEDS API, so that it can be re-enabled.
I tried it using PHP.
Cloudflare has been causing a lot of problems for curl.
I got a timeout.
I may have gotten past the 403 Forbidden, but did not have credentials and so the server dropped the connection.
An HTTP 504 status code (Gateway Timeout) indicates that when
CloudFront forwarded a request to the origin (because the requested
object wasn't in the edge cache), one of the following happened: The
origin returned an HTTP 504 status code to CloudFront. The origin
didn't respond before the request expired.
AWS Cloudflare Curl Issues
bypassing CloudFlare 403
How to Fix Error 403 Forbidden on Cloudflare
403 Forbidden cloudflare
██████████████████████████████████████████████████████████████
This is a conversion from you curl.
The Content-Type:application/data is added by default when you send JSON data.
I do not know about your json_data.dump or you putting the JSON in parentheses.
import requests
headers = {
'accept': 'application/json',
}
json_data = {
'searchText': '*:*',
'fq': [
'totalPtoDays:[1 TO 99999]',
'appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]',
],
'fl': '*',
'mm': '100%',
'df': 'patentTitle',
'facet': 'true',
'sort': 'applId asc',
'start': '0',
}
response = requests.post('https://ped.uspto.gov/api/queries', headers=headers, json=json_data)
We are using c# revit addin in our project.
Design automation requests are getting failed with the error "status": "failedUpload",
Here is the report file for more details
[09/08/2022 17:30:21] Finished running. Process will return: Success
[09/08/2022 17:30:21] ====== Revit finished running: revitcoreconsole ======
[09/08/2022 17:30:25] End Revit Core Engine standard output dump.
[09/08/2022 17:30:25] End script phase.
[09/08/2022 17:30:25] Start upload phase.
[09/08/2022 17:30:25] Uploading 'T:\Aces\Jobs\ad78879202894efba8c5145367e8275d\result.rvt': verb - 'PUT', url - 'https://developer.api.autodesk.com/oss/v2/buckets/generatedmodels/objects/20220908164214_1305 - Interior purged.rvt'
[09/08/2022 17:31:27] Error: Retrying on GatewayTimeout. Request is 'PUT' 'https://developer.api.autodesk.com/oss/v2/buckets/generatedmodels/objects/20220908164214_1305 - Interior purged.rvt'
[09/08/2022 17:32:34] Error: Retrying on GatewayTimeout. Request is 'PUT' 'https://developer.api.autodesk.com/oss/v2/buckets/generatedmodels/objects/20220908164214_1305 - Interior purged.rvt'
[09/08/2022 17:33:51] Error: Retrying on GatewayTimeout. Request is 'PUT' 'https://developer.api.autodesk.com/oss/v2/buckets/generatedmodels/objects/20220908164214_1305 - Interior purged.rvt'
[09/08/2022 17:35:23] Error: Retrying on GatewayTimeout. Request is 'PUT' 'https://developer.api.autodesk.com/oss/v2/buckets/generatedmodels/objects/20220908164214_1305 - Interior purged.rvt'
[09/08/2022 17:35:23] Error: Failed - 504 (GATEWAY_TIMEOUT)
Request: PUT https://developer.api.autodesk.com/oss/v2/buckets/generatedmodels/objects/20220908164214_1305 - Interior purged.rvt
Request Headers:
Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IlU3c0dGRldUTzlBekNhSzBqZURRM2dQZXBURVdWN2VhIn0.eyJzY29wZSI6WyJidWNrZXQ6Y3JlYXRlIiwiYnVja2V0OnJlYWQiLCJidWNrZXQ6ZGVsZXRlIiwiZGF0YTpyZWFkIiwiZGF0YTp3cml0ZSIsImRhdGE6Y3JlYXRlIiwiY29kZTphbGwiXSwiY2xpZW50X2lkIjoidFBvVGQ0dENuTEhrajlZMEtRYWRyVFdBT0pLSUxzN1AiLCJhdWQiOiJodHRwczovL2F1dG9kZXNrLmNvbS9hdWQvYWp3dGV4cDYwIiwianRpIjoiU0pnc1N2TjdKTjhFSkhNeFRpdUhmMko2Y3ZTTXhZVW9UZXphcmRJcFNocHZjOWFDM1pzdVVPeDhESjNRc0toViIsImV4cCI6MTY2MjY1ODc0OH0.PJPp7LroxjWkFD7i3_ErLGyM_wS_D0ir1Qr-w9TfHazaEpmZSrwQ6QsRKcJ9ibXS5z9RY_5WGtzojDPyNTF4kP9TISpgJlyJivbLTnxv7oqW_acd0FvQYlvsaozrx_HIfRJIJLLuF_k1gGwpeArK9yQrKtWYSY1_5c3t1QQSEAs4i5HVyWlPPT8eEsQDtY_EYj32QQoeIMnfI3XWWQBkhD1LnbI9yIzLJ0D8ZWzXbbzD78wAhYudzLsW_0ay3YQRd6fTerUVLHaQ0UgyjvFfTVOV5mFZimERqtpyKynIEnF4JBKZGzhzxv-OlEVNe31o5CLr4oy1QBj_E53q5ZX4Ug
Request Content Headers:
Content-Length: 818163712
Response Headers:
Connection: keep-alive
Response Content Headers:
Content-Length: 0
Response Body:
[09/08/2022 17:35:23] Error: Upload failed. Reason = Response status code does not indicate success: 504 (GATEWAY_TIMEOUT).
[09/08/2022 17:35:23] Error: An unexpected error happened during phase Publishing of job.
[09/08/2022 17:35:24] Job finished with result FailedUpload
[09/08/2022 17:35:24] Job Status:
{
"status": "failedUpload",
"reportUrl": "https://dasprod-store.s3.amazonaws.com/workItem/tPoTd4tCnLHkj9Y0KQadrTWAOJKILs7P/ad78879202894efba8c5145367e8275d/report.txt?X-Amz-Expires=48600&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEJj%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQCgs4MB0vOFtfkJkyouuTSkaG5tzGqnIwaHN41D2ec5IgIgO8%2BddPJXEyQV0NP3R6iKxG7LGccF%2FJtCOZfmDc6oANsqgAMIMBADGgwyMjA0NzMxNTIzMTAiDMhdkXkWdvEoT%2BJc%2FSrdAk0REkrcSRlPtnlJTvPShrpoDhptZ%2FsoZAkNW%2BSWnSY%2FZom%2BNsa2qpmWopbOljmbOcBcmhh2K4JG9dN1AbErLjwd73f3cEId3gQoaE7O1CuBNhzl27K7MQ4tCva%2FxdkYNFV6r0Z%2FdJSylvSYrFyrrTS1jK5da2bL6tXBaJ1GByvRiOlGlDrXFyvNSnp%2FTJT1tJhX6wUGgpUaydqSHGfBsqXR%2Bwzskf8iKLZ7Z75oqCkZwmC9azQARwRT0PoSLTcR4RotaqDDdV8xYqCDUE0Us1ihVMYHKW%2FIrgMiDt5Vb3Cx5YJ68SNbNUAPG5JTmTyG%2BlWQiVXaV3IrH4SWktb494A5CJNJo6CkRmusDEPrvnEfiebWxAUwepcJl%2FU%2FJ8jj7Et1oROuReOS9obYtNHlodcEIOuaG62bsgw53awHzz095z9FV9ZNbStdmqb032uJmiSBWfaPr%2F8Zz7oSiykwpZDomAY6ngF5QHnzayjkxc04F41tGu31CD6DyAhmHq6XVZIUa3hwer76cIjHaZFSM1hWlrCwUUfGesfXXVI81KJrqaqyXkUkUshGBD2tEtvvyYb1CnYvnVssn28Rc0LNNl8WgFwPlfrPutbxT5yRsYTD1aAD2B80wHwBJ7vNLNUnonRMWUzY8QW%2BTO9F8pzb%2FGjLqQTlj3Ne4SM4Fpz8EXIng4gmNA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIATGVJZKM3BTQV7AHO/20220908/us-east-1/s3/aws4_request&X-Amz-Date=20220908T164217Z&X-Amz-SignedHeaders=host&X-Amz-Signature=f60e89e4fbd1527ba46c42480e64268483c15c5efdcb80ed2623144fbd222317",
"stats": {
"timeQueued": "2022-09-08T16:42:17.9517893Z",
"timeDownloadStarted": "2022-09-08T16:42:18.2481411Z",
"timeInstructionsStarted": "2022-09-08T16:43:20.1483203Z",
"timeInstructionsEnded": "2022-09-08T17:30:25.5436836Z",
"timeUploadEnded": "2022-09-08T17:35:23.6506512Z",
"bytesDownloaded": 1574150920
},
"id": "ad78879202894efba8c5145367e8275d"
}
Please help me to figure out the solution for this problem?
You need to finalize the upload, i.e, after design automation puts file to your s3 bucket you need to call buckets/:bucketKey/objects/:objectKey/signeds3upload
This can be done in OnCallback event, refer https://github.com/Autodesk-Forge/learn.forge.designautomation/blob/7d57b39014de5fccb0f082a9dd50a039bdcb2569/forgesample/Controllers/DesignAutomationController.cs#L456
Refer - https://forge.autodesk.com/en/docs/design-automation/v3/tutorials/revit/step6-prepare-cloud-storage/#step-4-complete-the-upload
I have used below code available on the Google Cloud Docs platform:
from google.auth.transport.requests import Request
from google.oauth2 import id_token
import requests
IAM_SCOPE = 'https://www.googleapis.com/auth/iam'
OAUTH_TOKEN_URI = 'https://www.googleapis.com/oauth2/v4/token'
def trigger_dag(data, context=None):
"""Makes a POST request to the Composer DAG Trigger API
When called via Google Cloud Functions (GCF),
data and context are Background function parameters.
For more info, refer to
https://cloud.google.com/functions/docs/writing/background#functions_background_parameters-python
To call this function from a Python script, omit the ``context`` argument
and pass in a non-null value for the ``data`` argument.
"""
# Fill in with your Composer info here
# Navigate to your webserver's login page and get this from the URL
# Or use the script found at
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/composer/rest/get_client_id.py
client_id = '87431184677-jitlhi9o0u9sin3uvdebqrvqokl538aj.apps.googleusercontent.com'
# This should be part of your webserver's URL:
# {tenant-project-id}.appspot.com
webserver_id = 'b368a47a354ddf2f6p-tp'
# The name of the DAG you wish to trigger
dag_name = 'composer_sample_trigger_response_dag'
webserver_url = (
#'https://'
webserver_id
+ '.appspot.com/admin/airflow/tree?dag_id='
+ dag_name
#+ '/dag_runs'
)
# Make a POST request to IAP which then Triggers the DAG
make_iap_request(
webserver_url, client_id, method='POST', json={"conf": data, "replace_microseconds": 'false'})
# This code is copied from
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/iap/make_iap_request.py
# START COPIED IAP CODE
def make_iap_request(url, client_id, method='GET', **kwargs):
"""Makes a request to an application protected by Identity-Aware Proxy.
Args:
url: The Identity-Aware Proxy-protected URL to fetch.
client_id: The client ID used by Identity-Aware Proxy.
method: The request method to use
('GET', 'OPTIONS', 'HEAD', 'POST', 'PUT', 'PATCH', 'DELETE')
**kwargs: Any of the parameters defined for the request function:
https://github.com/requests/requests/blob/master/requests/api.py
If no timeout is provided, it is set to 90 by default.
Returns:
The page body, or raises an exception if the page couldn't be retrieved.
"""
# Set the default timeout, if missing
if 'timeout' not in kwargs:
kwargs['timeout'] = 90
# Obtain an OpenID Connect (OIDC) token from metadata server or using service
# account.
google_open_id_connect_token = id_token.fetch_id_token(Request(), client_id)
# Fetch the Identity-Aware Proxy-protected URL, including an
# Authorization header containing "Bearer " followed by a
# Google-issued OpenID Connect token for the service account.
resp = requests.request(
method, url,
headers={'Authorization': 'Bearer {}'.format(
google_open_id_connect_token)}, **kwargs)
if resp.status_code == 403:
raise Exception('Service account does not have permission to '
'access the IAP-protected application.')
elif resp.status_code != 200:
raise Exception(
'Bad response from application: {!r} / {!r} / {!r}'.format(
resp.status_code, resp.headers, resp.text))
else:
return resp.text
# END COPIED IAP CODE
You encounter the error "405 method not found" because you are trying to send a request to your_webserver_id.appspot.com/admin/airflow/tree?dag_id=composer_sample_trigger_response_dag which is the URL seen "Tree view" in the airflow webserver.
To properly send request to the Airflow API you need to construct the webserver_url just like in the documentation in Trigger DAGs in Cloud Functions. The webserver_url was constructed to use Trigger a new DAG endpoint to send requests. So if you'd like to trigger the DAG you can stick with the code below.
Airflow run DAG endpoint:
POST https://airflow.apache.org/api/v1/dags/{dag_id}/dagRuns
webserver_url = (
'https://'
+ webserver_id
+ '.appspot.com/api/experimental/dags/'
+ dag_name
+ '/dag_runs'
)
Moving forward, if you would like to perform different operations using the Airflow API you can check the Airflow REST reference
I use this sample code:
axios({
method: 'post',
headers: { 'content-type': 'application/json' },
url: 'http://somePlace:8040/someWSendpoint',
data: {
firstName: 'Fred',
lastName: 'Flintstone'
}
});
But that request never reach its destination. Apparently, because the POST method is transformed to OPTIONS method, and rejected by the endpoint url.
This is what the 'Network' window shows in the Chrome inspector:
Request Method: OPTIONS
Status Code: 405 Method Not Allowed
This is what the 'Console' window shows in the Chrome inspector:
Access to XMLHttpRequest at 'http://somePlace:8040/someWSendpoint' from origin 'http://localhost:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
xhr.js?b50d:160 POST http://somePlace:8040/someWSendpoint net::ERR_FAILED
dispatchXhrRequest # xhr.js?b50d:160
xhrAdapter # xhr.js?b50d:11
dispatchRequest # dispatchRequest.js?5270:59
Promise.then (async)
request # Axios.js?0a06:51
wrap # bind.js?1d2b:9
send_firma # AltaFirma.vue?c240:504
click # AltaFirma.vue?3ed8:1812
invokeWithErrorHandling # vue.runtime.esm.js?2b0e:1854
invoker # vue.runtime.esm.js?2b0e:2179
original._wrapper # vue.runtime.esm.js?2b0e:6917
createError.js?2d83:16 Uncaught (in promise) Error: Network Error
at createError (createError.js?2d83:16)
at XMLHttpRequest.handleError (xhr.js?b50d:69)
I am wondering if people "at the other side" need to configure something related to that CORS thing in his server ¿?
Any help will be very appreciated.
https://gist.github.com/kenken64/b40ef906076018dc11aef1929b7e04a5
I am getting the following error after submit a job while checking status
200
application/json; charset=utf-8
b'{"type":"manifest","hasThumbnail":"false","status":"failed","progress":"complete","region":"US","urn":"dXJuOmFkc2sub2JqZWN0czpvcy5vYmplY3Q6Zm9yZ2Vfc2FtcGxlMl9jNmVheHJpZmxxeXp2bWlybXQzMDZoc21hd2VodjlpZy9yYWMxLnJ2dA","version":"1.0","derivatives":[{"name":"LMV Bubble","hasThumbnail":"false","status":"failed","progress":"complete","messages":[{"type":"error","message":"Translation failure","code":"TranslationWorker-InternalFailure"}],"outputType":"svf"}]}'
check complete translate data returned status code 200.
Please help !
https://developer.api.autodesk.com/modelderivative/v2/designdata/job
{"input": {"urn": "dXJuOmFkc2sub2JqZWN0czpvcy5vYmplY3Q6Zm9yZ2Vfc2FtcGxlM19jNmVheHJpZmxxeXp2bWlybXQzMDZoc21hd2VodjlpZy9yYWMxLnJ2dA"}, "output": {"formats": [{"type": "svf", "views": ["2d", "3d"]}]}}
{'Content-Type': 'application/json', 'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsImtpZCI6Imp3dF9zeW1tZXRyaWNfa2V5In0.eyJjbGllbnRfaWQiOiJDNmVheHJpRmxRWVp2bWlybXQzMDZIc01BV0VodjlJZyIsImV4cCI6MTUwMTUwMjc2NSwic2NvcGUiOlsiZGF0YTpyZWFkIiwiZGF0YTp3cml0ZSIsImRhdGE6Y3JlYXRlIiwiZGF0YTpzZWFyY2giLCJidWNrZXQ6Y3JlYXRlIiwiYnVja2V0OnJlYWQiLCJidWNrZXQ6dXBkYXRlIiwiYnVja2V0OmRlbGV0ZSJdLCJhdWQiOiJodHRwczovL2F1dG9kZXNrLmNvbS9hdWQvand0ZXhwNjAiLCJqdGkiOiJYMVh4R1l6UFZGVlpwZHlsR29MTmZKYjh4T2s1N0dEMDE0c2pNWWZhY1pzc1hDNmgwT0o2VTRIUWVhSEZHWGt4In0.janHAXhsbRtNQYZ9q-Pz7IsGZjF0Em_e_UoOurPr-4Q'}
201
application/json; charset=utf-8
b'{"result":"created","urn":"dXJuOmFkc2sub2JqZWN0czpvcy5vYmplY3Q6Zm9yZ2Vfc2FtcGxlM19jNmVheHJpZmxxeXp2bWlybXQzMDZoc21hd2VodjlpZy9yYWMxLnJ2dA","acceptedJobs":{"output":{"formats":[{"type":"svf","views":["2d","3d"]}]}},"registerKeys":["2d8ddceb-9f0a-48e5-ae6e-53839cc6ded6"]}'
translate data returned status code 201.
It seems that you have some wrong configurations while calling the Model Derivative Job API. Please change following line and try it again:
compressedUrn: true to compressedUrn: false, since your model is a RVT file, it's not compressed.
rootFilename: "A5.iam" to rootFilename: "rac1.rvt".
P.S. Please call this API https://developer.api.autodesk.com/modelderivative/v2/designdata/dXJuOmFkc2sub2JqZWN0czpvcy5vYmplY3Q6Zm9yZ2Vfc2FtcGxlMl9jNmVheHJpZmxxeXp2bWlybXQzMDZoc21hd2VodjlpZy9yYWMxLnJ2dA/manifest to delete the failure job before you resent the translation job.