I am writing a Dynamo script to extract the properties out of all of the models in my modelset in BIM 360. Some of the files (notably the ones with more objects) will give me a pre-signed propertiesUrl value that looks like this:
https://bim360-nucleus-production-us-east-1-temporary-index.s3.amazonaws.com/36174d28-fbd6-4146-a3af-3f3a7bb8ec52/index/v2/indexes/5JAa5g6Nk-t5KQsgFe5OJA/properties.ldjson.gz?X-Amz-Expires=60&X-Amz-Security-Token={{token_value}}-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential={{Credential_value}}/20220509/us-east-1/s3/aws4_request&X-Amz-Date=20220509T223945Z&X-Amz-SignedHeaders=host&X-Amz-Signature={{Amz_Signature_Value}}
This pre-signed URL has been troublesome because it will expire within 60 seconds of being issued and Dynamo is having trouble making the requests on time, or because I will get a network transport error when using the DynaWeb package (only 1/3 requests are made successfully):
Dynaweb Network Transport Error
When I try using Dynamo's OOTB node "Web Request", it will give me values that contain characters that aren't even close to what the response should be.
Dynamo Web Request OOTB Response
Is there a way to circumvent the need for a pre-signed url for the "propertiesUrl" endpoint? I can't find anything in the API in how to download the properties from the indices besides what is detailed here (which shows how to get properties with a Bearer token, but not how to opt out of a pre-signed url format):
https://forge.autodesk.com/en/docs/acc/v1/reference/http/index-v2-index-properties-get/
curl -v 'https://developer.api.autodesk.com/construction/index/v2/projects/cd743656-f130-48bd-96e6-948175313637/indexes/da39a3ee5e6b4b0d/properties' -H 'Authorization: Bearer <token>'
Related
I am trying to make an API call to Microsoft's form recognizer to analyze a form against a custom model and I can't figure out how to do it.
Here is the documentation on the API
https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api/operations/AnalyzeWithCustomModel
The request body is blank and I don't really know how that ought to be formatted in order to be sent off.
If you look at the POST Train model method, I was able to use that request body to send make that api call work. This indicates that the problem is me and not the API.
I have successfully done this with curl through command line...
curl -X POST "https://formrecognizerbp.cognitiveservices.azure.com/formrecognizer/v1.0-preview/custom/models/[MODEL ID]/analyze" -H "Content-Type: multipart/form-data" -F "form=#\"C:\Temp\Capture1.jpg\";type=image/jpeg" -H "Ocp-Apim-Subscription-Key: [SUBSCRIPTION ID]"
I don't really know/can't figure out how to convert that into a request body similar in format to what the POST Train Model method has.
I keep getting this error because I don't know how to format the request properly.
Internal : Unexpected error Error during Web API HTTP Request
HTTP Status Code: 400
HTTP Response Content: {"value":{"error":{"code":"BadRequest","message":"Could not process incoming request: 'Missing content-type boundary.'. Please ensure that it is well-formed."}},"formatters":[],"contentTypes":[],"statusCode":400}
So I guess formatters and contentTypes are the missing pieces but would that just be the file path and the image/jpeg parts of the curl ?
The /trainCustomModel API expects the data to be present on the Azure Blob storage. The request body to this request needs to contain a valid SAS URL to the training data. Once you successfully create a custom trained model ID, you could use that to analyze the forms. The /AnalyzeWithCustomModel API expects the data to be on your local file storage. Please ensure that you have the replaced the ModelId, API Subscription Key (Note this is not the same as subscription ID) and the local path to the image correctly.
The issue was that I did not realize that the curl script was overwritting the multipart/form-data Content-Type with image/jpeg and when I was trying to build out this call differently I was forcing a multipart/form-data Content-Type on what was a jpeg.
I am working on an online shop using 3dcart, i want to connect the store to an inventory management store called ChannelGrabber. Channel Grabber has provided me with a public and private key with some bits of their API.
$ curl -v -X POST -d "grant_type=client_credentials&client_id=f836e7675c46adbc33d98e32c06dfc6f&client_secret=2f4e72f89bda7f15062a2ba9d107adb5" https://api.orderhub.io/accessToken
> POST /accessToken HTTP/1.1
> User-Agent: curl/7.35.0
> Host: api.orderhub.io
> Accept: */*
> Content-Length: 119
> Content-Type: application/x-www-form-urlencoded
>
< ...response headers...
{
"access_token": "aVSyKhKNPi5XXJqlIMCNfeZwSfvTvasTcWyX2lv2",
"token_type": "Bearer",
"expires_in": 3600
}
$ curl -v -X GET -H "Authorization: Bearer aVSyKhKNPi5XXJqlIMCNfeZwSfvTvasTcWyX2lv2" https://api.orderhub.io/ping
> GET /ping HTTP/1.1
> User-Agent: curl/7.35.0
> Host: api.orderhub.io
> Accept: */*
> Authorization: Bearer aVSyKhKNPi5XXJqlIMCNfeZwSfvTvasTcWyX2lv2
>
< ...response headers...
pong
3d cart have provided the following git project has an example of how to connect up to their clients API. https://github.com/3dcart/REST-API-Client/tree/master/3dCartRestAPIClient.
My issue is that i have basically no idea on how to go about connecting the 2 services up. What language to use other then using Json but i'm not even sure that is possible, I'm only still a student and still quite new to the world of programming so i don't want to have to say i can't do this project and i would quite like to learn how to do this.
Can anyone point me in the right direction?
I was asking myself a similar question some time ago. I am already familiar with basic requesting of rest apis using python. I want to connect the apis of an online sales tool called pipedrive and a tool for generating invoices and bills called billomat. both come with a sophisticated rest api and I know how to get data from them or create new data into them.
If I now create a python script on my local computer I can imagine what I'd have to code to pull eg customer data from pipedrive and create this customers data into billomat. The thing now is that this process is completely manual.
To have the process be completely automatic, I came to the following conclusion:
Use webhooks in pipedrive to send out data when certain events are happening
the data can only be sent to a url which generally should also be a rest api
this url cannot be billomat directly because it wont unterstand or know what to do with the data
thats why I decided to code a litte api myself and host it on my private webserver
this api receives data from pipedrive, processes it, eg maps field names from a customers record to the corresponding field names in billomat, and then sends the prepared data over to billomat in a format that it expects and understands
I know this does not directly answer the OPs question, but would be my suggestion for a fully automatic solution in case you cannot alter the behaviour of at least one of the two apis you'd like to connect.
REST (Representational state transfer) is a way of interfacing your data. The idea is that the action should be defined by the HTTP request method (GET, PUT, POST etc.), while the URL should have no verb/action, just kind of data.
JSON is just the way of communication between server and client. It's like 2 people deciding to speak the same language.
Now, in your client, you can make requests to as many services as you need, and interpret the results. This can be achieved in virtually any programming language. You will find a lot of libraries for both handling HTTP request and parsing JSON responses.
As for the right direction. Pick a programming language you are more familiar with (if it's hard to decide I would recommend python which is fairly easy to start with) and look for libraries for sending HTTP request and parsing json strings.
I need to be able to do the following (with plain cURL & JSON server-side- no frameworks or Java):
Use a string representation of a Keycloak access token I have been given by a 3rd party to verify that the token is valid.
If the token is valid, get the Keycloak ID for that user.
How do I do this using plain old HTTP posts? I've found lots of Java examples but I need to know the raw HTTP POSTs and responses underneath.
Is it something like this to validate the token?
/auth/realms/<realm>/protocols/openid-connect/validate?access_token=accesstokenhere
What does this return in terms of data (sorry I currently have no test server to interrogate)?
Thanks.
The validate endpoint does not seem to work now. It used to return access token. I am using the keycloak 2.5.1 now. As mentioned in post by Matyas (and in the post referenced by him), had to use introspect token endpoint.
In my testing Bearer authentication did not work. Had to use Basic authentication header along with base64 encoded client credentials.
base64.encode("<client_id:client_secret>".getBytes("utf-8"))
The response from introspect endpoint is in JSON format as shared in post referenced by Maytas, has many fields based on type of token being introspected. In my case token_type_hint was set as access_token.
requestParams = "token_type_hint=access_token&token=" + accessToken
The response included required user details like username, roles and resource access. Also included OAuth mandated attributes like active, exp, iss etc. See rfc7662#page-6 for details.
Maybe you need this:
http://lists.jboss.org/pipermail/keycloak-user/2016-April/005869.html
The only one problem is that, introspect is not working with public clients.
The key url is:
"http://$KC_SERVER/$KC_CONTEXT/realms/$REALM/protocol/openid-connect/token/introspect"
You need to authorize your client e.g. with basic auth, and need to give the requester token to introspect:
curl -u "client_id:client_secret" -d "token=access_token_to_introspect" "http://$KC_SERVER/$KC_CONTEXT/realms/$REALM/protocol/openid-connect/token/introspect"
A friend and I are following the Autodesk Viewer tutorial here:
https://developer.autodesk.com/en/docs/viewer/v2/tutorials/basic-viewer/
We've both completed all the previous steps except for the very last of using the APIs POST job endpoint to send a base64 encoded urn off to the Model Derivative API for conversion to a SVF file so that Autodesk's Viewer may use it.
This is what I'm using to send the POST job request:
curl -X 'POST' -H 'Authorization: Bearer cywr9WWu8kml3rQIVnPDohFDsoRl' -H
'Content-Type: application/json' -v 'https://developer.api.autodesk.com/modelderivative/v2/designdata/job'
-d '{"input": {"urn": "dXJuOmFkc2sub2JqZWN0czpvcy5vYmplY3Q6bXl0ZXN0YnVja2V0dGVzdDIvZ2lyaV9zaXJfY29weTEucnZ0"},
"output": {"formats": [{"type": "svf","views": ["2d","3d"]}]}}'
and this is the APIs response:
{"type":"manifest","hasThumbnail":"false","status":"failed","progress":"complete",
"region":"US","urn":"dXJuOmFkc2sub2JqZWN0czpvcy5vYmplY3Q6bXl0ZXN0YnVja2V0dGVzdDIvZ2lyaV9zaXJfY29weTEucnZ0",
"derivatives":[{"name":"giri_sir_copy1.rvt","hasThumbnail":"false","status":"failed",
"progress":"complete","messages":[{"type":"error","code":"Revit-InternalError",
"message":"<message>We have encountered some issues while preparing the file for viewing.
Please contact support for assistance.</message>"},{"type":"error",
"message":"Unrecoverable exit code from extractor: -1073741831",
"code":"TranslationWorker-InternalFailure"}],"outputType":"svf"}]}
Any insights is greatly appreciated.
Summarizing the comments on the question: here is the link to the sample running and source code (postJob function).
Please note this sample is storing the file directly under the developer account (OSS), not under the user account (Data Management), it's an important difference! The first uses 2-legged token, the second uses 3-legged token. Data Management contains some abstractions/metadata to organize the files, like hubs, projects and folders, but ultimately stores on OSS. Depending on the way it was stored, you need different tokens/permissions to read/write it.
I've set up a MediaWiki server on an Azure website with the PluggableAuth and OpenID Connect extensions. The latter uses the PHP OpenID Connect Basic Client library. I am an administrator in the Azure AD domain example.com, wherein I've created an application with App ID URI, sign-on URL and reply URL all set to https://wiki.azurewebsites.net/. When I navigate to the wiki, I observe the following behavior (cookie values omitted for now):
Client Request
GET https://wiki.azurewebsites.net/ HTTP/1.1
RP Request
GET https://login.windows.net/example.com/.well-known/openid-configuration
IP Response
(some response)
RP Response
HTTP/1.1 302 Moved Temporarily
Location: https://login.windows.net/{tenant_id}/oauth2/authorize?response_type=code&redirect_uri=https%3A%2F%2Fwiki.azurewebsites.net%2F&client_id={client_id}&nonce={nonce}&state={state}
Client Request
(follows redirect)
IP Response
HTTP/1.1 302 Found
Location: https://wiki.azurewebsites.net/?code={code}&state={state}&session_state={session_state}
Client Request
(follows redirect)
RP Request (also repeats #2 & #3)
POST https://login.windows.net/{tenant_id}/oauth2/token
grant_type=authorization_code&code={code}&redirect_uri=https%3A%2F%2Fwiki.azurewebsites.net%2F&client_id={client_id}&client_secret={client_secret}
IP Response
(As interpreted by MediaWiki; I don't have the full response logged at this time)
AADSTS50001: Resource identifier is not provided.
Note that if I change the OpenID PHP client to provide the 'resource' parameter in step 8, I get the following error response from AAD instead:
RP Request
POST https://login.windows.net/{tenant_id}/oauth2/token
grant_type=authorization_code&code={code}&redirect_uri=https%3A%2F%2Fwiki.azurewebsites.net%2F&resource=https%3A%2F%2Fwiki.azurewebsites.net%2F&client_id={client_id}&client_secret={client_secret}
IP Response
AADSTS90027: The client '{client_id}' and resource 'https://wiki.azurewebsites.net/' identify the same application.
(This has come up before.)
Update
I've made some progress based on #jricher's suggestions, but after working through several more errors I've hit one that I can't figure out. Once this is all done I'll submit pull requests to the affected libraries.
Here's what I've done:
I've added a second application to the example.com Azure AD domain, with the App ID URI set to mediawiki://wiki.azurewebsites.net/, as a dummy "resource". I also granted the https://wiki.azurewebsites.net/ application delegated access to this new application.
Passing in the dummy application's URI as the resource parameter in step #8, I'm now getting back the access, refresh, and ID tokens in #9!
The OpenID Connect library requires that the ID token be signed, but while Azure AD signs the access token it doesn't sign the ID token. It comes with the following properties: {"typ":"JWT","alg":"none"}. So I had to modify the library to allow the caller to specify that unsigned ID tokens are considered "verified". Grrr.
Okay, next it turns out that the claims can't be verified because the OpenID Provider URL I specified and the issuer URL returned in the token are different. (Seriously?!) So, the provider has to be specified as https://sts.windows.net/{tenant_id}/, and then that works.
Next, I found that I hadn't run the MediaWiki DB upgrade script for the OpenID Connect extension yet. Thankfully that was a quick fix.
After that, I am now left with (what I hope is) the final problem of trying to get the user info from AAD's OpenID Connect UserInfo endpoint. I'll give that its own section.
Can't get the user info [Updated]
This is where I am stuck now. After step #9, following one or two intermediate requests to get metadata and keys for verifying the token, the following occurs:
RP Request:
(Updated to use GET with Authorization: Bearer header, per MSDN and the spec.)
GET https://login.windows.net/{tenant_id}/openid/userinfo
Authorization: Bearer {access_token}
IP Response:
400 Bad Request
AADSTS50063: Credential parsing failed. AADSTS90010: JWT tokens cannot be used with the UserInfo endpoint.
(If I change #10 to be either a POST request, with access_token in the body, or a GET request with access_token in the query string, AAD returns the error: AADSTS70000: Authentication failed. UserInfo token is not valid. The same occurs if I use the value of the id_token in place of the access_token value that I received.)
Help?
Update
I'm still hoping someone can shed light on the final issue (the UserInfo endpoint not accepting the bearer token), but I may split that out into a separate question. In the meantime, I'm adding some workarounds to the libraries (PRs coming soon) so that the claims which are already being returned in the bearer token can be used instead of making the call to the UserInfo endpoint. Many thanks to everyone who's helped out with this.
There's also a nagging part of me that wonders if the whole thing would not have been simpler with the OpenID Connect Basic Profile. I assume there's a reason why that was not implemented by the MediaWiki extension.
Update 2
I just came across a new post from Vittorio Bertocci that includes this helpful hint:
...in this request the application is asking for a token for itself! In Azure AD this is possible only if the requested token is an id_token...
This suggests that just changing the token request type in step 8 from authorization_code to id_token could remove the need for the non-standard resource parameter and also make the ugly second AAD application unnecessary. Still a hack, but it feels like much less of one.
Justin is right. For authorization code grant flow, your must specify the resource parameter in either the authorization request or the token request.
Use &resource=https%3A%2F%2Fgraph.windows.net%2F to get an access token for the Azure AD Graph API.
Use &resource=https%3A%2F%2Fmanagement.core.windows.net%2F to get a token for the Azure Service Management APIs.
...
Hope this helps
Microsoft's implementation of OpenID Connect (and OAuth2) has a known bug where it requires the resource parameter to be sent by the client. This is an MS-specific parameter and requiring it unfortunately breaks compatibility with pretty much every major OAuth2 and OpenID Connect library out there. I know that MS is aware of the issue (I've been attempting to do interoperability testing with their team for quite a while now), but I don't know of any plans to fix the problem.
So in the mean time, your only real path is to hack your client software so that it sends a resource parameter that the AS will accept. It looks like you managed to make it send the parameter, but didn't send a value that it liked.
I had issues getting this running on Azure, even though I got something working locally. Since I was trying to setup a private wiki anyway, I ended up enabling Azure AD protection for the whole site by turning on:
All Settings -> Features -> Authentication / Authorization
From within the website in https://portal.azure.com
This made it so you had to authenticate to Azure-AD before you saw any page of the site. Once you were authenticated a bunch of HTTP Headers are set for the application with your username, including REMOTE_USER. As a result I used the following plugin to automatically log the already authenticated user into Azure:
https://www.mediawiki.org/wiki/Extension:Auth_remoteuser