What is the correct way to hit an internal Foundry API from a Code Repository using, for example, a Python transform?
This is possible but somewhat discouraged because of the security impacts. Specifically the token that is used to call the API. Historically, Foundry jobs were run with the building user's complete token. This allows making any API call the user could make, but could be abused by a nefarious actor. Therefore most build today use a project-scoped token which can only read and write datasets, and not make API calls.
Thus you must either un-project-scope the repository so that it uses user tokens, which can be done through the Jemma API, or by supplying a hard-coded token, which can be done through a secured dataset with an appropriate marking, but be aware anyone who can read this dataset could steal the token.
A product support solution called logic flows is coming to make this process smoother.
Once you have a token making the API calls is similar to any other API. Here's an example in python, there's more information in the documentation.
URL = f"https://foundry.url/stemma/api/repos/{repo}/checks"
headers = {
"Authorization": "Bearer " + token
}
req = requests.get(URL, headers=headers)
if req.status_code > 299:
continue
req_json = req.json()
Currently the functionality for accessing Foundry APIs from within a Code Repository is not by default supported.
Because of the mentioned Foundry limitations and project scoped tokens, we create dedicated Service Accounts for automations, create a bearer token for the service account and store it in a dataset which we secure with a Marking. In the transform where we use the token to make api calls, we stop_propagating the Marking to downstream transforms.
This is, unfortunately, rather cumbersome and we are looking here at Palantir's product development team for a better solution.
Related
I have a Google Cloud Function. I also have a web application. I want to authenticate requests to the cloud function by using a service account.
I have the json key file.
I know that I have to follow https://cloud.google.com/functions/docs/securing/authenticating#service-to-function. But that is leading me to an IAP page that does not apply to google cloud functions.
Another similar instructions are found in https://developers.google.com/identity/protocols/oauth2/service-account
But if I am following the python library code, I end up with the sample code there :
import googleapiclient.discovery
sqladmin = googleapiclient.discovery.build('sqladmin', 'v1beta3', credentials=credentials)
response = sqladmin.instances().list(project='exciting-example-123').execute()
This does not directly relate to invoking a cloud function.
This question's answer somewhat deals with my requirement but is using a Call API which is only suitable for testing.
Also, I want to expose this API to multiple applications using another tech like .net. So I believe the best option for me will be to use the HTTP method (given on the same page):
https://developers.google.com/identity/protocols/oauth2/service-account#httprest
But whatever I do I am unable to get the signature right.
Any help to get this sorted will be highly appreciated as I am stuck on this for the past few days.
You can use the Google auth library like this
from google.oauth2.id_token import fetch_id_token
from google.auth.transport import requests
audience="my_audience"
r = requests.Request()
token=fetch_id_token(r,audience)
print(token)
The fetch_id_token method will use the default credentials
The service account key file defined in the environment variable GOOGLE_APPLICATION_CREDENTIALS
The service account loaded in the Google Cloud environment
For now, I followed this answer in PHP
In the claims section, I removed the scope. Instead added a claim of target_audience.
"target_audience" => "google-function-http-trigger"
the cloud function http trigger will look like https://us-central1-test-project-name.cloudfunctions.net/function-name",
This will give the required assertion key.
Then I follow https://developers.google.com/identity/protocols/oauth2/service-account#httprest to get the id_token
Then with the id_token as the bearer token we can call the cloud function.
please note that the token expires depending on the time set in the "exp" claim. Once expired you have to redo the steps to generate the new id_token
I want to authenticate requests to the cloud function by using a service account.
I am not sure I understand the context correctly, but I would try to assign a roles/cloudfunctions.invoker IAM role to that service account (which is used to run your code in the web application) - see Cloud Functions IAM Roles .
In that case a code under that service account "Can invoke an HTTP function using its public URL"
I reckon no json keys are required in this case.
I have a google cloud function in Java.
Client will invoke the function using HTTP trigger URL.
But that is not secure. I have gone through some docs saying that you should pass a token or client ID and then verify it in server side.
Can anyone explain that in detail and please provide a code example if any.
My doubt is to authenticate the client while they invoke the function using Http trigger
This page explains quite well all the capacity that you have to authenticate a requester on Cloud Functions.
If you have users, the best way is to use Firebase Auth (our Google Cloud Identity Platform which is simply a more advance solution than Firebase Auth with more features)
However, you need to grant all you user with cloudfunction.invoker role, to allow them to invoke the Cloud Functions. It could be difficult. You can also perform the check on your side, but in this case you remove the security (filter) layer of google and you have to check all the traffic by yourselves (not really safe, in term of billing and in case of attack).
The latest solution, API keys, is not recommended, especially for the users. But for machine to machine it's sometime the only solution. However, there isn't out of the box solution and for this I wrote an article, that explains how to create a Cloud Endpoint (or now a Cloud API Gateway which is the serverless solution of Cloud Endpoint with ESPv2) to accept API Keys.
With this latest solution, if you change your security definition, you can also accept OAuth2 tokens coming from Firebase Auth (or Cloud Identity Platform), but this time, you don't need to grant all the users on your Cloud Functions IAM role. The token only need to be valid and it's the Cloud Endpoint service account which is used to perform the call (and thus which needs to be authorized on the Cloud Functions).
In addition, because you can accept OAuth2 token, you can also accept non Google token, and thus have your users in any IDP OAuth2 compliant (KeyCloak, Okta,...)
You could use external OAuth server like keycloack (https://github.com/keycloak/keycloak), or use somethging like Json Web Tokens -- https://jwt.io/ -- available for various languages, siutable for microservices.
I'm the administrator of a Bim360 Docs Hub,and i'm trying to retrieve some data from the forge API. I've already created a test app, and i get a Bim 360 Account Id.
I'm not able to get an authentication bearear token (OAuth), to make simple get requests towards projects.
Where do i get the auth token ?
Thanks
See tutorial here to aquire OAuth tokens and note that you will need different permutations of OAuth scopes (esp. data:read data:write etc.) for different operations, as is specified in the docs of each and every API, see an example here.
You may use a RESTful client like Postman to talk to our APIs and manage the authentication flow and operate on response data with JavaScript as you like so as to build workflows such as to create a new project.
Last but not least before you can access BIM360 APIs make sure you provision access to your app.
I am working with an API for automating tasks in a company I work for.
The software will run from a single server and there will only one instance of the sensitive data.
I have a tool that our team uses at the end of every day.
The token only needs to be requested once since it has a +-30 minute timeout.
Since I work with Salesforce API, the user has to enter his/her password either way since it relates the ticket to their account.
The API oAuth2 tokens and all of its sensitive components need to be secured.
I use PowerShell & a module called FileCryptograhy to produce an AES version of my config.json.
In my config file, I store all the component keys that need to be used to generate the token itself.
Steps
Base64 encode strings
Use FileCyptography module to encrypt the JSON file with a secret key into an AES file.
When API needs to produce a token, it works in reverse to get all the data.
Is this a valid way of securing sensitive API data, or is there a more efficient way?
P.S: I understand that nothing is very secure and can be reverse engineered, I just need something that will keep at least 90% of people away from this data.
In a restful API, POST should be used to create, and GET should be used to read.
Sometimes, for security reasons, you can't pass sensitive variables in the URI. For example, if you have an /accounts API that requires you to pass "accountNumber". If your security team won't let you put that in the URI, then you have to use the POST method instead of the GET method to service the "read" operation, and you can then provide the account number in the request body.
That brings me to my question: If you've used up the POST method to service a read operation, how do you service the "create" operation of the same API?
"/accounts/create" wouldn't be advised because your APIs should be nouns, not verbs. It doesn't seem right to use up a different HTTP method like PUT. This issue has to come up a lot so I'm curious what people are doing to get around it?
If you're using POST for this, you are not building a RESTful service. The way I see this, you have 2 options:
Accept this, and build a more RPC-like system.
Change the id's in your application so that they are not a security risk. If knowing an id is a problem, consider using something else.