I want to create pre-authenticated request for an object inside a bucket in the OCI object storage using python SDK. I found out that I can use get_preauthenticated_request for the bucket to put objects inside the bucket but not to get the objects pre-authenticated. I can create a pre-authenticated request using the OCI console but I need to do it in a python script. can anybody help me in this issue?
You can use create_preauthenticated_request (see code) for both buckets and individual objects.
The difference is in the access type:
ANY_OBJECT_WRITE is for the whole bucket
OBJECT_READ, OBJECT_READ_WRITE and OBJECT_WRITE are for objects
So you should be able to create a Pre-Authenticated Request with something like
request_details = create_preauthenticated_request_details()
request_details.access_type("ObjectReadWrite")
par = create_preauthenticated_request("namespace", "bucket", request_details)
You can find more on the request details here and for the request itself here.
Let me know if this works for you, I don't have an account to test against at the moment.
Related
I have already a code to retrieve the objects in the bucket using oci-java-sdk and this is working as expected. I would like to retrieve the URL of the file which was uploaded to the bucket in object storage and when I use this URL, this should redirect to the actual location without asking any credentials.
I saw preauthenticated requests but again i need to create one more request. I dont want to send one more request and want to get URL in the existing GetObjectResponse.
Any suggestions>
Thanks,
js
The URL of an object is not returned from the API but can be built using information you know (See Update Below!). The pattern is:
https://{api_endpoint}/n/{namespace_name}/b/{bucket_name}/o/{object_name}
Accessing that URL will (generally, see below) require authentication. Our authentication mechanism is described at:
https://docs.cloud.oracle.com/en-us/iaas/Content/API/Concepts/signingrequests.htm
Authentication is NOT required if you configure the bucket as a Public Bucket.
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/managingbuckets.htm?TocPath=Services%7CObject%20Storage%7C_____2#publicbuckets
As you mentioned, Pre-authenticated Requests (PARs) are an option. They are generally used in this situation, and they work well.
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/usingpreauthenticatedrequests.htm
Strictly speaking, it is also possible to use our Amazon S3 Compatible API...
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm
...and S3's presigned URLs to generate (without involving the API) a URL that will work without additional authentication.
https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html
Update: A teammate pointed out that the OCI SDK for Java now includes a getEndpoint method that can be used to get the hostname needed when querying the Object Storage API. https://docs.cloud.oracle.com/en-us/iaas/tools/java/1.25.3/com/oracle/bmc/objectstorage/ObjectStorage.html#getEndpoint--
Can anyone comment on if it should be possible to use rclone's swift support to access buckets in OCI object storage (new OCI, not classic).
I'm interested in it because S3 compatibility mode is limited to a single designated compartment and I'd like to be able to use rclone with any bucket in my tenancy.
I know that for public buckets there is still a swift style URL. The 3 functional URLs styles seem to be:
Native: https://objectstorage.{region}.oraclecloud.com/n/{object-storage-namespace}/b/{bucket}/o/{filename}
Swift: https://swiftobjectstorage.{region}.oraclecloud.com/v1/{object-storage-namespace}/{bucket}/{filename}
S3: https://{object-storage-namespace}.compat.objectstorage..oraclecloud.com/{bucket}/{filename}
https://docs.cloud.oracle.com/en-us/iaas/Content/Identity/Tasks/managingcredentials.htm talks a little bit about Swift password (Auth Tokens) and you can create one in the console.
But I can't find anything about what the auth URL would be for the non-classic version of object storage. And storage_url with a auth_token doesn't seem to work either.
Using -vvvv doesn't show anything more than 401 Unauthorized.
I'm interested in it because S3 compatibility mode is limited to a single designated compartment and I'd like to be able to use rclone with any bucket in my tenancy.
The designated compartment only controls where buckets created via that protocol (S3 or Swift) are placed. The designated compartment does not affect authorization. Authorization is controlled by the relevant IAM policies.
But I can't find anything about what the auth URL would be for the non-classic version of object storage. And storage_url with a auth_token doesn't seem to work either.
The new/current OCI Object Storage does not support auth URLs. You must use HTTP basic-style auth with Swift on OCI. It does not seem that rclone supports HTTP basic auth with swift directly (it is possible to create the basic auth header yourself and have rclone send it).
All that said, using rclone with s3 is the best approach for OCI Object Storage. Ensure you set the "region" option to the correct region name like "us-phoenix-1" and you should be good.
Thanks!
I am trying to get weather forecasts data from OpenWeatherMap and integrate them in Orion by performing a registeration request.
I was able to register and get the API key from OpenWeatherMap, however, the latter returns a JSON file with all the data inside, which is not supported by ORION.
I have followed the step by step tutorial https://fiware-tutorials.readthedocs.io/en/latest/context-providers/index.html#context-provider-ngsi-proxy where they have acquired the data from OpenWeatherMap using NGSI proxy, an API key is required to be indicated in the docker-compose file as an environment variable, however, the data acquired is the "current data" and not forecast and also specific to Berlin.
I have tried to access the files inside the container "fiware/tutorials.context-provider" and try to modify and match the parameters to my needs but I feel like I am taking a long blocked path.
I don't think that's even considered as good practice but I have run out of ideas :(
Can anyone suggest how I could bring the forecast data to Orion and register it as a context provider?
Thank you in advance.
I imagine you aim to implement a context provider, able to speak NGSI with Orion.
OpenWeatherMap surely doesn't implement NGSI ...
If you have the data from OpenWeatherMap, as a JSON string, perhaps you should parse the JSON and create your entities using some select key-values from the parsed OpenWeatherMap? Save the entity (entities) locally and then register those keys in Orion.
Alternatively (easier but I wouldn't recommend it), create local entities with the entire OpenWeatherMap data as the value of an attribute of the entity:
{
"id": "id-from-OpenWeatherMap",
"type": "OpenWeatherMap",
"weatherData": {
"value":
...
}
...
}
Then you register id/weatherData in Orion.
I am trying to figure out how exactly is the authorization-string for IBM Cloud object storage created. I do have an accessKeyId and accessKeySecret.
I am able to connect to the bucket using cyberDuck. But need to manage the objects using Java and Rest APIs.
Any pointers ?
Found the signature generation sample for java(and others) here : https://console.bluemix.net/docs/services/cloud-object-storage/hmac/hmac-signature.html#java-example
Additional info : S3 admin can set the authorization from AWS Signature to a BASE64 auth as well.
I'd like to write an object to gcp object store, while using the x-goog-if-generation-match feature. Using #google-cloud/storage npm library, the file object does not seem to have an option for setting the required object generation.
What are the alternatives?
As you noticed, the #google-cloud/storage npm library doesn't support generation and metageneration preconditions.
As an alternative, you may use either the Storage XML API or the Storage JSON API which do support it. Depending on if you want to use one or the other, you'll be able to use preconditions via HTTP Headers or query string parameters. You'll find the whole list of those here.
Another alternative is to use some kind of optimistic locking:
get the generation id
write object
get the generation id again
repeat until generation after = generation before + 1