Is there a way to change the download name when calling a pre-authorized request url in oci object storage? Currently it uses the name of the storage object id when it downloads the object to the local disk. I want to use the opc-meta-filename instead.
We wont be able to change the object storage name, When you upload objects, you can provide optional response headers and user-defined metadata. Response headers are HTTP headers sent from Object Storage to Object Storage clients when objects are downloaded, so we wont be able to modify it.
Related
I am routing messages from an Azure IoT Hub to a blob container (Azure Storage as a routing endpoint). The messages sent to the IoT Hub are of Content Type: 'application/json' and Content Encoding: 'UTF-8'. However, when they arrive in blob storage several of these messages are batched together into one file with Content Type 'application/octet-stream'. Thus, for instance Power BI is not able to read these files in JSON format when reading directly from the blob.
Is there any way to route these messages so that each single message is saved as a json file in the blob container?
Tl;dr : Please make use of the Encoding option to specify AVRO or JSON format & Batch Frequency/Size to control the batch.
"With an Azure Storage container as a custom endpoint, IoT Hub will write messages to a blob based on the batch frequency and block size specified by the customer. After either the batch size or the batch frequency is hit, whichever happens first, IoT Hub will then write the enqueued messages to the storage container as a blob. You can also specify the naming convention you want to use for your blobs, as shown below."
The below image shows how we navigate to the IoTHub's message routing section to add a custom endpoint of a blob storage account.
-The below image shows how we configure the settings of the batch count and the size. Also please make use of the Encoding section to specify the message format such as AVRO or JSON
Please leave a comment below to let us know if you need further help in this matter.
The message encoding needs to be done by the device stream or as part of a module to translate the protocol. Each protocol (AMQP, MQTT, and HTTP) uses a different method to encode the message from base64 to UTF-8.
To route messages based on message body, you must first add property 'contentType' (ct) to the end of the MQTT topic and set its value to be application/json;charset=utf-8. An example is shown below.
devices/{device-id}/messages/events/$.ct=application%2Fjson%3Bcharset%3Dutf-8
https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
I am using Python SDK for OCI. I tried the Upload manager example and its working perfectly fine when i try to upload files from file system. But i have to expose this python code as REST service (using flask) and files to be uploaded to object storage will come as payload for REST. Does it have to multipart/mixed content type in this case or can it be multipart/form-data as well.
#user1945183, are you asking if Upload Manager can support multipart/form-data? Yes, Upload Manager can take in multipart/form-data.
The Upload Manager uses Object Storage's CreateMultipartUpload API. You can learn more about the CreateMultipartUpload API doc.
From CreateMultipartUploadDetails Reference you will find that content-type is optional and have no effect on Object Storage behavior.
The optional Content-Type header that defines the standard MIME type format of
the object to upload. Specifying values for this header has no effect on
Object Storage behavior. Programs that read the object determine what to do
based on the value provided. For example, you could use this header to
identify and perform special operations on text only objects.
I have already a code to retrieve the objects in the bucket using oci-java-sdk and this is working as expected. I would like to retrieve the URL of the file which was uploaded to the bucket in object storage and when I use this URL, this should redirect to the actual location without asking any credentials.
I saw preauthenticated requests but again i need to create one more request. I dont want to send one more request and want to get URL in the existing GetObjectResponse.
Any suggestions>
Thanks,
js
The URL of an object is not returned from the API but can be built using information you know (See Update Below!). The pattern is:
https://{api_endpoint}/n/{namespace_name}/b/{bucket_name}/o/{object_name}
Accessing that URL will (generally, see below) require authentication. Our authentication mechanism is described at:
https://docs.cloud.oracle.com/en-us/iaas/Content/API/Concepts/signingrequests.htm
Authentication is NOT required if you configure the bucket as a Public Bucket.
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/managingbuckets.htm?TocPath=Services%7CObject%20Storage%7C_____2#publicbuckets
As you mentioned, Pre-authenticated Requests (PARs) are an option. They are generally used in this situation, and they work well.
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/usingpreauthenticatedrequests.htm
Strictly speaking, it is also possible to use our Amazon S3 Compatible API...
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm
...and S3's presigned URLs to generate (without involving the API) a URL that will work without additional authentication.
https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html
Update: A teammate pointed out that the OCI SDK for Java now includes a getEndpoint method that can be used to get the hostname needed when querying the Object Storage API. https://docs.cloud.oracle.com/en-us/iaas/tools/java/1.25.3/com/oracle/bmc/objectstorage/ObjectStorage.html#getEndpoint--
I am trying to get a JSON file from a url. It is stored on "abc.com", but I am working on "xyz.com".
I do not have access to change the JSON url or add CORS to the server.
You must be on our internal network to access "abc.com", so I can't use any sort of external proxy.
Is there any other way for me to get a JSON file from an external url?
Thanks
I want to know where does ExtJS store keeps all the data? I know the data is stored in the memory but I want to know does it used HTML 5 local storage internally or if any other technique is employed?
Thanks,
Deepesh
It depends.
In every case, the data of the store is stored in a Javascript object. The store persists its data via a proxy. It is a matter of configuration how this data is stored. You can configure different types of proxies:
Client side storage
LocalStorageProxy - saves its data to localStorage if the browser supports it
SessionStorageProxy - saves its data to sessionStorage if the browsers supports it
MemoryProxy - holds data in memory only, any data is lost when the page is refreshed
Server side storage
Ajax - sends requests to a server on the same domain
JsonP - uses JSON-P to send requests to a server on a different domain
Rest - uses RESTful HTTP methods (GET/PUT/POST/DELETE) to communicate with server
Direct - uses Ext.direct.Manager to send requests
More details are in the docs.
The data is stored in an in memory collection called a MixedCollection. It's an ordered collection, but it also allows you to look up data by key, so it's like having an ordered hashmap.