I have already a code to retrieve the objects in the bucket using oci-java-sdk and this is working as expected. I would like to retrieve the URL of the file which was uploaded to the bucket in object storage and when I use this URL, this should redirect to the actual location without asking any credentials.
I saw preauthenticated requests but again i need to create one more request. I dont want to send one more request and want to get URL in the existing GetObjectResponse.
Any suggestions>
Thanks,
js
The URL of an object is not returned from the API but can be built using information you know (See Update Below!). The pattern is:
https://{api_endpoint}/n/{namespace_name}/b/{bucket_name}/o/{object_name}
Accessing that URL will (generally, see below) require authentication. Our authentication mechanism is described at:
https://docs.cloud.oracle.com/en-us/iaas/Content/API/Concepts/signingrequests.htm
Authentication is NOT required if you configure the bucket as a Public Bucket.
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/managingbuckets.htm?TocPath=Services%7CObject%20Storage%7C_____2#publicbuckets
As you mentioned, Pre-authenticated Requests (PARs) are an option. They are generally used in this situation, and they work well.
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/usingpreauthenticatedrequests.htm
Strictly speaking, it is also possible to use our Amazon S3 Compatible API...
https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm
...and S3's presigned URLs to generate (without involving the API) a URL that will work without additional authentication.
https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html
Update: A teammate pointed out that the OCI SDK for Java now includes a getEndpoint method that can be used to get the hostname needed when querying the Object Storage API. https://docs.cloud.oracle.com/en-us/iaas/tools/java/1.25.3/com/oracle/bmc/objectstorage/ObjectStorage.html#getEndpoint--
Related
Just started working on an existing project making use of the Exact Online API.
While I was debugging the project I suddenly only started receiving XML results instead of JSON results from the API. I did not change anything about the endpoints being queried I was just running the existing queries trying to figure some things out.
These are the REST API docs: https://start.exactonline.nl/docs/HlpRestAPIResources.aspx
These are the XML docs: https://support.exactonline.com/community/s/knowledge-base#All-All-DNO-Content-xmlsamplecode
Typical REST API endpoints look like this:
https://start.exactonline.be/api/v1/xxxxxx/salesinvoice/SalesInvoices
Typical XML endpoints look like this:
https://start.exactonline.be/docs/XMLDownload.aspx
I also did not change any settings. I only have access to the tokens and api. I don't have access to the account.
This is an example of an endpoint and query where I previously received JSON but am now receiving XML:
https://start.exactonline.be/api/v1/xxxxxx/salesinvoice/SalesInvoices?$filter=InvoiceID eq guid'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx'&$select=InvoiceID
I tried this manually with Postman and also using the existing code from the project.
Is there some setting I am unaware of? Am i querying the wrong way? Maybe there have been some changes to the API I am unaware of that aren't listed in the release notes?
Please provide the request header Accept in your HTTP request that specifies what content format you prefer to receive: application/json. The default of Exact Online APIs is XML (but seldom to never used).
Can anyone comment on if it should be possible to use rclone's swift support to access buckets in OCI object storage (new OCI, not classic).
I'm interested in it because S3 compatibility mode is limited to a single designated compartment and I'd like to be able to use rclone with any bucket in my tenancy.
I know that for public buckets there is still a swift style URL. The 3 functional URLs styles seem to be:
Native: https://objectstorage.{region}.oraclecloud.com/n/{object-storage-namespace}/b/{bucket}/o/{filename}
Swift: https://swiftobjectstorage.{region}.oraclecloud.com/v1/{object-storage-namespace}/{bucket}/{filename}
S3: https://{object-storage-namespace}.compat.objectstorage..oraclecloud.com/{bucket}/{filename}
https://docs.cloud.oracle.com/en-us/iaas/Content/Identity/Tasks/managingcredentials.htm talks a little bit about Swift password (Auth Tokens) and you can create one in the console.
But I can't find anything about what the auth URL would be for the non-classic version of object storage. And storage_url with a auth_token doesn't seem to work either.
Using -vvvv doesn't show anything more than 401 Unauthorized.
I'm interested in it because S3 compatibility mode is limited to a single designated compartment and I'd like to be able to use rclone with any bucket in my tenancy.
The designated compartment only controls where buckets created via that protocol (S3 or Swift) are placed. The designated compartment does not affect authorization. Authorization is controlled by the relevant IAM policies.
But I can't find anything about what the auth URL would be for the non-classic version of object storage. And storage_url with a auth_token doesn't seem to work either.
The new/current OCI Object Storage does not support auth URLs. You must use HTTP basic-style auth with Swift on OCI. It does not seem that rclone supports HTTP basic auth with swift directly (it is possible to create the basic auth header yourself and have rclone send it).
All that said, using rclone with s3 is the best approach for OCI Object Storage. Ensure you set the "region" option to the correct region name like "us-phoenix-1" and you should be good.
Thanks!
I want to create pre-authenticated request for an object inside a bucket in the OCI object storage using python SDK. I found out that I can use get_preauthenticated_request for the bucket to put objects inside the bucket but not to get the objects pre-authenticated. I can create a pre-authenticated request using the OCI console but I need to do it in a python script. can anybody help me in this issue?
You can use create_preauthenticated_request (see code) for both buckets and individual objects.
The difference is in the access type:
ANY_OBJECT_WRITE is for the whole bucket
OBJECT_READ, OBJECT_READ_WRITE and OBJECT_WRITE are for objects
So you should be able to create a Pre-Authenticated Request with something like
request_details = create_preauthenticated_request_details()
request_details.access_type("ObjectReadWrite")
par = create_preauthenticated_request("namespace", "bucket", request_details)
You can find more on the request details here and for the request itself here.
Let me know if this works for you, I don't have an account to test against at the moment.
I got a (Flask) backend powering an API that serves JSON to an Angular app.
I love the fact that my backend (algorithms, database) is totally disconnected from my frontend (design, UI) as it could literally run from two distinct servers. However since the view is entirely generated client side everyone can access the JSON data obviously. Say the application is a simple list of things (the things are stored in a JSON file).
In order to prevent direct access to my database through JSON in the browser console I found these options :
Encrypting the data (weak since the decrypting function will be freely visible in the javascript, but not so easy when dealing with minified files)
Instead of $http.get the whole database then filtering with angular, $http.get many times (as the user is scrolling a list for example) so that it is programmatically harder to crawl
I believe my options are still weak. How could I make it harder for a hacker to crawl the whole database ? Any ideas ?
As I understand this question - the user should be permitted to access all of the data via your UI, but you do not want them to access the API directly. As you have figured out, any data accessed by the client cannot be secured but we can make accessing it a little more of PITA.
One common way of doing this is to check the HTTP referer. When you make a call from the UI the server will be given the page the request is coming from. This is typically used to prevent people creating mashups that use your data without permission. As with all the HTTP request headers, you are relying on the caller to be truthful. This will not protect you from console hacking or someone writing a scraper in some other language. #see CSRF
Another idea is to embed a variable token in the html source that bootstraps your app. You can specify this as an angular constant or a global variable and include it in all of your $http requests. The token itself could be unique for each session or be a encrypted expiration date that only the server can process. However, this method is flawed as well as someone could parse the html source, get the code, and then make a request.
So really, you can make it harder for someone, but it is hardly foolproof.
If users should only be able to access some of the data, you can try something like firebase. It allows you to define rules for who can access what.
Security Considerations When designing web applications, consider
security threats from:
JSON vulnerability XSRF Both server and the client must cooperate in
order to eliminate these threats. Angular comes pre-configured with
strategies that address these issues, but for this to work backend
server cooperation is required.
JSON Vulnerability Protection A JSON vulnerability allows third party
website to turn your JSON resource URL into JSONP request under some
conditions. To counter this your server can prefix all JSON requests
with following string ")]}',\n". Angular will automatically strip the
prefix before processing it as JSON.
For example if your server needs to return:
['one','two'] which is vulnerable to attack, your server can return:
)]}', ['one','two'] Angular will strip the prefix, before processing
the JSON.
Cross Site Request Forgery (XSRF) Protection XSRF is a technique by
which an unauthorized site can gain your user's private data. Angular
provides a mechanism to counter XSRF. When performing XHR requests,
the $http service reads a token from a cookie (by default, XSRF-TOKEN)
and sets it as an HTTP header (X-XSRF-TOKEN). Since only JavaScript
that runs on your domain could read the cookie, your server can be
assured that the XHR came from JavaScript running on your domain. The
header will not be set for cross-domain requests.
To take advantage of this, your server needs to set a token in a
JavaScript readable session cookie called XSRF-TOKEN on the first HTTP
GET request. On subsequent XHR requests the server can verify that the
cookie matches X-XSRF-TOKEN HTTP header, and therefore be sure that
only JavaScript running on your domain could have sent the request.
The token must be unique for each user and must be verifiable by the
server (to prevent the JavaScript from making up its own tokens). We
recommend that the token is a digest of your site's authentication
cookie with a salt for added security.
The name of the headers can be specified using the xsrfHeaderName and
xsrfCookieName properties of either $httpProvider.defaults at
config-time, $http.defaults at run-time, or the per-request config
object.
Please Kindly refer the below link,
https://docs.angularjs.org/api/ng/service/$http
From AngularJS DOCs
JSON Vulnerability Protection
A JSON vulnerability allows third party website to turn your JSON resource URL into JSONP request under some conditions. To counter this your server can prefix all JSON requests with following string ")]}',\n". Angular will automatically strip the prefix before processing it as JSON.
There are other techniques like XSRF protection and Transformations which will further add security to your JSON communications. more on this can be found in AngularJS Docs https://docs.angularjs.org/api/ng/service/$http
You might want to consider using JSON Web Tokens for this. I'm not sure how to implement this in Flask but here is a decent example of how it can be done with a Nodejs backend. This example at least shows how you can implement it in Angularjs.
http://www.kdelemme.com/2014/03/09/authentication-with-angularjs-and-a-node-js-rest-api/
Update: JWT for Flask:
https://github.com/mattupstate/flask-jwt
I am trying to build my application's admin UI using sling's userManager REST interface, but I would like to customize the json rendering. For example, I would like the response of "Get group" to include the members only if the requestor is a member.
I started by adding libs/sling/group/json.esp but I don't understand how I can get hold of the default response and customize it. Even if I had to query and form the json from scratch, where can I find information about APIs available to get this data from JCR/Sling?
I found that I could use ResourceTraversor to dump the resource object in json form but using new Packages.org.apache.sling.servlets.get.impl.helpers.ResourceTraversor(-1, 10000, resource, true) in the esp throws up an error
There are a few things to note here.
First, you should avoid putting your code under the libs directory. Your app code should live under the apps directory. When attempting to resolve a servlet for a URI, Sling will check apps before it checks libs so if you need to completely override functionality delivered with Sling, you would place your code in apps.
Second, what is (probably, depending on how you have things setup) happening when you request http://localhost:8080/system/userManager/group/administrators.tidy.1.json is the request is being handled by Sling's default GET servlet, because it finds no other script or servlet which is applicable. For research purposes it might be worth looking at the code for the default get servlet, org.apache.sling.servlets.get.impl.DefaultGetServlet, to see what it's using to render JSON. If you need to handle the rendering of a user group in a manner different than what the default GET servlet is doing, then you would need to create a servlet which is listening for requests for resources of type sling/group. It would probably be ideal to create a servlet for this purpose and register it with OSGI. http://sling.apache.org/site/servlets.html provides the various properties you would need to set to ensure the servlet resolver finds your servlet. Your servlet then would handle the request and as such would have direct and easy access to the requested resource.
Third, the particular need you specified is that you do not want the group members to render unless the requesting user is a member of the group requested. This is more of an access control issue than a rendering issue. Sling and Jackrabbit, out of the box, make as few assumptions as possible concerning how you might want your application to be setup. That being the case, you need to establish the access controls that are applicable for your particular use case. The wiki post on Access Control in the Jackrabbit wiki ( http://wiki.apache.org/jackrabbit/AccessControl ) goes into this to an extent.
Using directions from Paul Michelotti's answer, I researched further and found a suitable solution to my problem.
Sling accepts request filters (javax.servlet.Filter) through SCR annotations like the one below
#SlingFilter(scope = SlingFilterScope.REQUEST, order = Integer.MIN_VALUE)
Every request is passed down to the filter before it is processed by the servlet. Using the resourceType, I was able to distinguish requests to group.1.json and group/mygroup.1.json. Since the filter also has access to the current user, I was able to decide to deny the request if it did not abide by my security model and return a 404 status code.
Please refer to this page for details on filters. You can also check out the sample project urlfilter for directions on usage.