Reason for failure in MultipartObjectAssembler OCI object storage - oracle-cloud-infrastructure

I'm using MultipartObjectAssembler to upload data from a Database to OCI object storage.
Is there a way to know the reason for failure when using multipart upload?
When I try to commit assembler I'm getting IllegalStateException with the message "One or more parts were have not completed upload successfully".
I would like to know why any part got failed? I couldn't find a way to get this information from SDK.
try {
assembler.addPart(new ByteArrayInputStream(part, 0, length),
length,
null);
assembler.commit();
} catch (Exception e) {
assembler.abort();
throw new RuntimeException(e.getMessage());
}
Edit: I need to get an Exception thrown by a failed part and propagate the error message.

Is there a reason you are not using the UploadManager? It should do everything for you, including adding parts and committing. Here's an end-to-end example: https://github.com/oracle/oci-java-sdk/blob/master/bmc-examples/src/main/java/UploadObjectExample.java
If, for some reason, you cannot use UploadManager, please take a look at its source code nonetheless, since it demonstrates the intended usage of MultipartObjectAssembler: https://github.com/oracle/oci-java-sdk/blob/master/bmc-objectstorage/bmc-objectstorage-extensions/src/main/java/com/oracle/bmc/objectstorage/transfer/UploadManager.java#L175-L249
You create the MultipartObjectAssembler:
MultipartObjectAssembler assembler =
createAssembler(request, uploadRequest, executorServiceToUse);
You create a new request. This gives you back a MultipartManifest, which will later let you check if parts failed.
manifest =
assembler.newRequest(
request.getContentType(),
request.getContentLanguage(),
request.getContentEncoding(),
request.getOpcMeta());
Then you add all the parts:
assembler.addPart(
ProgressTrackingInputStreamFactory.create(
chunk, progressTrackerFactory.getProgressTracker()),
chunk.length(),
null);
Then you commit. This is where your code currently throws. I suspect not all parts have been added.
CommitMultipartUploadResponse response = assembler.commit();
If something goes wrong, check MultipartManifest.listCompletedParts(), MultipartManifest.listFailedParts(), and MultipartManifest.listInProgressParts(). The manifest should tell you what parts failed. Unfortunately not why; for that, you can enable ERROR level logging for com.oracle.bmc.objectstorage.transfer (for the class com.oracle.bmc.objectstorage.transfer.internal.MultipartTransferManager in particular).
If I have misunderstood something, please let me know. In that case, a larger, more complete code snippet would help me debug. Thanks!

Related

Typo3 extbase : How can i catch an exception?

I'm currently develloping an extension in Typo3 10.4 and I can't working out a problem
I'm using some external libs for mailing or payments which sometimes throw exceptions.
My problem is that when this is happening i got an OOPS error even if I try to catch the exception
for exemple :
//CODE BEFORE
try{
//SOME CODE WHO SENDS EMAIL AND SOMETIMES THROW EXCEPTION
//BECAUSE THE CONNECTION FAILED
}catch(Exception $e){
//DO SOMETHING
}
//CODE AFTER
And it's a pretty annoying problem because some DB actions are not completed then
I'm sure there is a way to deal with the exception without stopping all the script but i don't know how...
Can someone help plz?
Thanks guys
Your approach is correct and TYPO3 shouldn't interfere. Thus, there seems to be another issue, so a few things to consider here:
Please keep in mind that your TYPO3 extension code is namespaced, if you really catch(Exception $e) it means \Your\Namespace\Exception - you probably want catch(\Exception $e).
That said, additionally configure a way to show the exception stack trace instead of "Oops" as output (the more simple approach, see e.g. How do I enable Error Reporting in TYPO3?) or connect a remote debugger like xdebug (more advanced) in order to see more than the Oops and get the real cause of the error and how it propagates.
If you aren't able to change the actual TYPO3 system you work on for reasons, an alternative could be to use an easy to set up local dev system like DDEV. This offers a simplified way to create a local TYPO3 instance, see e.g. the TYPO3 blog article

Exception handling with Realbrowserlocusts

In using realbrowserlocusts class it appears that I'm limited in any exception handling.
The only reference that partially works is: self.client.wait.until(EC.visibility_of_element_located ....
In a failed condition where the element is not found the script simply starts over again. With the script I'm working with I need to maintain a solid session state; I need to throw and exception(report an error), log the user out and then let the script start over again. I've been testing out the behavior with the locust.py script that Nick B. created with several approaches to "try, except" and they work running without realbrowserlocusts (selenium only) but with it the execution just stops.
Any examples would be greatly appreciated.
In its current format I've been able to run 3x the amount of a browser-based load per/agent/slave than our commercial tool. My goal is to replace it with a locust/selenium approach.
locust-plugins's WebdriverUser has a little bit better exception handling I think. A failure to find an element will log a failed request and if you use RescheduleTaskOnFail (as in the the example) it will restart the task when that happens.
https://github.com/SvenskaSpel/locust-plugins/blob/master/examples/webdriver_ex.py

How to handle "Unexpected EOF at target" error from API calls?

I'm creating a Forge application which needs to get version information from a BIM 360 hub. Sometimes it works, but sometimes (usually after the code has already been run once this session) I get the following error:
Exception thrown: 'Autodesk.Forge.Client.ApiException' in mscorlib.dll
Additional information: Error calling GetItem: {
"fault":{
"faultstring":"Unexpected EOF at target",
"detail": {
"errorcode":"messaging.adaptors.http.flow.UnexpectedEOFAtTarget"
}
}
}
The above error will be thrown from a call to an api, such as one of these:
dynamic item = await itemApi.GetItemAsync(projectId, itemId);
dynamic folder = await folderApi.GetFolderAsync(projectId, folderId);
var folders = await projectApi.GetProjectTopFoldersAsync(hubId, projectId);
Where the apis are initialized as follows:
ItemsApi itemApi = new ItemsApi();
itemApi.Configuration.AccessToken = Credentials.TokenInternal;
The Ids (such as 'projectId', 'itemId', etc.) don't seem to be any different when this error is thrown and when it isn't, so I'm not sure what is causing the error.
I based my application on the .Net version of this tutorial: http://learnforge.autodesk.io/#/datamanagement/hubs/net
But I adapted it so I can retrieve multiple nodes asynchronously (for example, all of the nodes a user has access to) without changing the jstree. I did this to allow extracting information in the background without disrupting the user's workflow. The main change I made was to add another Route on the server side that calls "GetTreeNodeAsync" (from the tutorial) asynchronously on the root of the tree and then calls it on each of the returned children, then each of their children, and so on. The function waits until all of the nodes are processed using Task.WhenAll, then returns data from each of the nodes to the client;
This means that there could be many api calls running asynchronously, and there might be duplicate api calls if a node was already opened in the jstree and then it's information is requested for the background extraction, or if the background extraction happens more than once. This seems to be when the error is most likely to happen.
I was wondering if anyone else has encountered this error, and if you know what I can do to avoid it, or how to recover when it is caught. Currently, after this error occurs, it seems that every other api call will throw this error as well, and the only way I've found to fix it is to rerun the code (I use Visual Studio so I just rerun the server and client, and my browser launches automatically)
Those are sporadic errors from our apigee router due to latency issues in the authorization process that we are currently looking into internally.
When they occur please cease all your upcoming requests, wait for a few minutes and retry again. Take a look at stuff like this or this to help you out.
And our existing reports calling out similar errors seem to point to concurrency as one of the factors leading up to the issue so you might also want to limit your concurrent requests and see if that mitigate the issue.

Which values can DocumentClientException.Error.Code have?

I want my data access layer to handle exceptions thrown by DocumentDB API provided via Microsoft.Azure.Documents.Client.DocumentClient class. For example, the optimistic concurrency check implemented using AccessCondition class, but others as well.
By looking at the exception thrown, the best way to recognize different DocumentClient-specific exceptions seems to be something like this:
try { ... }
catch (DocumentClientException exception)
when (exception.Error.Code == "Some magic here")
{
//let the user know how to recover from this..
}
I don't like such magic strings as they are not verifiable compile-time. It may contract a typo, or it may change on random moment with DocumentDB client/server changes, etc. Also, it is not clear which such magic codes I could/should be handling since I don't see the Microsoft.Azure.DocumentDB .net API containing any ErrorCodes enum or constants, nor find any list in documentation.
Where can I find a list of possible Error.Code values DocumentClient API can throw?
To make it even more confusing, the XmlDoc for DocumentClient.CreateDocumentAsync method suggest working instead on http status codes.
UPDATE: This question is not about Http status codes but DocumentClientException.Error.Code field as I assume the latter is more precise.
Where can I find a list of possible error codes values DocumentClient API can throw?
It's hard to find the completely list of error code that DocumentClinet API throw. The exception is depend on what your request.
For example, the optimistic concurrency check
Azure Cosmos DB uses ETags for handling optimistic concurrency.
When we retrieve a document from Azure Cosmos DB, it always contains an ETag property as apart of our document.
When we then want to send our request to replace a document, we can specify an AccessCondition with the ETag we received when we fetched out our document.
If the ETag we send is not current, the server will return a 412 Precondition Failed status code. In our .NET SDK, this is wrapped up in a DocumentClientException.
Here is an example that show the possible problems when the concurrency occurred.
By decompile the version 1.22.0 client, the code is set as a HttpStatusCode enum. I think all the possible values can be found here https://msdn.microsoft.com/en-us/library/system.net.httpstatuscode(v=vs.110).aspx then.
However, what really contains richer information for debug is the Error.Message. Might need to decompile the whole library to figure out, or wait for Microsoft to release the source codes, which is unlikely to happen since the latest update in github was 2 or 3 years ago.
public Error Error
{
get
{
if (this.error == null)
{
this.error = new Error()
{
Code = this.StatusCode.ToString(),
Message = this.Message
};
}
return this.error;
}
}
There is a list of the HTTP Status Codes for Azure Cosmos DB
I use the following code in my catch blocks
catch (DocumentClientException e)
{
var resp = new HttpResponseMessage
{
StatusCode = (HttpStatusCode) e.StatusCode,
Content = new StringContent(e.Message)
};
return resp;
}
Letting the user know how to handle the exception should be done on the client application.

Autodesk DM API: Is Retry appropriate here?

I've got an application that's been working for a long time.
Recently we created a new app/keys for it, and it's behaving strangely.
(I did figure out the scope requirements had been put in place. I am requesting bucket:create bucket:read data:read data:write).
When I upload a file to a bucket, I've traditionally called done the call to get the object details afterwards, to verify that it's successfully uploaded.
With the new key, I am intermittently getting this error:
GetObjectDetails: InternalServerError {"fault":{"faultstring":"Execution of ServiceCallout servicecallout-auth-acm-request failed. Reason: timeout occurred servicecallout-auth-acm-request","detail":{"errorcode":"steps.servicecallout.ExecutionFailed"}}}
Is this something I should be re-trying with a sleep in between? or is it indicative of something wrong with the upload?
(FYI - putting in a retry seems to have have resolved this for me, but I still don't know if that's the right answer - and if this issue might happen on other calls).
It could be that the service requires a slight delay between a put object and a get, so I would suggest either use a timer or a retry as you mentioned. However a successful response from the upload should be enough to ensure your object has been placed to the bucket without the need to double check.