I'm using C# and via a webapi I need to send a request to a server and process the response. The data that comes back from the server has the form like:
{"A_Field":a,"B_Field":"B_FieldValue","C_Field":5}
{"TYPE":"Values","Column1":["1234","456789","12345"],"Column2":["abc","defg","high"]}
{"TYPE":"Values","Column1":["45353","23422","6755"],"Column2":["sfs","fhfghg","vzczzc"]}
{"TYPE":"Values","Column1":["4564","5664","12345"],"Column2":["abfsdc","vxvx","cxvbvx"]}
{"TYPE":"Values","Column1":["5664","979","7978"],"Column2":["ryrt","qweq","nbmm"]}
....
{"A_Field":b,"COMPLETE:"YES"}
I need to process the parts that start with {"TYPE":"Values"....}
There are an exceptionally large amount of these messages that are returned from the server. So to be able to process each line/message as it is received would be ideal. The last line that has "COMPLETE:"YES"} signals the end of the message. I tried the below code:
try{
using (var handler = new HttpClientHandler())
{
handler.Credentials = new System.Net.NetworkCredential("user", "password");
using (var client = new HttpClient(handler))
{
client.BaseAddress = new Uri("http://<server>:port/api/rest/");
await client.GetAsync(blah_query)
.ContinueWith(taskwithresponse =>
HttpResponseMessage response = taskwithresponse.Result;
}
}
}
catch(Exception e)
{
Console.WriteLine("Error {0}",e.Message);
}
The output gives:
Error: System.AggregateException: One or more errors occurred. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Http.HttpContent.LimitMemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Http.StreamToStreamCopy.TryStartWriteSync(Int32 bytesRead)
at System.Net.Http.StreamToStreamCopy.BufferReadCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task 1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task 1.get_Result() at WebCall.Program.
at System.Threading.Tasks.ContinuationTaskFromResultTask`1.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) atSystem.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at WebCall.Program.<RunAsync>d__3.MoveNext() in
---> (Inner Exception #0) System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
...
at System.Net.Http.HttpContent.LimitMemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Http.StreamToStreamCopy.TryStartWriteSync(Int32 bytesRead)
at System.Net.Http.StreamToStreamCopy.BufferReadCallback(IAsyncResult ar)<---
This fails when the response is very large. I'm guessing that processing has to be done in the loop? But I believe what is happening here is that it's waiting for the entire message to return. Or is this the completely wrong way to go about it.
I'm not familiar with web programming. But can someone help advise on how I can process the returned data in an optimal way? Is it possible to do it a line/message at a time? A code sample/tutorial would be a great help. Most of what i've seen uses a small amount of data.
Does anyone know how I progress this?
Related
I'm running into an exception in C# code intended to set the contents of a WinUI3 Image object. I don't know how to interpret the exception message.
Here's the code:
using var responseStream = await response.Content.ReadAsInputStreamAsync();
var memStream = new InMemoryRandomAccessStream();
await RandomAccessStream.CopyAsync(responseStream, memStream);
memStream.Seek(0);
var retVal = new Image();
var source = new BitmapImage();
// the exception is thrown on the next line
await source.SetSourceAsync( outStream );
retVal.Source = source;
The exception message is
Unspecified error (Error HRESULT E_FAIL has been returned from a call to a COM component.)
What's weird is that calls to this code sometimes succeed and sometimes don't, even when the exact same parameters (which define the Uri that the response stream is derived from) are specified and the exact same response is received.
That suggests there's something in the context of the call that's the problem. The cases that succeed are when the code is called from a viewmodel in a WinUI3 test app. The cases that fail are when the code is called from a custom WinUI3 control I've written.
I thought there might be a "called from the wrong thread" problem but in both the success and failure cases the code is being called from the main UI thread.
Another odd thing is that the code throwing the exception is contained within a try/catch block (not shown for simplicity)...but the catch block doesn't catch the exception. Instead, it gets caught by the unhandled exception handler generated by WinUI3.
Advice on how to proceed with researching what's going wrong -- or potential solutions! -- would be appreciated, thanx.
After searching for days now and reading pretty much everything related to that, I'm finally posting my question here, since I couldn't find a solution for my specific problem.
I want my REST WebServices to return the original Exception, that has been thrown or at least the correct StackTrace. To test this, I'm using JUnit integrationtests and Wildfly 13 as app-server. After researching I found 2 possible solutions.
1.Using Exception Mappers
While this magical thing catches all of my Exceptions and allows me to return a Response, I've noticed that my StackTrace is changed if I use it like in the example. For example, "com.test.TestClass" is turned into "null.thread" or "null.interceptor". It seems like somehow the exception is changed on the way and the paths to the class are lost or censored, but I can't make sense of it.
Also I couldn't find any restrictions for the Response.entity, be it size, datatype or security.
As far as I understand, you can catch the ExceptionMapper Response OR a WebApplicationException, which contains the response. In my case, the response in the WebApplicationException contains all the relevant data except the (correct) StackTrace.
2.Using WebApplicationException
Another Solution would be to simply throw WebApplicationException instead of ECEException and not using a mapper. If I do that and catch it, the Exception is empty though. It doesn't contain any of the data set, it's always 500 - InternalServerError (I guess, Wildfly couldn't handle it then and threw an exception itself).
Or is it not supposed to be catched/thrown like that? Do I need to convert it to JSon or can I expect it to simply work out of the box with my annotations in the WebServiceInterface and the Response MediaType? Does it even make sense to put a full Response within a WebApplicationException? I mean, both contain fields for the ErrorCode, which seems redundand, even though there is a constructor for that approach.
Long story short:
What's the best approach to catch all possible exceptions and retrieve the full stack trace? Reading this post, I guess catching all "Exception"s is fine and they are always returned as WebApplicationExceptions, but the stack trace is still gone/malformed... your thoughts?
**JUnitTest**
#Test
public void testCreateTask_ClusterInvalid() throws IOException {
final RPETask taskToCreate = new RPETask();;
try
{
final long tid = taskManagerWebService.createTask(taskToCreate);
}
catch (WebApplicationException e) //Responses are ALWAYS catched as WebApplicationException
{
Response response = e.getResponse();
String emString = response.readEntity(String.class);
Gson gson = new Gson();
ECEWebErrorMessage errorMessage = gson.fromJson(emString, ECEWebErrorMessage.class);
errorMessage.displayErrorInformationOnConsole();
}
}
**WebServiceInterface**
#POST
#Path(URI_CREATE_TASK)
#Consumes(WebServiceNames.JSON)
#Produces(WebServiceNames.JSON)
long createTask(final RPETask task) throws ECEException;
**WebService**
#Override
public long createTask(final RPETask task) throws ECEException {
if (LOGGER.isTraceEnabled()) {
LOGGER.trace("createTask(" + task + ")");
}
return taskManager.createTask(task);
}
**ManagerBeanInterface**
long createTask(RPETask task) throws ECEException;
**ManagerBean**
#Override
public long createTask(final RPETask task) throws ECEException {
final ClusterEngineBean cluster = find(ClusterEngineBean.class, task.getCluster());
if (cluster == null) {
throw new ECEObjectNotFoundException(ClusterEngineBean.class, task.getCluster());
}
}
**ExceptionMapper**
#Provider
public class GenericWebExceptionMapper implements ExceptionMapper<Exception> {
final Log logger = LogFactory.getLog(getClass());
#Override
public Response toResponse(Exception ex) {
//At this point, the Exception is fully available -> Sending it as Response breaks it!
logger.error("GenericWebExceptionMapper -> toResponse(Throwable ex)", ex);
ECEWebErrorMessage errorMessage = new ECEWebErrorMessage(500,
ex.getMessage(),
ex.getClass().getCanonicalName(),
ex.getStackTrace());
return Response.status(Status.INTERNAL_SERVER_ERROR)
.entity(errorMessage)
.type(MediaType.APPLICATION_JSON)
.build();
}
}
After more research I've finally found a solution for myself.
Why is the StackTrace gone/malformed?
It's for security reasons. Wildfly automatically detects outgoing StackTraces and censors them, using interceptors. Im not sure if you can do anything about that, but I guess you shouldn't do that anyway.
What is the best approach?
Using Exception Mappers worked for me. Instead of catching them as WebApplicationException, you can always expect a response with the appropriote error code and handle them that way. For example error code 200 = OK, do this... error code 404 = NOTFOUND, do that...I that case your WebServices should always return Responses and contain the object you want to retreive in the entity field of the Response.
Feel free to add additional information to this solution.
I'm trying to build a WP8 app and I keep getting the following error:
Any ideas?
Error 12 The "GenerateWinPRTManifest" task failed unexpectedly.
System.Xml.XmlException: '.', hexadecimal value 0x00, is an invalid character. Line 1, position 1.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)
at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Char[] data, Int32 length, Int32 invCharPos)
at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
at System.Xml.Linq.XDocument.Load(String uri, LoadOptions options)
at Microsoft.Silverlight.Build.Tasks.GenerateWinPRTManifest.IsManifestUptoDate()
at Microsoft.Silverlight.Build.Tasks.GenerateWinPRTManifest.SaveDocumentIfUpdated()
at Microsoft.Silverlight.Build.Tasks.GenerateWinPRTManifest.ExecuteImplementation()
at Microsoft.Silverlight.Build.Tasks.GenerateWinPRTManifest.Execute()
at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__20.MoveNext()
Not sure what was wrong but deleting the contents of the Bin and obj folder and rebuilding resolved the issue.
I have a project on WinForms with this code:
AppDomain.CurrentDomain.UnhandledException += CurrentDomainUnhandledException;
private void CurrentDomainUnhandledException(object sender, UnhandledExceptionEventArgs e)
{ }
The e.ExceptionObject contains full StackTrace.
In Win Store project:
this.UnhandledException += (s, e) =>{
{
MarkedUp.AnalyticClient.LogLastChanceException(e);
};
e.Exception.StackTrace is null.
Both exceptions were generated by this code:
int a=0;
....
try
{
int i = 1 / a;
}
catch (Exception exp)
{
throw;
}
Any Ideas?
The reference on MSDN suggests that this is a limitation: http://msdn.microsoft.com/en-us/library/windows/apps/windows.ui.xaml.application.unhandledexception
A notable limitation is that the UnhandledException event arguments don’t contain as much detail as the original exception as propagated from app code. Whenever possible, if the app requires specific processing of a certain exception, it’s always better to catch the exception as it propagates, because more detail will be available then. The UnhandledException event arguments expose an exception object through the Exception property. However, the type, message, and stack trace of this exception object are not guaranteed to match those of the original exception that was raised. The event arguments do expose a Message property. In most cases, this will contain the message of the originally raised exception.
Are you running the solution in Debug mode? There seems to be happening something in the App.g.i.cs file when you run the solution in Debug mode. When I run your sample in Release mode the stacktrace is available in the UnhandledException event.
In my test solution it breaks first here:
#if DEBUG && !DISABLE_XAML_GENERATED_BREAK_ON_UNHANDLED_EXCEPTION
UnhandledException += (sender, e) =>
{
if (global::System.Diagnostics.Debugger.IsAttached) global::System.Diagnostics.Debugger.Break();
};
#endif
And after that one it goes to the UnhandledException handler that I have defined in the app.xaml.cs file. In Debug the stacktrace is gone, in Release mode the stacktrace and the exception details are there.
There is a web-service deployed on tomcat 6 and exposed via apache-cxf 2.3.3. A generated sources stubs using wsdl2java to be able to call this service.
Things seemed fine until I sent big request(~1Mb). This request wasn't processed and failing with exception:
Interceptor for {http://localhost/}ResourceAllocationServiceSoapService has thrown
exception, unwinding now org.apache.cxf.binding.soap.SoapFault:
Error reading XMLStreamReader.
...
com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog
at [row,col {unknown-source}]: [1,0]
Is some kind of max request length here, I'm totally stuck with it.
Vladimir's suggestion worked. This code below will help others with understanding where to put the 1000000.
public void handleMessage(SoapMessage message) throws Fault {
// Get message content for dirty editing...
InputStream inputStream = message.getContent(InputStream.class);
if (inputStream != null)
{
String processedSoapEnv = "";
// Cache InputStream so it can be read independently
CachedOutputStream cachedInputStream = new CachedOutputStream(1000000);
try {
IOUtils.copy(inputStream,cachedInputStream);
inputStream.close();
cachedInputStream.close();
InputStream tmpInputStream = cachedInputStream.getInputStream();
try{
String inputBuffer = "";
int data;
while((data = tmpInputStream.read()) != -1){
byte x = (byte)data;
inputBuffer += (char)x;
}
/**
* At this point you can choose to reformat the SOAP
* envelope or simply view it just make sure you put
* an InputStream back when you done (see below)
* otherwise CXF will complain.
*/
processedSoapEnv = fixSoapEnvelope(inputBuffer);
}
catch(IOException e){
}
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Re-set the SOAP InputStream with the new envelope
message.setContent(InputStream.class,new ByteArrayInputStream( processedSoapEnv.getBytes()));
/**
* If you just want to read the InputStream and not
* modify it then you just need to put it back where
* it was using the CXF cached inputstream
*
* message.setContent(InputStream.class,cachedInputStream.getInputStream());
*/
}
}
I figured out what was wrong. Actually it was bug inside interceptor's code:
CachedOutputStream requestStream = new CachedOutputStream()
When I replaced this with
CachedOutputStream requestStream = new CachedOutputStream(1000000);
things start working fine.
So the request was just trunkated during copying of streams.
I run into same issue of geting "com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog" when using CachedOutputStream class.
Looking at sources of CachedOutputStream class the threshold is used to switch between storing stream's data from "in memory" to "a file".
Assuming stream operates on data that exceeds threshold it gets stored in a file thus following code is going to break
IOUtils.copy(inputStream,cachedInputStream);
inputStream.close();
cachedInputStream.close(); //closes the stream, the file on disk gets deleted
InputStream tmpInputStream = cachedInputStream.getInputStream(); //returned tmpInputStream is brand *empty* one
// ... reading tmpInputStream here will produce WstxEOFException
Increasing 'threshold' does help as all stream data is stored into memory and in such scenario calling cachedInputStream.close() does not really close the underlying stream implementation so one can still read from it later on.
Here is 'fixed' version of above code (at least it worked without exception for me)
IOUtils.copy(inputStream,cachedInputStream);
inputStream.close();
InputStream tmpInputStream = cachedInputStream.getInputStream();
cachedInputStream.close();
// reading from tmpInputStream here works fine
Temporary file gets deleted when close() is called on tmpInputStream and there are no more other references to it, see source code of CachedOutputStream.maybeDeleteTempFile()