In Node-Red is there a way to integrate a JSON-Msg, into an INSERT-Statement as a string without the usage of stringify? Because after 2000 Characters, the Message is cut off.
Messages shown in the debug sidebar are truncated for performance reasons, the actual message is not truncated.
You can see the whole message by sending it to the console (with the tick box in the debug node).
You can also edit/set the debugMaxLength in settings.js to change the truncation point.
Related
Hello I have an error page for user when their account gets locked. When I view the source code of page it just contains one line
%ERROR%
I am trying to understand if this is some kind of string interpolation as I have never seen it done with % sign. The error message that is displayed is this
The raw file you are viewing is a response page template. When that page is presented to the client making the request, the %MACRONAME% values are substituted and the HTML (or JSON) content returned from the web reverse proxy has that substituted text. Hint, you can further customize the messages to the user by substituting the macro into a javascript variable within the response page template, and then make logic decisions based on that client side javascript code. For example, you could catch one of the IBM error codes and translate it into saying something like "Account is locked".
More information is available on the macros within the IBM documentation.
I have a basic NIFI flow with one GenerateFlowFile processor and one LookupRecord processor with RestLookupService.
I need to do a rest lookup and enrich the incoming flow file with rest response.
I am getting errors that the lookup service is unable to lookup coordinates with the value I am extracting from the incoming flow file.
GenerateFlowFile is configured with simple JSON
LookupRecord is configured to extract the key from the JSON and populate it to the RestLookupService. Also, JsonReader and JsonSetWriter is configured to read the incoming flow file and to write the response back to the flow file
The RestLookupService itself exits with JsonParseException about unexpected character '<'
RestLookupService is configured with my API running in the cloud in which I am trying to use the extracted KEY from the incoming flow file.
The most interesting bug is that when I configure the URL to point for example to mocky.io everything works correctly so it means that the issue itself is tight with the API URL I am using (http://iotosk.ddns.net:3006/devices/${key}/getParsingData). I have tried also removing the $key, using the exact URL, using different URLs..
Of course the API is working OK over postman/curl anything else. I have checked the logs on the container that the API is running on and there is no requests in the logs what means that nifi is failing even before reaching the API. At least on application level...
I am absolutely out of options without any clue how to solve this. And with nifi also google is not helpful.
Does anybody see any issue in the configuration or can point me in some direction what can cause this issue?
After some additional digging. The issue was connected with authentication logic even
before the API receives it and that criples the request and and returned XML as Bryan Bende suggested in the comment.
But definitely better logging in nifi will help to solve this way faster...
I have noticed that many http clients including Firefox and Chrome don't allow file downloads for http response codes with 4XX and 5XX. However, some clients allow these downloads, like curl and wget (with --content-on-error option).
Both Chrome and Firefox don't provide nice exception messages.
Chrome fails with ERR_INVALID_RESPONSE. Firefox fails with File not found. As stated above for the curly and wget work for the same URL.
I was wondering if there is a specification that defines the correct behavior in this case? Are there good reasons why the request can't be processed by Chrome and Firefox? Also, it seems strange that they don't provide proper feedback.
I think for most cases a download for failing requests makes no sense, but for some cases it would be helpful. One good example where downloading a file even in the error case would be if there is a client that only communicates with the server using some 3rd party format. The client would have to download a generated file for the request. In the case of an error, the client should download a file containing the error description.
For example the RFC7231 states
Response messages with an error status code
usually contain a payload that represents the error condition, such
that it describes the error state and what next steps are suggested
for resolving it.
The 4xx (Client Error) class of status code indicates that the client
seems to have erred. Except when responding to a HEAD request, the
server SHOULD send a representation containing an explanation of the
error situation, and whether it is a temporary or permanent
condition. These status codes are applicable to any request method.
User agents SHOULD display any included representation to the user.
This doesn't forbid downloading in the case of an error.
Edit because of the first answer:
I don't think that this behavior is user friendly and I don't think that user friendliness is really the reason behind this. For example it would make way more sense to show the error code and error message (provided in the header) to the user. Or least indicate the error with an error message like "cannot download the file, because the server responded with an error". There might be servers that can only respond with XML or any other random file format.
What bugs me the most is that both browsers respond with different but arbitrary errors that don't hint any information about the underlying issue.
It might be that this is an undocumented edge case and both Chrome and Firefox just fall back to a default error, but this seems unlikely, especially because this is an edge case that has a special flag in wget.
4XX: Why would you assume a file download if your client did something wrong?
If we assume that an API has an endpoint that replies with a certain file format, it is fair to assume that also the error message including a hint what the client did wrong is provided in that format. So the file can help to fix the client error.
I'm not aware of any specification for that topic.
The behavior should be as user friendly as possible.
4XX:
Why would you assume a file download if your client did something wrong? Furthermore, the client software could not differ between the case of wrong usage(e.g. invalid url) and handling a file download.
5xx:
As you stated most api provide error information, but you could also not differ the case of downloading and for example an internal error providing the file.
You can use such behavior with wget and curl as you mentioned, but its not user friendly nor practical for using such an API programmatically.
The above info in mind, Chrome and firefox just try to be user friendly.
I hope I could somehow answer your question or challenge the idea behind it. :)
Looking at chromium handle download and not 2xx we see:
// The response code indicates that this is an error page, but we don't
// know how to display the content. We follow Firefox here and show our
// own error page instead of intercepting the request as a stream or a
// download.
So Chrome followed Firefox, and both are entirely consistent with the RFCs, the browser knows this payload is unidentified data relating to an error condition, so saving it as the file in question is not an option. Since it is being downloaded, presumably the browser can't display the payload, but in either case has been instructed not to, so displaying it in the error context is not a safe option. Since it is an error there is also a high likelihood that the sender has combined a partial response with an error code meaning that the payload contents may be an incomplete or corrupt representation of data from a 2xx response/etc.
If you look back at wget, --content-on-error is a specific option because it is the wrong thing to do as a general browser. A client side that works with the payload type could examine the errors when it is directly interacting with a server and wget is only providing options to help you debug such an interaction. A normal browser has less features to help emulate other clients for debugging than a text CLI, since a text CLI exists primarily to emulate some other client while debugging.
I was wondering if there is a specification that defines the correct
behavior in this case? Are there good reasons why the request can't be
processed by Chrome and Firefox? Also, it seems strange that they
don't provide proper feedback.
There is no such specification for this, but the chromium project member finds this as a trivial issue and unlikely to be fixed in near future. Instead of they fixing in the chromium they suggest that it should be fixed on the server by sending proper HTTP status.
Response from Chromium Project Member: "This issue has been Available for over a year. If it's no longer
important or seems unlikely to be fixed, please consider closing it
out. If it is important, please re-triage the issue."
Sorry for the inconvenience if the bug really should have been left as
Available.
You can check more details here Issue 479265
What's happening beneath the surface?
I further checked the source code of the chromium to find what actually happening and found that for any non 200 status for downloads, they are simply throwing ERR_INVALID_RESPONSE (Invalid Server Response) error.
To cut a long story short, you have to live with this behaviour of the browser, it is not going to be improved.
Building on #lossleader's answer, it looks like Chromium decided to follow Firefox's decision to not download files if the response was not successful.
It seems like this issue has a history. In 2005 an AOL website had an issue that returned a status code 500 and resulted in users downloading an .exe file. There was a "fix" that simply returns a 404 for responses that trigger a download and with erroneous responses. The corresponding issue can be found here.
There is an open issue from 2008, that complains about this error and states that it would is misleading. The corresponding issue can be found here.
I found a more detailed answer about this on Super User.
I still think that it would be correct to at least offer a choice to the user to download the file nevertheless or at least show a more meaningful error page. On the other hand, in most cases a download for a response code != 2XX is unintended and hints a server error. Therefore it seems that this issue has a low priority for browser vendors and seems "not worth the trouble".
These answers all seem to bypass the fundamental here: You're trying to give a browser-specific interpretation to an error in your code. From my point of view, in all associated cases, your code is failing in some manner without error handling.
4xx error? You've sent a bad request to the server, according to rules you have determined. It's not, technically, the browser's fault.
5xx error? Your server crashed and didn't throw a pretty error. On some types of server, (Django) a 500 error will be a bunch of debug information you probably shouldn't show the user.
Thus what you're asking for is strange from an architectural standpoint; you want to cover up the fact that you've screwed up by modifying the browser's response rather than fixing your code to respond appropriately.
What are the good choice for API error code response pattern?
Instead of using different codes indicating different type of error
100001 // username not provided
100002 // password not provided
100003 // password too short
...
I see some other use patterns like the following (non-sequential) ...
20000
20001
20004
20015
Are there any other recommendations?
In my experience developing and using web services, I have found that a strategy of using a combination of top-level HTTP status codes and lower level API error codes work reasonably well. Note that the lower level API error codes don't need to be integers, but can be any enumeration. For a well-known public example, AWS Simple Email Service (SES) uses this strategy of using both HTTP status codes and API level error codes. You can see a sample error code response for SES here. Note that although SES uses XML response error payloads, this strategy works equally well for JSON response payloads.
In my experience, there are a few things that you need to keep in mind when using this strategy:
Strive to return the correct HTTP response code: HTTP is a ubiquitous protocol and is no doubt understood by your web container. Its response codes fit naturally into REST web services. As such, leverage it! If your web service encounters an error condition, you should do your best to return the correct HTTP status code in whose context, the API error code has meaning. One my biggest headaches in debugging issues with web services occur when developers just unconditionally throw arbitrary (usually runtime) exceptions back up the stack. The result is that everything gets returned back to the caller as an HTTP 500 (Internal Server Error) status code even when that's not the case (e.g. the client sends garbage data and the server just can't process it. Some common HTTP status codes you might want to design for include:
400 Bad Request: There is an issue with the client's request. Note this error isn't just used for things like broken JSON syntax in a POST request, but it is also a legitimate response code for semantic issues as well (i.e. the JSON request payload conformed to the prescribed schema, but there was an issue with the data in the payload, such as a number being negative when it is supposed to be only positive).
401 Unauthorized: The caller's credentials were invalid (i.e. authorization error).
403 Forbidden: The caller's credentials were valid, but their access level isn't sufficient to access the resource (i.e. authentication error).
404 Not Found: The resource of the URL doesn't exist.
500 Internal Server Error: Something bad happened inside the server itself, this error could be anything.
502 Bad Gateway: An error occurred when calling downstream service.
503 Service Unavailable: A useful response code for when you get hammered with a ton of "happy" customers who are inadvertently DDOS'ing your service.
504 Gateway Timeout: Like the 502 status code, but indicates a timeout instead of an actual error with the downstream service, per se.
HTTP response codes are the top-level codes, and API error codes only have meaning within that context: By this, I mean that your API error codes are only meaningful for certain HTTP response codes. For example, in the table of SES error codes, each error code is only tied to a single HTTP(S) response code. The error codes ConfigurationSetDoesNotExist and InvalidParameterValue only make sense when a 400 Bad Request is returned by SES - it wouldn't make sense to return these status codes when a 500 Internal Server Error is returned. Similarly, if you were writing a web service that called downstream services and databases, you might have a FooDownstreamServiceTimedOut error code that you would return with a 504 Gateway Timeout HTTP status code when a downstream web service call timed out to the "Foo" web service. You might also have a MyDatabaseError error code that you would return with a 500 Internal Server Error HTTP status code when your query to the internal DB fails.
Have a uniform error code schema irrespective of status codes: Your clients need to be able to process your error content programmatically. As such, it needs to conform to a certain schema. Ideally, your API error code schema should include the error code (i.e. name or ID, etc.). You also probably want to include a natural language description of the error code and the ID/GUID of the request that you are responding to. For an example of an error schema, see this sample AWS SES response and schema. Additionally, you might also want to consider returning a client ID in the response. This is as much for your own benefit as the client's since it can help you drill down into the data to see if one particular client is getting a glut of particular errors vs. your other clients.
Consider returning natural language descriptions of the error codes in the response: To make things easier on your clients, you might want to consider not just returning the error code in the error payload, but a natural language description as well. This kind of behavior can immediately help confused and busy engineers who really don't care that much about your service quickly diagnose what's happening so that they can resolve the issue ASAP. btw, enabling engineers to quickly diagnose issues with your service increases the all-important "uptime" metric that your customers and managers will no doubt care about.
Don't feel obliged to use integers, use enumerations instead: The notion of "error codes" conjures up images of outdated technologies and codebooks where you had to look up what an error meant. It arose from the programming dark ages when engineers needed to fit all possible errors into a byte of space, or a nibble or whatever. Those days are gone, and your error code can be a string, likely without any meaningful impact on performance. You might as well take advantage and make the error code meaningful, as a means of keeping things simple.
Return info to clients that they might need to debug, but be mindful of security: If possible, return whatever debug info your clients may need. However, if your service potentially deals with sensitive information such as credit card numbers and the like, you probably don't want to pass that info around for obvious reasons.
Hope that helps.
A recommendation by the IETF (internet standards body) is using the application/problem+json mediatype.
Notable is that they don't use random numbers, they use strings (specifically uris) to identify errors.
This is a subjective question, but even if you don't use their format, I'd argue that username-not-provided is better in almost every way to 100001.
I would say this heavily depends on what kind of API you're providing.
I were to always include a field called ack or something similar in every response that has three states: failure, warning, success. Success obviously being everything went well. On warning, the request went through and the JSON will contain the expected output, but it will also include a warning string, or even better in case multiple warnings could occur an array called errors which consists of multiple objects containg code, string and type. This array will also be returned in case of failure, and nothing else but this array.
The array contains one object per error or warning, having a code (I would suggest going with your initial idea of 10001, 10002, ...) and a string explaining the error in a very short phrase (e.g. Username contains invalid characters). The type is either error or warning, which is useful in case of a failure ack that contains not only errors but also warnings.
This makes it easy to look up errors by their code (I would provide a page, also with an API, that contains all the error codes in a table along with their short and long description plus common causes/fixes/etc. - All this information should also be available via an API where they can be accessed by providing the error code) while still having a quick short text response so the user can tell what's wrong in most cases without having to look up the error.
This also allows for easy output of warnings and errors to the end user, not just the developers. Using my idea with the API call to get informations about an error, developers using your API could easily provide full information about errors to end-users when needed (including causes/fixes/whatever you see fit).
Instead of writing your own API standard from scratch adopt one of the already available, for example the JSON API standard:
If you’ve ever argued with your team about the way your JSON responses should be formatted, JSON API can be your anti-bikeshedding tool.
By following shared conventions, you can increase productivity, take advantage of generalized tooling, and focus on what matters: your application.
Clients built around JSON API are able to take advantage of its features around efficiently caching responses, sometimes eliminating network requests entirely.
If you decide to go with JSON API it has a section dedicated to errors and a few error examples.
For many years, many developent companies have created things like bitmask for errors, so they can encode multiple variables inside the error:
000 - all ok
001 - something failed with X
010 - something failed with Y
011 - something failed with X and Y
100 - something failed with Z
101 - something failed with X and Z
The limitation is that that limits the error space into however many bytes you decide on the encoding, like 16 or 32 possible combinations, it may be enough for you, or not.
You see this being common in COM+
https://learn.microsoft.com/en-us/windows/desktop/com/com-error-codes-1
I hope this helps.
I have started learning about NServiceBus to determine whether we can use NServiceBus the way we want.
When I configured NServiceBus to use the JsonSerializer for serialization, I expected to be able to read the contents of the messages directly in the queue. But instead of being JSON formatted text, the body is hexadecmial. We would like to be able to open up a queue/table and view the contents of the messages without needing to convert every message from hex to text. Is this possible?
PS.: I am using NServiceBus v 5.2.14 and NServiceBus.Host v 6.0.0
There are several options:
MSMQ
For MSMQ I honestly believe the best option would be to use ServiceInsight. It gives you much more than just viewing the message payload but if you're in a position that you need to know why your system is behaving the way it is, this is the best option.
Regarding native tools for MQMS there are several options http://docs.particular.net/nservicebus/msmq/viewing-message-content-in-msmq
SQL Server Transport
While ServiceInsight works across all the transports, if you are using the SQL Transport and you just want to see content of your messages you can run a simple SQL query query:
SELECT CONVERT(VARCHAR(max), [Body]) FROM [YourQueue]
Having said that, you are technically converting message to text through this query.