My question concerns 0x7E8 response messages where the message is greater than 8 bytes.
I am sure OBD2 experts won't need much of the detail below but for the sake of clarity......
When a diagnostic device (eg scantool) sends a 0x7DF message with a Mode and PID value, most 0x7E8 responses consist of a single message of up to 8 bytes.
However some response messages require more than 8 bytes. eg request for the VIN and many vehicle manufacturer specific Mode/PID values. In this case ISO 15765-2 specifies that the first response 0x7E8 message has the code 0x10, Number of bytes, Mode, PID followed by data.
The diagnostic device then responds with a 0x7E0 flow control message according to the ISO 15765-2 spec. Noting that there is no information in that flow control message to identify which 0x7E8 response message is being flow controlled.
When the flow control message is received, the responding device then sends further 0x7E8 messages with the 0x2? code where "?" is the sequence number of the message. Again there is no information is those further 0x7E8 messages to identify which 0x7DF message Mode/PID is being responded to.
So my question is - Is it possible to have 2 or more diagnostic devices on an OBD2 bus sending 0x7DF messages where the response is more than 8 bytes?
As there is no information in either the 0x7E0 flow control message or the following 0x7E8 0x2? coded messages to identify the original 0x7DF request, multiple diagnostic devices on the OBD2 bus will not be able to identify which 0x7E8 response messages correspond to their request.
I have seen this 0x7E8 message confusion happen in practice with more than one scantool connected to a vehicle. But maybe I have missed something?
If you have more than one scan tool on the bus, then you need to take special care about orchestration, i.e. only the tester who initiated a question is allowed to send flow control ACKs.
Related
What are the good choice for API error code response pattern?
Instead of using different codes indicating different type of error
100001 // username not provided
100002 // password not provided
100003 // password too short
...
I see some other use patterns like the following (non-sequential) ...
20000
20001
20004
20015
Are there any other recommendations?
In my experience developing and using web services, I have found that a strategy of using a combination of top-level HTTP status codes and lower level API error codes work reasonably well. Note that the lower level API error codes don't need to be integers, but can be any enumeration. For a well-known public example, AWS Simple Email Service (SES) uses this strategy of using both HTTP status codes and API level error codes. You can see a sample error code response for SES here. Note that although SES uses XML response error payloads, this strategy works equally well for JSON response payloads.
In my experience, there are a few things that you need to keep in mind when using this strategy:
Strive to return the correct HTTP response code: HTTP is a ubiquitous protocol and is no doubt understood by your web container. Its response codes fit naturally into REST web services. As such, leverage it! If your web service encounters an error condition, you should do your best to return the correct HTTP status code in whose context, the API error code has meaning. One my biggest headaches in debugging issues with web services occur when developers just unconditionally throw arbitrary (usually runtime) exceptions back up the stack. The result is that everything gets returned back to the caller as an HTTP 500 (Internal Server Error) status code even when that's not the case (e.g. the client sends garbage data and the server just can't process it. Some common HTTP status codes you might want to design for include:
400 Bad Request: There is an issue with the client's request. Note this error isn't just used for things like broken JSON syntax in a POST request, but it is also a legitimate response code for semantic issues as well (i.e. the JSON request payload conformed to the prescribed schema, but there was an issue with the data in the payload, such as a number being negative when it is supposed to be only positive).
401 Unauthorized: The caller's credentials were invalid (i.e. authorization error).
403 Forbidden: The caller's credentials were valid, but their access level isn't sufficient to access the resource (i.e. authentication error).
404 Not Found: The resource of the URL doesn't exist.
500 Internal Server Error: Something bad happened inside the server itself, this error could be anything.
502 Bad Gateway: An error occurred when calling downstream service.
503 Service Unavailable: A useful response code for when you get hammered with a ton of "happy" customers who are inadvertently DDOS'ing your service.
504 Gateway Timeout: Like the 502 status code, but indicates a timeout instead of an actual error with the downstream service, per se.
HTTP response codes are the top-level codes, and API error codes only have meaning within that context: By this, I mean that your API error codes are only meaningful for certain HTTP response codes. For example, in the table of SES error codes, each error code is only tied to a single HTTP(S) response code. The error codes ConfigurationSetDoesNotExist and InvalidParameterValue only make sense when a 400 Bad Request is returned by SES - it wouldn't make sense to return these status codes when a 500 Internal Server Error is returned. Similarly, if you were writing a web service that called downstream services and databases, you might have a FooDownstreamServiceTimedOut error code that you would return with a 504 Gateway Timeout HTTP status code when a downstream web service call timed out to the "Foo" web service. You might also have a MyDatabaseError error code that you would return with a 500 Internal Server Error HTTP status code when your query to the internal DB fails.
Have a uniform error code schema irrespective of status codes: Your clients need to be able to process your error content programmatically. As such, it needs to conform to a certain schema. Ideally, your API error code schema should include the error code (i.e. name or ID, etc.). You also probably want to include a natural language description of the error code and the ID/GUID of the request that you are responding to. For an example of an error schema, see this sample AWS SES response and schema. Additionally, you might also want to consider returning a client ID in the response. This is as much for your own benefit as the client's since it can help you drill down into the data to see if one particular client is getting a glut of particular errors vs. your other clients.
Consider returning natural language descriptions of the error codes in the response: To make things easier on your clients, you might want to consider not just returning the error code in the error payload, but a natural language description as well. This kind of behavior can immediately help confused and busy engineers who really don't care that much about your service quickly diagnose what's happening so that they can resolve the issue ASAP. btw, enabling engineers to quickly diagnose issues with your service increases the all-important "uptime" metric that your customers and managers will no doubt care about.
Don't feel obliged to use integers, use enumerations instead: The notion of "error codes" conjures up images of outdated technologies and codebooks where you had to look up what an error meant. It arose from the programming dark ages when engineers needed to fit all possible errors into a byte of space, or a nibble or whatever. Those days are gone, and your error code can be a string, likely without any meaningful impact on performance. You might as well take advantage and make the error code meaningful, as a means of keeping things simple.
Return info to clients that they might need to debug, but be mindful of security: If possible, return whatever debug info your clients may need. However, if your service potentially deals with sensitive information such as credit card numbers and the like, you probably don't want to pass that info around for obvious reasons.
Hope that helps.
A recommendation by the IETF (internet standards body) is using the application/problem+json mediatype.
Notable is that they don't use random numbers, they use strings (specifically uris) to identify errors.
This is a subjective question, but even if you don't use their format, I'd argue that username-not-provided is better in almost every way to 100001.
I would say this heavily depends on what kind of API you're providing.
I were to always include a field called ack or something similar in every response that has three states: failure, warning, success. Success obviously being everything went well. On warning, the request went through and the JSON will contain the expected output, but it will also include a warning string, or even better in case multiple warnings could occur an array called errors which consists of multiple objects containg code, string and type. This array will also be returned in case of failure, and nothing else but this array.
The array contains one object per error or warning, having a code (I would suggest going with your initial idea of 10001, 10002, ...) and a string explaining the error in a very short phrase (e.g. Username contains invalid characters). The type is either error or warning, which is useful in case of a failure ack that contains not only errors but also warnings.
This makes it easy to look up errors by their code (I would provide a page, also with an API, that contains all the error codes in a table along with their short and long description plus common causes/fixes/etc. - All this information should also be available via an API where they can be accessed by providing the error code) while still having a quick short text response so the user can tell what's wrong in most cases without having to look up the error.
This also allows for easy output of warnings and errors to the end user, not just the developers. Using my idea with the API call to get informations about an error, developers using your API could easily provide full information about errors to end-users when needed (including causes/fixes/whatever you see fit).
Instead of writing your own API standard from scratch adopt one of the already available, for example the JSON API standard:
If you’ve ever argued with your team about the way your JSON responses should be formatted, JSON API can be your anti-bikeshedding tool.
By following shared conventions, you can increase productivity, take advantage of generalized tooling, and focus on what matters: your application.
Clients built around JSON API are able to take advantage of its features around efficiently caching responses, sometimes eliminating network requests entirely.
If you decide to go with JSON API it has a section dedicated to errors and a few error examples.
For many years, many developent companies have created things like bitmask for errors, so they can encode multiple variables inside the error:
000 - all ok
001 - something failed with X
010 - something failed with Y
011 - something failed with X and Y
100 - something failed with Z
101 - something failed with X and Z
The limitation is that that limits the error space into however many bytes you decide on the encoding, like 16 or 32 possible combinations, it may be enough for you, or not.
You see this being common in COM+
https://learn.microsoft.com/en-us/windows/desktop/com/com-error-codes-1
I hope this helps.
We had a discussion today about an transfer operation resulting in status code 200, OK. There were two objects returned looking like this.
First one being fairly graspable (and following the expected contract).
{ name: "john", age: 34, city: "stockholm" }
Second one, following the contract but with unquestionably wrong data.
{ name: null, age: -3.141526, city: "http://some.com/address/poof" }
One party claimed that the status code 200 is incorrect because the values are wrong. The other side argued that the status code describes the operation as such and the format of the request/response, which went well because the transfer agrees with the contract.
It's fairly obvious that the REST endpoint gets an exception from the sources it fetches the data from. And so, the first party wanted the result to be either 404 not found or 500 internal error. The other side was open to it under the condition that the object structure is empty (nulls all the way) in the former case and that it doesn't attempt to follow the agreed format in the latter case.
Checking out the Kamasutra it's said that:
The request has succeeded. The information returned with the response is dependent on the method used in the request.
Now, technically speaking, we can't know for sure if the resource requested has a name, might be planned to be born in PI years and happens to reside in a city that changed its name to an URL. That is actually possible, although highly unlikely. However, I'd like to see an explicit statement of what isn't included in status code 200.
The question: is it valid to demand status code 400 or higher because the values are seemingly (or even obviously) wrong?
Don't use the RFC 2616
The RFC 2616 is completely irrelevant nowadays once it has been replaced by a set of new RFCs that together define the HTTP/1.1 protocol:
RFC 7230: HTTP/1.1: Message Syntax and Routing
RFC 7231: HTTP/1.1: Semantics and Content
RFC 7232: HTTP/1.1: Conditional Requests
RFC 7233: HTTP/1.1: Range Requests
RFC 7234: HTTP/1.1: Caching
RFC 7235: HTTP/1.1: Authentication
Status codes
For the HTTP status codes, refer to the RFC 7231. Such document defines what each status code indicates. Pick the one that best gives the result of the attempt to understand and satisfy the request.
This document also defines the classes of the status codes, that helps to determine the most suitable status for the response:
The first digit of the status-code defines the class of response. The last two digits do not have any categorization role. There are five values for the first digit:
1xx (Informational): The request was received, continuing process
2xx (Successful): The request was successfully received,
understood, and accepted
3xx (Redirection): Further action needs to be taken in order to
complete the request
4xx (Client Error): The request contains bad syntax or cannot be
fulfilled
5xx (Server Error): The server failed to fulfill an apparently
valid request
Just bear in mind that HTTP status codes are extensible. The RFC 7231 does not include extension status codes defined in other specifications. The complete list of status codes is maintained by IANA.
Unprocessable entity
The 2xx class of status code indicates that request was successfully received, understood, and accepted. Once you won't accept invalid data, that is, an entity that cannot be processed by the server, the 200 status code is not suitable for the this situation.
The 422 status code is what you are looking for: the syntax of the request payload is valid but it cannot be processed due to invalid data. Have a look:
11.2. 422 Unprocessable Entity
The 422 (Unprocessable Entity) status code means the server
understands the content type of the request entity (hence a
415 (Unsupported Media Type) status code is inappropriate), and the
syntax of the request entity is correct (thus a 400 (Bad Request)
status code is inappropriate) but was unable to process the contained
instructions. For example, this error condition may occur if an XML
request body contains well-formed (i.e., syntactically correct), but
semantically erroneous, XML instructions.
For your situation, just read JSON instead of XML.
The 422 is registered in IANA and defined in the RFC 4918, the document that defines WebDAV, an extension for the HTTP protocol.
Decision charts
Michael Kropat put together a set of decision charts that helps determine the best status code for each situation. The status codes are grouped into three rough categories:
Start here:
Choosing 2xx and 3xx status codes:
Choosing 4xx status codes:
Choosing 5xx status codes:
I am designing a RESTful API and am using the usual HTTP status codes like 200, 404, 500, etc.
Given the situation where occasionally the API will require additional data. When this event arises it needs to respond with a relevant status code so that the client application can present the relevant user interface.
Example:
User fills in form and hits submit (using AJAX)
Server sends response with some status code to indicate that further information is required which is accompanied with additional user interface.
Additional user interface is displayed
User resubmits form
I have this working with a 200 status code, but I would prefer to use a different status code so that my AJAX handler can detect and handle this scenario automatically.
Are there any relevant status codes or would I need to use a sub status code which is returned as a part of my JSON response?
I would go with a 400 Bad Request, since there is more data needed.
Alternatively, a 100 Continue could also work.
Either of those codes indicates that more info is needed.
how about 202 Accepted
"The request has been accepted for processing, but the processing has not been completed"
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
I would suggest 412 Precondition Failed.
How are the messages encoded or sent/received by peers?
If there is a message
have: <len=0005><id=4><piece index>
How is this sent(in binary,how is it translated to binary?) and received?
Is there a specific order in which the messages are sent to peers?
I have read the specification but it leaves me with questions.
Thanks
I'll answer the ordering question.
In general, you can send any message at any time. But there are some messages which have special rules. The BITFIELD message has to be sent out early for instance. Most clients send PIECEs back in the order they were REQUESTed, but I don't think that is a requirement if memory serves.
In general the messages are of two types. One kind are control-oriented messages telling peers about general status (HAVE messages falls into this group). The other kind are data-oriented messages that actually transfers the file and requests new data from the peer. These message types are "interleaved" and one of the reasons you send PIECE messages no larger than 16 kilobytes is to make sure control messages can be interleaved in between. A trick is that when a PIECE message has been sent, then send all control-oriented messages by priority before the next PIECE message. That way, you quickly tell the other party of your intent.
There is also a "bug" in the original protocol which is solved by the FAST extension. It effectively make each REQUEST result in either a PIECE message or a REJECT-REQUEST message. This is another example of an ordering. If you get a REJECT-REQUEST message for something you never REQUESTED you disconnect the peer.
Prior to declaring the have message the specification says:
All of the remaining messages in the protocol take the form of <length prefix><message ID><payload>. The length prefix is a four byte big-endian value. The message ID is a single decimal byte. The payload is message dependent.
You've got the binary format for length and id right there. The 'piece index' part is this message's specific payload. It should be four bytes long since the message has a fixed size of 5 bytes and 1 byte went to the message ID (viewing other messages with the same format should give you a clue).
We are using NService bus for our messaging framework.Sometime the message is not coming as par the sequence of sending .Sometimes last message is coming first and than later first message.
Please help me out Thanks
The nature of NServiceBus does not guarantee that messages will be received in the order they were sent. Each message is meant to be processed independently.
If an action can only be undertaken after two related messages arrive, then you need to utilize a Saga
Edit in response to first comment:
You mention you're sending the same message in chunks. Does this mean that you have a large payload that you have to split up into multiple parts to transmit via MSMQ?
If so, you have a few options:
Store the payload out of band, in a database or file system, and only put enough data in one message (an ID or file system path) to load the data from the message handler.
Make the message a MessagePart that contains a BundleID, PartNumber, TotalParts, and PayloadChunk. Then, create a saga for MessagePart that stores each part and when all parts have been received, reconstitute the chunks together and do what you need. Of course, if you need to then send the resulting large object back onto the Bus, this would get annoying really quickly, so then the out-of-band option would look much more attractive.
In any case, there are a ton of reasons why any MSMQ message, not just NServiceBus messages, could arrive out of order, so you have to be able to deal with it.
Would Bus.Sending a collection of Imessages work? NServiceBus allows batching of messages