A Cloud Function triggered by an HTTP request has a corresponding function-execution-id for each calling request (in the request and response header). It is used for tracing and viewing the log of a specific request in Stack Driver Logging. In my case, it is a string of 12 characters. When I continuously do HTTP requests to a cloud function and see the function-execution-id, I get the result below:
j8dorcyxyrwb
j8do4wolg4i3
j8do8bxu260m
j8do2xhqmr3s
j8dozkdlrjzp
j8doitxtpt29
j8dow25ri4on
On each line, the first 4 characters are the same "j8do" but the rest are different, so I wonder what is the structure of function-execution-id.
How was it generated?
The execution id is opaque, meaning that it doesn't contain any useful data. It is just a unique ID. How it was generated should not be of any issue to you, the consumer. From examination, it looks like it might be some time-based value similar to UUIDv1, but any code that you write that consumes these IDs should make no assumptions about how they were generated.
Related
What are the good choice for API error code response pattern?
Instead of using different codes indicating different type of error
100001 // username not provided
100002 // password not provided
100003 // password too short
...
I see some other use patterns like the following (non-sequential) ...
20000
20001
20004
20015
Are there any other recommendations?
In my experience developing and using web services, I have found that a strategy of using a combination of top-level HTTP status codes and lower level API error codes work reasonably well. Note that the lower level API error codes don't need to be integers, but can be any enumeration. For a well-known public example, AWS Simple Email Service (SES) uses this strategy of using both HTTP status codes and API level error codes. You can see a sample error code response for SES here. Note that although SES uses XML response error payloads, this strategy works equally well for JSON response payloads.
In my experience, there are a few things that you need to keep in mind when using this strategy:
Strive to return the correct HTTP response code: HTTP is a ubiquitous protocol and is no doubt understood by your web container. Its response codes fit naturally into REST web services. As such, leverage it! If your web service encounters an error condition, you should do your best to return the correct HTTP status code in whose context, the API error code has meaning. One my biggest headaches in debugging issues with web services occur when developers just unconditionally throw arbitrary (usually runtime) exceptions back up the stack. The result is that everything gets returned back to the caller as an HTTP 500 (Internal Server Error) status code even when that's not the case (e.g. the client sends garbage data and the server just can't process it. Some common HTTP status codes you might want to design for include:
400 Bad Request: There is an issue with the client's request. Note this error isn't just used for things like broken JSON syntax in a POST request, but it is also a legitimate response code for semantic issues as well (i.e. the JSON request payload conformed to the prescribed schema, but there was an issue with the data in the payload, such as a number being negative when it is supposed to be only positive).
401 Unauthorized: The caller's credentials were invalid (i.e. authorization error).
403 Forbidden: The caller's credentials were valid, but their access level isn't sufficient to access the resource (i.e. authentication error).
404 Not Found: The resource of the URL doesn't exist.
500 Internal Server Error: Something bad happened inside the server itself, this error could be anything.
502 Bad Gateway: An error occurred when calling downstream service.
503 Service Unavailable: A useful response code for when you get hammered with a ton of "happy" customers who are inadvertently DDOS'ing your service.
504 Gateway Timeout: Like the 502 status code, but indicates a timeout instead of an actual error with the downstream service, per se.
HTTP response codes are the top-level codes, and API error codes only have meaning within that context: By this, I mean that your API error codes are only meaningful for certain HTTP response codes. For example, in the table of SES error codes, each error code is only tied to a single HTTP(S) response code. The error codes ConfigurationSetDoesNotExist and InvalidParameterValue only make sense when a 400 Bad Request is returned by SES - it wouldn't make sense to return these status codes when a 500 Internal Server Error is returned. Similarly, if you were writing a web service that called downstream services and databases, you might have a FooDownstreamServiceTimedOut error code that you would return with a 504 Gateway Timeout HTTP status code when a downstream web service call timed out to the "Foo" web service. You might also have a MyDatabaseError error code that you would return with a 500 Internal Server Error HTTP status code when your query to the internal DB fails.
Have a uniform error code schema irrespective of status codes: Your clients need to be able to process your error content programmatically. As such, it needs to conform to a certain schema. Ideally, your API error code schema should include the error code (i.e. name or ID, etc.). You also probably want to include a natural language description of the error code and the ID/GUID of the request that you are responding to. For an example of an error schema, see this sample AWS SES response and schema. Additionally, you might also want to consider returning a client ID in the response. This is as much for your own benefit as the client's since it can help you drill down into the data to see if one particular client is getting a glut of particular errors vs. your other clients.
Consider returning natural language descriptions of the error codes in the response: To make things easier on your clients, you might want to consider not just returning the error code in the error payload, but a natural language description as well. This kind of behavior can immediately help confused and busy engineers who really don't care that much about your service quickly diagnose what's happening so that they can resolve the issue ASAP. btw, enabling engineers to quickly diagnose issues with your service increases the all-important "uptime" metric that your customers and managers will no doubt care about.
Don't feel obliged to use integers, use enumerations instead: The notion of "error codes" conjures up images of outdated technologies and codebooks where you had to look up what an error meant. It arose from the programming dark ages when engineers needed to fit all possible errors into a byte of space, or a nibble or whatever. Those days are gone, and your error code can be a string, likely without any meaningful impact on performance. You might as well take advantage and make the error code meaningful, as a means of keeping things simple.
Return info to clients that they might need to debug, but be mindful of security: If possible, return whatever debug info your clients may need. However, if your service potentially deals with sensitive information such as credit card numbers and the like, you probably don't want to pass that info around for obvious reasons.
Hope that helps.
A recommendation by the IETF (internet standards body) is using the application/problem+json mediatype.
Notable is that they don't use random numbers, they use strings (specifically uris) to identify errors.
This is a subjective question, but even if you don't use their format, I'd argue that username-not-provided is better in almost every way to 100001.
I would say this heavily depends on what kind of API you're providing.
I were to always include a field called ack or something similar in every response that has three states: failure, warning, success. Success obviously being everything went well. On warning, the request went through and the JSON will contain the expected output, but it will also include a warning string, or even better in case multiple warnings could occur an array called errors which consists of multiple objects containg code, string and type. This array will also be returned in case of failure, and nothing else but this array.
The array contains one object per error or warning, having a code (I would suggest going with your initial idea of 10001, 10002, ...) and a string explaining the error in a very short phrase (e.g. Username contains invalid characters). The type is either error or warning, which is useful in case of a failure ack that contains not only errors but also warnings.
This makes it easy to look up errors by their code (I would provide a page, also with an API, that contains all the error codes in a table along with their short and long description plus common causes/fixes/etc. - All this information should also be available via an API where they can be accessed by providing the error code) while still having a quick short text response so the user can tell what's wrong in most cases without having to look up the error.
This also allows for easy output of warnings and errors to the end user, not just the developers. Using my idea with the API call to get informations about an error, developers using your API could easily provide full information about errors to end-users when needed (including causes/fixes/whatever you see fit).
Instead of writing your own API standard from scratch adopt one of the already available, for example the JSON API standard:
If you’ve ever argued with your team about the way your JSON responses should be formatted, JSON API can be your anti-bikeshedding tool.
By following shared conventions, you can increase productivity, take advantage of generalized tooling, and focus on what matters: your application.
Clients built around JSON API are able to take advantage of its features around efficiently caching responses, sometimes eliminating network requests entirely.
If you decide to go with JSON API it has a section dedicated to errors and a few error examples.
For many years, many developent companies have created things like bitmask for errors, so they can encode multiple variables inside the error:
000 - all ok
001 - something failed with X
010 - something failed with Y
011 - something failed with X and Y
100 - something failed with Z
101 - something failed with X and Z
The limitation is that that limits the error space into however many bytes you decide on the encoding, like 16 or 32 possible combinations, it may be enough for you, or not.
You see this being common in COM+
https://learn.microsoft.com/en-us/windows/desktop/com/com-error-codes-1
I hope this helps.
I have a doubt. When is a GET request sent. I mean, I have seen a lot of people using if request.method == 'GET', when they render the form for the first time, but when the form is submitted, they do a `POST' request.
While they explicitly mention when defining the form in html that the method will be 'POST', they don't do the same for 'GET' request which is made when an empty form is requested.
How does django know it's a GET request?
And, why is it done so?
Thanks,
I'm not an expert but I think Django "knows" this because like all things on the Internet it uses the HTTP protocol. There are several HTTP methods. If not specified the default method will always be GET
GET
A GET is usually used to retrieve information. Typically a GET function has no side-effects (this means that data in the database is not changed, that no files in the filesystem are modified, etc.).
Strictly speaking this is not always true, since some webservers log requests (themselves) and thus add an entry to the database that a specific user visited a specific page at a specific timestamp, etc.
A typical GET request is idempotent. This means that there is no difference between doing a query one time, or multiple times (two times, three times, five times, thousand times).
GET queries are therefore typically used to provide static content, as well as pages that contain data about one or more entries, search queries, etc.
POST
POST on the other hand typically ships with data (in the POST parameters), and usually the idea is that something is done with this data that creates a change in the persistent structures of the webserver. For example creating a new entry in some table, or updating the table with the values that are provided. Since these operations are not always idempotent, it can be dangerous if the user refreshes the page in the browser (since that could for example create two orders, instead of the single order the user actually wanted to create).
Therefore in Django a POST request will typically result in some changes to the database, and a redirect as result. This means that the user will typically obtain a new address, and perform a GET request on that page (and that GET is idempotent, hence it will not construct a new order).
PUT, PATCH and DELETE
Besides the popular GET and POST, there are other typical requests a client can make to a webserver. For example PUT, PATCH and DELETE.
PUT
PUT is the twin of a POST request. The main difference is that the URI it hits, specifies what entry to construct or update. PUT is usually an idempotent operation.
This means that if we would for example perform a POST server.com/blog/create to create a blog, PUT will typically look like PUT server.com/blog/123/. So we specify the id in advance. In case the object does not yet exists, a webserver will typically construct one. In case the entity already exists, a new entity will typically be constructed for that URI. So performing the same PUT operation twice, should have no effect.
Note that in case of a PUT request, typically one should specify all fields. The fields that are not specified will typically be filled in with default values (in case such values exist). We thus do not really "update" the entity: we destroy the old entity and create a new one in case that entity already exists.
PATCH
PATCH is a variant of PUT that updates the entity, instead of creating a new one. The fields that are thus missing in a PATCH request, typically remain the same as the values in the "old" entity so to speak.
DELETE
Like the name already suggests, if we perform a DELETE server.com/blog/123/ request, then we will typically remove the corresponding element.
Some servers do not immediately remove the corresponding element. You can see it as scheduling the object for deletion, so sometimes the object is later removed. The DELETE request, thus typically means that you signal the server to eventually remove the entity.
Actually Django is based on HTTP responses-requests. HTTP is fully textuall. So Django parses each request and finds in its header information about what kind of request is it. I may be mistaken in details, but as I understand when server receives request - Django creates it's object request, which contains all the data from HTTP. And then you decide if you need a specific action on GET or POST and you check the type of request with request.method.
P.S. And yes, by default each request is GET.
Please forgive my ignorance as I'm a jmeter noob. My webservice accepts JSON objects so I was able to write a rudimentary test where I create an HTTP Request with a JSON object in the "Post Body" portion of the http request.
Anyway, what I want to do is have the HTTP Request choose a different JSON object from a csv file or some other input mechanism so that I can randomize the types of queries that are being run during the load test. Is there a way to do this? The closest is probably using variables (section 4.11 in the user manual) but I have a feeling that's not how variables are used.
A second way I've theorized (although I haven't tried yet since I think the method above is easier) is to create a HTTP Request Default obj with a bunch of HTTP Requests with different JSON objects in them and then use a Random Controller to randomly go thru my multiple HTTP Requests on each pass.
If there's a third way, I'm all ears to learn how to use this tool. I'll continue to read and possibly experiment with plan B above. Thanks in advance for any help you can give me.
UPDATE: So I tried the second way and it seems to work. I had 3 different HTTP requests and the number of times each request gets hit varies from run to run. I still invite answers from the community since I'd like to see what the pros do for issues similar to mine.
You have partially answered your question yourself, by saying "csv file or". Here are the specifics.
You will have to use CSV data set config in your test plan to read data from CSV. In your post body, use the variables read from CSV.
Here is a screen cast showing how to use csv data set config.
So, in playframework, I can stream any response back so when i get a json request, I can do http chunking and stream the response back for some really really large responses. I was wondering if the same can be done on POST calls. If a client has a very very large POST call, can they stream me the request? Is this possible with html?
That said, if I can't do that, i need an api that curl or some other non-browser client will use to upload a file(the json request, or a csv, etc). How to create such an api?
I should note that I canNOT receive the whole request at once or will get out of memory. I need to receive pieces and as I receive pieces put that to the backend datastore a piece at a time.
Also, what would be the curl syntax to make sure it is streaming the file rather than sending it in one huge huge request that would break the server? How to force the client to stream the file in?
thanks,
Dean
You can get full control over HTTP request processing by using an EssentialAction. An EssentialAction processes the request body and returns a Result.
Normal Play Actions are a special case of EssentialAction. Actions process request bodies and return Results too, but they always perform their processing in two steps. Actions first parse the request body. Then the Actions parsethe parsed value to a function to get a Result. For you, having a separate parsing step is a problem because it means that the parsed value needs to be stored in memory.
If you use an EssentialAction then you can avoid storing a parsed value in memory, because you can just process the request body as it arrives.
What you need to do is add a method to your controller that returns an EssentialAction.
The signature of EssentialAction is:
trait EssentialAction extends (RequestHeader) ⇒ Iteratee[Array[Byte], SimpleResult]
The EssentialAction needs to accept the request header and then return an iteratee to process the request body. The iteratee will incrementally process the request body as it arrives. You can use your iteratee to put each piece into your data store as each piece arrives. When you're done processing all the pieces you can return a Result.
More info here: http://www.playframework.com/documentation/2.2.x/HttpApi.
I know differences and advantages of each command, my question is could I replace POST requests with GET everywhere? And which these commands calls by default while sending request from html form?
could I replace POST requests with GET everywhere
No (and it would be a terrible idea to try).
Things that a form can do with POST that you can't do with GET include:
Sending lots of data
Sending files
There are other things that would simply be stupid to do with GET.
From http://www.w3.org/TR/html5/forms.html#attr-fs-method :
The method and formmethod content attributes are enumerated attributes
with the following keywords and states:
The keyword get, mapping to the state GET, indicating the HTTP GET
method. The keyword post, mapping to the state POST, indicating the
HTTP POST method. The invalid value default for these attributes is
the GET state. (There is no missing value default.)
When using GET to transfer data from the client to the server, the data is added to the URL, there is not BODY of the request. There is usually a limit on how long a URL can be, in the old days this was 1024 characters but that really depends on the server software and server middleware and even the browser.
This means if you want to transfer loads or data or upload a file to the server, you can not do it with GET.