I used the following haproxy configuration in frontend to modify the response header of requests depending on a query string:
frontend my-frontend
acl is-foo urlp(foo) 1
http-response replace-header Set-Cookie "(.*)" "\1; SameSite=None" if is-foo
Depending on my information from the docs the acl should match for all requests like
example.com?a=b&foo=1&bar=2
example.com?foo=1
example.com?a=b&foo=1
And it should not match for requests like
example.com?a=b&foo=0&bar=2
example.com?a=b
example.com?a=b&foo=bar
The actual result is that the acl matches never.
If i invert the if i.e.: if !is-foo the replace-header happens on every request.
So the problem must be the acl which matches never.
I use haproxy 2.0.15
I got it working by myself.
It seems to be the case that urlp(foo) is not present at runtime when it has been executed for http-response.
So we need to store its value in a temporary variable using set-var(custom.name), before. At runtime in if condition we can access it with var(custom.name) and match it against our condition. I used urlp_val() instead of urlp() here because the value will be casted to int immediately.
frontend my-frontend
http-request set-var(txn.foo) urlp_val(foo)
http-response replace-header Set-Cookie "(.*)" "\1; SameSite=None" if { var(txn.foo) eq 1 }
Thank you for traveling.
Related
I'm using Apache JMeter 3.2 r1790748 on Mac.
I have a setUp Thread Group making an authentication call. The call works and outputs the tokens correctly. Now I need to pass that token to the HTTP Header Manager for all the calls I'm making.
First of all, here's my token json output:
{
"access_token": "aaaaaa555555555",
"token_type": "Access",
"user_id": "5555"
}
Here's what my HTTP Header manager looks like:
1 value: Authorization : Bearer ${access_token}
My network call:
GET https://my_server.com/some_path
GET data:
[no cookies]
Request Headers:
Connection: close
Authorization: Bearer ${access_token}
Host: my_server.com
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_91)
As you can see, the variable access_token is not being replaced with the value from the setup call.
What I've tried:
BeanShell PostProcessor:
I created this script, and it actually parses and outputs the access_token properly:
import org.apache.jmeter.protocol.http.control.Header;
import net.minidev.json.JSONObject;
import net.minidev.json.parser.JSONParser;
String jsonString = prev.getResponseDataAsString();
log.info("jsonString = " + jsonString);
JSONParser parser = new JSONParser(JSONParser.MODE_JSON_SIMPLE);
JSONObject json = (JSONObject) parser.parse(jsonString);
String access_token = json.getAsString("access_token");
log.info("access_token = " + access_token);
vars.put("access_token", access_token);
JSON Extractor:
Apply to: Main sample and sub-samples
Variable names: access_token
JSON Path expressions: access_token
Match No. (0 for Random): 1
Compute concatenation var (suffix _ALL): unchecked
Default Values: none
Any ideas as to why the header manager is not applying the value of the access_token result?
Thanks!
Since you set a variable in setUp Thread Group, you cannot use it in another thread groups, since thread groups don't share variables, only properties.
So in order to pass authentication, you need to save it as a property:
${__setProperty(access_token, ${access_token})};
In this example I am using value of variable named access_token (already set, but only available in setUp thread group) to set property with the same name, which will be available across thread groups. Or change BeanShell post-processor, add:
props.put("access_token", access_token);
And then in the other thread group, you retrieve it using __P or __property function:
${__P(access_token)}
Also keep in mind that HTTP Header Manager initializes before any thread starts, so you can't use variables there for that reason too. Check this question for instance.
If you still see empty value, I recommend adding Debug Sampler (with both JMeter Properties and JMeter Variables enabled) in both thread groups, and checking where the breakage is (on saving or retrieving).
As per Functions and Variables chapter of the JMeter User Manual
Variables are local to a thread; properties are common to all threads, and need to be referenced using the __P or __property function
So the variable you define in the setUp Thread Group cannot be accessed by:
other threads in the same Thread Group
other threads outside the Thread Group
So my recommendations are:
Switch to JMeter Properties instead of Jmeter Variables, JMeter Properties are global to all threads and in fact the whole JVM instance
Switch to JSR223 PostProcessor with Groovy language instead of Beanshell PostProcessor, JSR223 Elements performance is much better, moreover Groovy has built-in JSON support.
So:
The relevant Groovy code for getting access_token attribute value and storing it into the relevant property would be :
props.put('access_token', new groovy.json.JsonSlurper().parse(prev.getResponseData()).access_token)
You can refer the value in the HTTP Header Manager (or wherever you require) as:
${__P(access_token,)}
I'm using Jmeter for my API testing using various http requests and various samplers to validate them. Now, I'm writing test for another http request. In this test,the steps are:
Issue a http request. Handle response.
using xpath xtractor, I'm extracting the response and storing URL in another variable(store_url).
If variable has a URL, repeat step-1.
This loops has to be repeated until no value is stored in (store_URL).
There is not definite number, how many time the loop has to be repeated. It is based on store_url is empty or not.
How can I achieve this in jmeter? I know step-1 and step-2. But I'm looking how to repeat step-1 and step-2. Please help me.
set a jmeter variable loopCount to 1 for init value,
move your step 1 and 2 into a loop controller,
set to loop count to ${loopCount}
in your step 2,
increase loopCount if store_url is found after you finish xpath xtractor
Put your points 1 and 2 under While Controller
Use ${__javaScript(vars.get("store_URL") != null)} as While Controller's Condition
In condition __javaScript() function is used to check store_URL variable value.
if store_URL variable is set (has any value) - everything under the While Controller will start over
if store_URL variable becomes null - the look will break
vars - is a shorthand to JMeterVariables class instance, it provides read/write access to all JMeter Variables
See Using JMeter Functions article for more detailed information on __javaScript and other useful functions
I have a collection of IDs of RESTful resources (all the same type of resource), the number of which can be indefinitely large. I want to make a REST call to get the names of these resources. Something like this:
Send:
['005fc983-fe41-43b5-8555-d9a2310719cd', '4c6e6898-e519-4bac-b03e-e8873d3fa3f0',...]
Receive:
['Resource A', 'Resource B',...]
What is the best way to retrieve the names of these resources RESTfully?
Here are the ideas I have had and the problems I see with each approach:
The naive approach is to iterate through all IDs in my collection and do a 'GET /resource/:id' for each ID. This would be prohibitively slow and resource intensive because of the large number of HTTP calls I would have to make.
The next approach I thought of is to pass the IDs as parameters to a single GET call. The problem here is that most servers have a limit on the URL length, which would be quickly exceeded.
Next, I thought that putting the IDs in the body of a GET would work, but according to Roy Fielding, data in the GET body should not affect the results of a REST call: HTTP GET with request body
I could use a POST request and put the data on the POST body, but POST is intended for creating and modifying resources, which is not what I'm doing. Maybe I should ignore the intent of the verb and use it anyway?
I could split the request into multiple GET requests to avoid exceeding the max URL length. The problem here is that I have to combine the results after all calls have returned, which is potentially slow.
I could create a collection resource within my main resource by posting my list of IDs to 'POST /resource/collection', then use a 'GET /resource/collection/:id' call to retrieve the results. This actually works, but then I have to do a 'DELETE /resource/collection/:id' to clean up. It takes multiple calls, requires cleanup, and seems a bit clunky overall, so it's okay, but not ideal.
Is there a better way to do this?
Your last approach is RESTful and the one I recommend. I'd do this:
Step 1:
Request:
POST /resource/collection
Content-Tpye: application/json
{
"ids": [
"005fc983-fe41-43b5-8555-d9a2310719cd",
"4c6e6898-e519-4bac-b03e-e8873d3fa3f0"
]
}
Response:
201 Created
Location: /resource/collection/89AB8902-FDF1-11E4-ADDF-CD4FB664A5DC
Step 2:
Request:
GET /resource/collection/89AB8902-FDF1-11E4-ADDF-CD4FB664A5DC
Response:
200 OK
Content-Type: application/json
{
"resources": [ ... ]
}
but then I have to do a 'DELETE /resource/collection/:id' to clean up.
Not, that is not necessary. The server could implement a job that removes all collections that are older than a specific timestamp. It is not the client who has to do this.
If later a client access the collection again, the server would respond with
410 Gone
I'm trying to define a REST API and I'm having trouble with one requirement.
I have an action that the API user can do that is the same thing, but can be done in two different ways.
For example, say my user uses my API to change the intensity of a light. I will have an URL something like
api/light/intensity
One option the user has to change the intensity is to set as a % of the maximum luminosity, the other option is setting the intensity as an exact value, in lumens (there is a detector for that) and he can pass the "precision" that can be low, medium and high (it changes the time it takes to get to the correct intensity).
I want the user to be able to GET the current intensity, meaning in which mode he is and depending on the mode, the % or the value in lumens and the precision.
This is where I'm lost, my GET will return a JSON object for example, is it OK to send something like
{
"Mode" = "Percent",
"Percent" = 50.5
}
when I'm in "percentage" mode and
{
"Mode" = "Exact",
"Lumens" = 200,
"Precision" = "High"
}
When I'm in "lumens" mode?
If that seems OK, how would I tell the user which type of "object" he should parse?
What would be the best way to let the user send his changes? I was thinking about having two URL, one for each mode, like
PUT /api/light/intensity/exact and PUT /api/light/intensity/percent
And both being waiting for JSON objects similar to the ones above, without the Mode.
Use HTTP Content negotiation. This allows:
the client to tell the server what representation of a resource it wants to GET,
the server to tell the client what representation of a resource it returns to the client,
the client to tell the server what represenation of a resource it is PUTing to the server.
Define two vendor content types:
application/vnd.com.example.light.intensity.percentage+json
application/vnd.com.example.light.intensity.lumens+json
The client tells the server which of both it wants:
GET /api/light/intensity/
Accept: application/vnd.com.example.light.intensity+percentage
The server responds:
200 OK
Content-Type: application/vnd.com.example.light.intensity+percentage
{
"Percent" = 50.5
}
The client wants to change the intensity:
PUT /api/light/intensity/
Content-Type: application/vnd.com.example.light.intensity+percentage
{
"Percent" = 42.7
}
The server knows from the Content-Type header how to interpret the JSON body. In this example it handles the request as in 'Percent' mode.
If the second content type was used, client and server would know to interpret the request/response as in 'Lumes' mode.
Edit: Note that the GET and PUT request use the same URL because the requests are about the same resource: the light intensity. All that differs is the representation of this resource. The proper way to handle this are content types.
The specifics will depend a bit on your API, and the needs of your users. The same GET method call to a RESTful API should always return the same value: a representation of the resource as defined by the information in the URL and nothing else. If you're maintaining state in the system, you're violating a precept of REST. (Edit: as pointed out by Gimly, that statement is unclear. It's not a violation of RESTful design for the system to maintain its own internal state, especially if a request changes the state of the system with a PUT, POST or DELETE. It's a violation for a request to rely on that state to return a representation of the resource, or to request a state change. Each request should be self-contained.)
I'd use a query string to change the format of the representation:
GET /api/light/intensity
GET /api/light/intensity?f=percent
That way /api/light/intensity always refers to the same resource (defaulting to the "exact" representation, which has the most data), and the query string "filters" the representation, similarly to a search query. It removes some data (in this case, the exact luminosity and precision) in favor of a relative representation in percent of some maximum value. Alternately, you could think of it as controlling the output format: GET /foo.json vs GET /foo.xml. The resource is the same, but the representation differs.
For updating a resource, you can take an object as you've described. Your server will have to understand the different formats, but you could either PUT to the bare URL, or again use a query parameter to control the format expected by the server, and then let your payload be more abstract, using value instead of lumens or percentage:
PUT /api/light/intensity
Payload: {"value": 200, "precision": "high"}
PUT /api/light/intensity?f=percent
Payload: {"value": 50.5}
That allows you to structure the API for your light resource in such a way that intensity is one property of the resource. "Percent" then becomes a convenience representation in the output, so when you return the entire light resource, it would read something like:
"light": {
"name": "the light",
"id": 12345,
"intensity": 200,
"max-intensity": 400,
...
}
So the API user could calculate current percent based on intensity and max-intensity. (You could of course substitute "percent" for "max-intensity" and let the user do the math the other way, but it feels more natural to me to provide absolute values and let the math calculate relative values.
Edit
Please see Tichodroma's answer for the better way of handling this. I'm leaving the answer because the discussion in the comments was useful to me, and may be useful to others in the future.
How do I generate an ETag HTTP header for a resource file?
As long as it changes whenever the resource representation changes, how you produce it is completely up to you.
You should try to produce it in a way that additionally:
doesn't require you to re-compute it on each conditional GET, and
doesn't change if the resource content hasn't changed
Using hashes of content can cause you to fail at #1 if you don't store the computed hashes along with the files.
Using inode numbers can cause you to fail at #2 if you rearrange your filesystem or you serve content from multiple servers.
One mechanism that can work is to use something entirely content dependent such as a SHA-1 hash or a version string, computed and stored once whenever your resource content changes.
An etag is an arbitrary string that the server sends to the client that the client will send back to the server the next time the file is requested.
The etag should be computable on the server based on the file. Sort of like a checksum, but you might not want to checksum every file sending it out.
server client
<------------- request file foo
file foo etag: "xyz" -------->
<------------- request file foo
etag: "xyz" (what the server just sent)
(the etag is the same, so the server can send a 304)
I built up a string in the format "datestamp-file size-file inode number". So, if a file is changed on the server after it has been served out to the client, the newly regenerated etag won't match if the client re-requests it.
char *mketag(char *s, struct stat *sb)
{
sprintf(s, "%d-%d-%d", sb->st_mtime, sb->st_size, sb->st_ino);
return s;
}
From http://developer.yahoo.com/performance/rules.html#etags:
By default, both Apache and IIS embed data in the ETag that dramatically reduces the odds of the validity test succeeding on web sites with multiple servers.
...
If you're not taking advantage of the flexible validation model that ETags provide, it's better to just remove the ETag altogether.
How to generate the default apache etag in bash
for file in *; do printf "%x-%x-%x\t$file\n" `stat -c%i $file` `stat -c%s $file` $((`stat -c%Y $file`*1000000)) ; done
Even when i was looking for something exactly like the etag (the browser asks for a file only if it has changed on the server), it never worked and i ended using a GET trick (adding a timestamp as a get argument to the js files).
Ive been using Adler-32 as an html link shortener. Im not sure whether this is a good idea, but so far, I havent noticed any duplicates. It may work as a etag generator. And it should be faster then trying to hash using an encryption scheme like sha, but I havent verified this. The code I use is:
shortlink = str(hex(zlib.adler32(link)+(2**32-1)/2))[2:-1]
I would recommend not using them and going for last-modified headers instead.
Askapache has a useful article on this. (as they do pretty much everything it seems!)
http://www.askapache.com/htaccess/apache-speed-etags.html
The code example of Mark Harrison is similar to what used in Apache 2.2. But such algorithm causes problems for load balancing when you have two servers with the same file but the file's inode is different. That's why in Apache 2.4 developers simplified ETag schema and removed the inode part. Also to make ETag shorter usually they encoded in hex:
<inttypes.h>
char *mketag(char *s, struct stat *sb)
{
sprintf(s, "\"%" PRIx64 "-%" PRIx64 "\"", sb->st_mtime, sb->st_size);
return s;
}
or for Java
etag = '"' + Long.toHexString(lastModified) + '-' +
Long.toHexString(contentLength) + '"';
for C#
// Generate ETag from file's size and last modification time as unix timestamp in seconds from 1970
public static string MakeEtag(long lastMod, long size)
{
string etag = '"' + lastMod.ToString("x") + '-' + size.ToString("x") + '"';
return etag;
}
public static void Main(string[] args)
{
long lastMod = 1578315296;
long size = 1047;
string etag = MakeEtag(lastMod, size);
Console.WriteLine("ETag: " + etag);
//=> ETag: "5e132e20-417"
}
The function returns ETag compatible with Nginx. See comparison of ETags form different servers