I need to stress-test a system and http://locust.io seems like the best way to go about this. However, it looks like it is set up to use the same user every time. I need each spawn to log in as a different user. How do I go about setting that up? Alternatively, is there another system that would be good to use?
Locust author here.
By default, each HttpLocust user instance has an HTTP client that has it's own separate session.
Locust doesn't have any feature for providing a list of user credentials or similar. However, your load testing scripts are just python code, and luckily it's trivial to implement this yourself.
Here's a short example:
# locustfile.py
from locust import HttpLocust, TaskSet, task
USER_CREDENTIALS = [
("user1", "password"),
("user2", "password"),
("user3", "password"),
]
class UserBehaviour(TaskSet):
def on_start(self):
if len(USER_CREDENTIALS) > 0:
user, passw = USER_CREDENTIALS.pop()
self.client.post("/login", {"username":user, "password":passw})
#task
def some_task(self):
# user should be logged in here (unless the USER_CREDENTIALS ran out)
self.client.get("/protected/resource")
class User(HttpLocust):
task_set = UserBehaviour
min_wait = 5000
max_wait = 60000
The above code wouldn't work when running Locust distributed, since the same code runs on each slave node, and they don't share any state. Therefore you would have to introduce some external datastore which the slave nodes could use to share states (e.g. PostgreSQL, redis, memcached or something else).
Alternatively, you can create users.py module to hold the users' information you need in your test cases, in my example, it holds email and cookies. Then you can call them randomly in your tasks. See below:
# locustfile.py
from locust import HttpLocust, TaskSet, task
from user_agent import *
from users import users_info
class UserBehaviour(TaskSet):
def get_user(self):
user = random.choice(users_info)
return user
#task(10)
def get_siparislerim(self):
user = self.get_user()
user_agent = self.get_user_agent()
r = self.client.get("/orders", headers = {"Cookie": user[1], 'User-Agent': user_agent})
class User(HttpLocust):
task_set = UserBehaviour
min_wait = 5000
max_wait = 60000
User and user-agent can be called by a function. With this way, we could distribute the test with many users and different user-agents.
# users.py
users_info = [
['performancetest.1441926507#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926506#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926501#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926499#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926494#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926493#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926492#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926491#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926490#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926489#gmail.com', 'cookies_created_by_each_user'],
['performancetest.1441926487#gmail.com', 'cookies_created_by_each_user']]
I took a slightly different approach when implementing this for a distributed system. I utilized a very simple flask server that I made a get call to during the on_start portion of the TaskSet.
from flask import Flask, jsonify
app = Flask(__name__)
count = 0 #Shared Variable
#app.route("/")
def counter():
global count
count = count+1
tenant = count // 5 + 1
user = count % 5 + 1
return jsonify({'count':count,'tenant':"load_tenant_{}".format(str(tenant)),'admin':"admin",'user':"load_user_{}".format(str(user))})
if __name__ == "__main__":
app.run()
In this way I now have an endpoint I can get at http://localhost:5000/ on whatever host I run this. I just need to make this endpoint accessible to the slave systems and I wont have to worry about duplicate users or some type of round robin effect caused by having a limited set of user_info.
Piggy-backing on #heyman's answer here. The example code will work, but continuing to start / stop tests will eventually clear out the USER_CREDENTIALS list, and start to throw errors.
I ended up adding the following:
from locust import events # in addition to the other locust modules needed
def hatch_complete_handler(**kw):
global USER_CREDENTIALS
USER_CREDENTIALS = generate_users() # some function here to regenerate your list
events.hatch_complete += hatch_complete_handler
This refreshes your user list once your swarm finishes hatching.
Also keep in mind that you'll need a list longer than the number of users you wish to spawn.
Related
I am trying to use federated learning framework flower with TensorFlow. My code seems to compile fine but It's not showing federated loss and accuracy. What am I doing wrong?
ServerSide Code :
import flwr as fl
import sys
import numpy as np
class SaveModelStrategy(fl.server.strategy.FedAvg):
def aggregate_fit(
self,
rnd,
results,
failures
):
aggregated_weights = super().aggregate_fit(rnd, results, failures)
"""if aggregated_weights is not None:
# Save aggregated_weights
print(f"Saving round {rnd} aggregated_weights...")
np.savez(f"round-{rnd}-weights.npz", *aggregated_weights)"""
return aggregated_weights
# Create strategy and run server
strategy = SaveModelStrategy()
# Start Flower server for three rounds of federated learning
fl.server.start_server(
server_address = 'localhost:'+str(sys.argv[1]),
#server_address = "[::]:8080" ,
config={"num_rounds": 2} ,
grpc_max_message_length = 1024*1024*1024,
strategy = strategy
)
Server Side:
According to the source code of app.py, I realized that we can set force_final_distributed_eval = True. So we need to pass this to fl.server.start_server().
Not sure whether this is intended, but it solved my problem.
In jmeter,I want the results while the running the test,which beansheel code add to sampler and convert summary report values in to milliseconds and push those values in MySQL db automatically by adding one sampler.
please give me step by step process and all possible ways explain
and how create a table in particular values on jtl file values in avg,min,max,response time,error values in mysql db please explain
Wouldn't that be easier to use InfluxDB instead? JMeter provides Backend Listener which automatically sends metrics to InfluxDB and they can be visualized via Grafana. Check out How to Use Grafana to Monitor JMeter Non-GUI Results - Part 2 article for more details.
If you have to use MySQL the correct approach would be writing your own implementation of the AbstractBackendListenerClient
If you need a "single sampler" - take a look at JSR223 Listener, it has prev shorthand for SampleResult class instance providing access to all the necessary information like:
def name = prev.getSampleLabel() // get sampler name
def elapsed = prev.getTime() // get elapsed time (in milliseconds)
// etc.
and in order to insert them into the database you could do something like:
import groovy.sql.Sql
def url = 'jdbc:mysql://localhost:3306/your-database'
def user = 'your-username'
def password = 'your-password'
def driver = 'com.mysql.cj.jdbc.Driver'
def sql = Sql.newInstance(url, user, password, driver)
def insertSql = 'INSERT INTO your-table-name (sampler, elapsed) VALUES (?,?)'
def params = [name , elapsed]
def keys = sql.executeInsert insertSql, params
sql.close()
I'm using a Django database model to store objects corresponding to a remote ticket service. In testing this model, I'm running several tests to make sure that log messages work correctly for the database model - I'm using Django-nose to run these tests.
The database model looks something like this, using the JSONField from this StackOverflow answer with a small modification to support lists as well as dicts:
class TicketEntity(django.db.models.Model)
tickets = JSONField(null=True, blank=True, default=[], serialize=True)
queued_ticket_data = JSONField(null=True, blank=True, default=[], serialize=True)
...
#classmethod
def create(cls, *args, **kwargs):
instance = cls(*args, **kwargs)
instance.save()
return instance
def queue_output_for_ticket(self, log_data):
self.queued_ticket_data += [log_data]
self.save()
def push_ticket(self):
self.tickets += [self.queued_ticket_data]
self.queued_ticket_data = []
self.save()
return True
The tests (in the order they appear to get run) look like this:
def test_ticket_entity_push_ticket(self):
entity = TicketEntity.create()
entity.queue_output_for_ticket("log output")
entity.queue_output_for_ticket("more log output")
entity.push_ticket()
self.assertEquals(entity.tickets, [[log_data, log_data_1]])
self.assertFalse(entity.queued_ticket_data)
def test_ticket_entity_queue_output_for_ticket(self):
entity = TicketEntity.create()
log_data = "Here's some output to log.\nHere's another line of output.\nAnd another."
entity.queue_output_for_ticket(log_data)
self.assertEquals(entity.queued_ticket_data, [log_data])
The first test passes, perfectly fine. The second test fails on that assert statement, because the entity.queued_ticket_data looks like this:
["log output",
"more log output",
"Here's some output to log.\nHere's another line of output.\nAnd another."]
Those first two elements are there at the very start of the test, immediately after we call TicketEntity.create(). They shouldn't be there - the new instance should be clean because we're creating it from scratch.
Similarly, tickets is pre-filled as well. The TicketEntity has other fields that are not JSONFields, and they do not seem to exhibit this problem.
Can someone shed some light on why this problem might be happening, and what kind of modification we'd need for a fix?
I'd like to execute two scenarios that should be executed one after another and the data "produced" by the first scenario should be used as base for the second scenario.
So a case could be for example clearing of a credit card. The first scenarios is there to authorize/reserve of a certain amount on the card:
val auths = scenario("auths").during(durationInMinutes minutes) {
feed(credentials)
.feed(firstNames)
.feed(lastNames)
.feed(cards)
.feed(amounts)
.exec(http("send auth requests")
.post(...)
.check(...))}
The second one is there to capture/take the amount from the credit card:
val caps = scenario("caps").during(durationInMinutes minutes) {
feed(credentials)
.feed(RESPONSE_IDS_FROM_PREVIOUS_SCENARIO)
.exec(http("send auth requests")
.post(...)
.check(...))}
I initially thought about using the saveAs(...) option on check but I figured out that the saved field is only valid for the given session.
So basically I want to preserve the IDs I got from the auths scenario and use them in the caps scenario.
I cannot execute both steps in one scenario though (saveAs would work for that) because I have different requirement for both scenarios.
Quoting the documentation: "Presently our Simulation is one big monolithic scenario. So first let us split it into composable business processes, akin to the PageObject pattern with Selenium. This way, you’ll be able to easily reuse some parts and build complex behaviors without sacrificing maintenance." at gatling.io/Advanced Tutorial
Thus your there is no build-in mechanism for communication between scenarios (AFAIK). Recommendation is to structure your code that way that you can combine your calls to URIs subsequently. In your case (apart from implementation details) you should have something like this:
val auths = feed(credentials)
.feed(firstNames)
.feed(lastNames)
.feed(cards)
.feed(amounts)
.exec(http("send auth requests")
.post(...)
.check(...) // extract and store RESPONSE_ID to session
)
val caps = exec(http("send auth requests")
.post(...) // use of RESPONSE_ID from session
.check(...))
Then your scenario can look something like this:
val scn = scenario("auth with caps").exec(auths, caps) // rest omitted
Maybe even better way to structure your code is to use objects. See mentioned tutorial link.
More illustrative example (which compiles, but I didn't run it while domain is foo.com):
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class ExampleSimulation extends Simulation {
import scala.util.Random
import scala.concurrent.duration._
val httpConf = http.baseURL(s"http://foo.com")
val emails = Iterator.continually(Map("email" -> (Random.alphanumeric.take(20).mkString + "#foo.com")))
val names = Iterator.continually(Map("name" -> Random.alphanumeric.take(20).mkString))
val getIdByEmail = feed(emails)
.exec(
http("Get By Email")
.get("/email/$email")
.check(
jsonPath("userId").saveAs("anId")
)
)
val getIdByName = feed(names)
.exec(
http("Get By Name")
.get("/name/$name")
.check(
jsonPath("userId").is(session =>
session("anId").as[String]
)
)
)
val scn = scenario("Get and check user id").exec(getIdByEmail, getIdByName).inject(constantUsersPerSec(5) during (5.minutes))
setUp(scn).protocols(httpConf)
}
Hope it is what you're looking for.
Trying to port some SqlAlchemy to Django and I've got this tricky little bit:
version = Column(
BIGINT,
default=literal_column(
'UNIX_TIMESTAMP() * 1000000 + MICROSECOND(CURRENT_TIMESTAMP)'
),
nullable=False)
What's the best option for porting the literal_column bit to Django? Best idea I've got so far is a function to set as the default that executes the same raw sql, but I'm not sure if there's an easier way? My google-foo is failing me there.
Edit: the reason we need to use a timestamp created by mysql is that we are measuring how out of date something is (so we need to actually know time) and we want, for correctness, to have only one time-stamping authority (so that we don't introduce error using python functions that look at system times, which could be different across servers).
At present I've got:
def get_current_timestamp(self):
cursor = connection.cursor()
cursor.execute("SELECT UNIX_TIMESTAMP() * 1000000 + MICROSECOND(CURRENT_TIMESTAMP)")
row = cursor.fetchone()
return row
version = models.BigIntegerField(default=get_current_timestamp)
which, at this point, sounds like my best/only option.
If you don't care about having a central time authority:
import time
version = models.BigIntegerField(
default = lambda: int(time.time()*1000000) )
To bend the database to your will:
from django.db.models.expressions import ExpressionNode
class NowInt(ExpressionNode):
""" Pass this in the same manner you would pass Count or F objects """
def __init__(self):
super(Now, self).__init__(None, None, False)
def evaluate(self, evaluator, qn, connection):
return '(UNIX_TIMESTAMP() * 1000000 + MICROSECOND(CURRENT_TIMESTAMP))', []
### Model
version = models.BigIntegerField(default=NowInt())
because expression nodes are not callables, the expression will be evaluated database side.