Is there some API available for microsecond accurate timing? Some jitter is acceptable. Something equivalent to performance.now() is preferable.
In my 5 minutes of research I found the console object which does log times accurately enough but there is no easy way to retrieve those logged entries. Additionally I may call this timing function thousands of times which would clutter logs.
Related
I have an Azure function triggered via service bus which basically just does a long await (1 - 4.5 minutes (managed with cancellation token to prevent function timeout and host restarting)).
I want to process as many of these long await messages as I can. (Ideally about 1200 at the same time..)
First I ran my function on an App Service Plan (with Concurrent Calls = 1200), but I think each trigger creates a thread, and 1200 threads causes some issues.
So I decided to run it on Consumption, with Batch Size 32, with the idea what I can avoid creating tons of threads and scale out the consumption function instead when it sees the queue build up.
Unfortunately exactly the opposite happens, the Consumption function will process 32 messages, but never scales out even though the queue has 1000+ items in it. Even worse, some times the function just goes to sleep although there are still many items in the queue.
I feel my best option would be to group work in a message, so instead of 1 message = 1 long await, 1 message could be 100 awaits for example, but my architecture doesn't really allow to me group messages easily (because if some tasks fail but some succeed this is easily managed with dead letters, but with grouping I need to maintain state to track this). Is there an existing way to efficiently have many (independent) long running awaits on an azure function, either consumption or service plan.
i'll try to compare the same script done in Http/html with TruClient. In both of the scenarios, it has same think time/wait time, same number of vusers, same pacing.
Is it possible that they have approximately same time for each transactions but they are so different in term of total number of passed transactions?
Ty in advance
In web HTTP/HTMl protocol, Response time = Processing time + Latency (time taken by network while transferring data).
In Truclient protocol, Response time = Processing time + Latency + Rendering time
Hence you will found a difference between both response times.
And execution times will differ in both protocols, hence total number of passed transactions also vary.
The question comes on what are you trying to measure? Are you trying to measure the response time of your servers or are you trying to measure the weight of the client in the impact of response time? I put forward the hypothesis that it is possible to measure the client weight by examination of the times captured with the developer tools in both development and also in functional testing.
So much of this client weight is related to page architecture that if you are waiting for performance testing to show you that your page architecture is problematic then you likely will not have time to fix the issues and retest before going to production.
I also recommend the collected O'Reilly works of Steve Souders which will help to bring home the client bound concepts for page design and how much this impacts the end user experience over and above how fast the server responds.
http://www.oreilly.com/pub/au/2951
I'm developing a web app that needs to handle bursts of very high loads,
once per minute I get a burst of requests in very few seconds (~1M-3M/sec) and then for the rest of the minute I get nothing,
What's my best strategy to handle as many req /sec as possible at each front server, just sending a reply and storing the request in memory somehow to be processed in the background by the DB writer worker later ?
The aim is to do as less as possible during the burst, and write the requests to the DB ASAP after the burst.
Edit : the order of transactions in not important,
we can lose some transactions but 99% need to be recorded
latency of getting all requests to the DB can be a few seconds after then last request has been received. Lets say not more than 15 seconds
This question is kind of vague. But I'll take a stab at it.
1) You need limits. A simple implementation will open millions of connections to the DB, which will obviously perform badly. At the very least, each connection eats MB of RAM on the DB. Even with connection pooling, each 'thread' could take a lot of RAM to record it's (incoming) state.
If your app server had a limited number of processing threads, you can use HAProxy to "pick up the phone" and buffer the request in a queue for a few seconds until there is a free thread on your app server to handle the request.
In fact, you could just use a web server like nginx to take the request and say "200 OK". Then later, a simple app reads the web log and inserts into DB. This will scale pretty well, although you probably want one thread reading the log and several threads inserting.
2) If your language has coroutines, it may be better to handle the buffering yourself. You should measure the overhead of relying on our language runtime for scheduling.
For example, if each HTTP request is 1K of headers + data, want to parse it and throw away everything but the one or two pieces of data that you actually need (i.e. the DB ID). If you rely on your language coroutines as an 'implicit' queue, it will have 1K buffers for each coroutine while they are being parsed. In some cases, it's more efficient/faster to have a finite number of workers, and manage the queue explicitly. When you have a million things to do, small overheads add up quickly, and the language runtime won't always be optimized for your app.
Also, Go will give you far better control over your memory than Node.js. (Structs are much smaller than objects. The 'overhead' for the Keys to your struct is a compile-time thing for Go, but a run-time thing for Node.js)
3) How do you know it's working? You want to be able to know exactly how you are doing. When you rely on the language co-routines, it's not easy to ask "how many threads of execution do I have and what's the oldest one?" If you make an explicit queue, those questions are much easier to ask. (Imagine a handful of workers putting stuff in the queue, and a handful of workers pulling stuff out. There is a little uncertainty around the edges, but the queue in the middle very explicitly captures your backlog. You can easily calculate things like "drain rate" and "max memory usage" which are very important to knowing how overloaded you are.)
My advice: Go with Go. Long term, Go will be a much better choice. The Go runtime is a bit immature right now, but every release is getting better. Node.js is probably slightly ahead in a few areas (maturity, size of community, libraries, etc.)
How about a channel with a buffer size equal to what the DB writer can handle in 15 seconds? When the request comes in, it is sent on the channel. If the channel is full, give some sort of "System Overloaded" error response.
Then the DB writer reads from the channel and writes to the database.
After successfully using JMeter to profile our platform's performance I got the request to simulate a 24h load based on minute-by-minute transaction data extracted from the last year's logs.
At this point, having the static nature of thread creation in jmeter I am wondering if this is easily achievable. I studied the usual plugins together with those at jmeter-plugins.org but still I could not find a straightforward way to do this kind of shaping.
I am looking at the alternative to write a groovy script that dynamically feeds a throughput shaping timer but I am not sure if this is the proper way to go.
Any suggestions?
UPDATE:
I tried the follwing combination (as also Alon and Dan suggested):
- One thread group with one looping thread and a 60 seconds delay timer; this thread reads every minute from csv the number of requests for the next minute and passes it to the next thread group (using a groovy script and global props)
- the second thread group has a fixed number of threads and a Constant Throughput Timer that is updated every minute by the first thread group.
It works partially but the limitation here is that the load/min is divided among all active threads, so part of the threads will still wait to be executed even if the load request changed in the meanwhile.
I think that in order to have a correct simulation there should be a way that all threads that were not executed within the minute be interrupted and started again.
So for a concrete example:
I have 100 requests in the first minute and 5000 in the second (it is real data with big variations)
In the first minute 300 threads have been started (this is my max nr of concurrent connections accepted), but, because they execute very fast they are going to be delayed for more than a minute in order to fulfill the calculated throughput,
so the 5000 requests for the next minute don't have a chance to be executed because lots of threads are still sleeping.
So I am looking for a way to interrupt sleeping threads when more throughput is needed. Probably from Groovy or by modifying some JMeter code.
Thanks,
Dikran
You should use JMeter's constant throughput timer for this. In combination with a CSV file that includes all of the values, it should work perfectly.
See these links:
http://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer
http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config
Best,
Alon.
Use JSR 223 + Groovy for Scripting
You have a lot of options to do scripting with JMeter:
Beanshell
BSF and all supported languages Javascript, Scala , Groovy, Java ...
JSR223 and all supported languages Javascript, Scala , Groovy, Java
...
Although you can be lazy and choose the language you know, FORGET ABOUT IT.
Use the most efficient option, which is JSR223 + Groovy + Caching (supported since JMeter 2.8 in external script and in next upcoming JMeter 2.9 also supported with embedded scripts).
Using Groovy is as simple as adding
groovy-VERSION-all.jar in <JMETER_HOME>/lib folder.
But of course ensure your script is necessary and efficiently written, DON'T OVERSCRIPT
View more over here - http://blazemeter.com/blog/jmeter-performance-and-tuning-tips
I'm evaluating possible solutions for handling a large quantity of queued messages, which must be delivered to workers at a certain date and time. The result of executing them is mostly updates to stored data, and they may or may not be originally triggered by user action.
For example, think of what you'd implement in a hypothetical large-scale StarCraft game server for storing and executing users' actions, like upgrading a building, hatching a soldier, all of which requires to be applied to the game state after several seconds or minutes after the player initiates them.
The problem is I can't seem to find the right term to name this problem area. There are several that looks similar, but different:
cron/task/job scheduler
The content of the queue is not dynamic, it's predefined.
Each task is scheduled.
message queue
The content of the queue is dynamic.
Each task is intended to be delivered immediately.
???
The content of the queue is dynamic.
Each task is scheduled.
If there are message queues that allow conditional delivery of messages, that might be it.
Summary:
What are these kind of technology called?
What are some of the solutions out there?
This just sounds like a trivial priority queue on the surface. The priority in this case is the time of completion, and you check the front of the queue to see when the next event is due. Pretty much every language comes with a priority queue or something that can easily be used as one, so I'm not sure what the actual problem is here.
Is it that you're worried about scalability, when it comes to millions of messages? Obviously 'millions' is a meaningless term - if that's millions per day, it's a trivial problem. If it's millions per second, then you can just scale horizontally, splitting the queue across multiple processes. (And the benefit of such a queue system is that this parallelization is really simple.)
I would bet that when implementing a large scale real-time strategy game server you would hit networking problems long before you start hitting problems with the message queue.
Have you tried looking at push queues by Iron.io? The content of the queue can be anything you like, and you specify a webhook to where the messages will be pushed to. You can also set a delay for each of the messages.
The webhook is static though for each queue and delay isn't always exactly on time (could be up to a minute off). If timing is more important or the ability of providing a different webhook per message is important, try looking at boomerang.io.
They say they are pretty accurate on the timing, you can provide a delay or unix timestamp for the webhook to return and that is per message. Sounds like either of those might work for you.
For StarCraft, I would use the Red Dwarf server.
For a Java EE app, I would use Quartz Scheduler.
It seems to me that a queue-based solution would be best in this case for a number of reasons:
Management. Most queuing solutions provide support for inspecting the content of queues which makes it easier to debug, easier to take action when certain threshold are exceeded, ...
Performance. You can divide workload by having multiple enqueue/dequeue processes (gives you the ability to scale out).
Prioritizing. Most queues support prioritizing of messages (probably not all messages are equally important).
...
Remaining problem is the immediate delivery of messages in the queue. You have two ways to solve this: either delay enqueuing of messages or delay execution of dequeued messages. I would go with the first approach, delayed enqueuing.
A message then has two properties: (content, delay). You provide the message to a component in your system that queues the message at the appropriate time.
I'm not sure what programming language you're using, but the MS .NET 4 framework has support for such a scenario (delayed execution of tasks).