Transaction Throughput vs Threads since 0.1.0
Don't forget to set up saving thread counts when running in non-GUI mode!
This listenter is very similar to Response Times vs Threads, exept it shows total server's transaction throughput for active test threads. The formula for total server transaction throughput is <active threads> * 1 second / <1 thread response time>.
Example calculations:
- when you have one thread (user) sending requests to server and server responds after 100ms, you have 1 thread * 1000ms / 100ms = 10 transactions per second.
- when you have 10 threads sending requests to server and server responds after 100ms, you have 10 threads * 1000ms / 100ms = 100 transactions per second. This means that your server haven't reached its resources limits and the more users works with it the more transaction it processes.
- when you have 20 threads sending requests to server and server responds after 200ms, you have 20 threads * 1000ms / 200ms = 50 transactions per second. This means that at 20 parallel users server become responding slower and there's some overhead to handle parallelism, e.g. database locks.
So basically, it shows the statistical maximum possible number of transactions based on number of users accessing the application. In the bellow graph, we can see for the transaction named 'First Access':
- If one user alone access the application, he will have a maximum of 2 transactions per seconds
- If two users access the application, both will have a maximum of 4 transactions per seconds
- etc.
I am having trouble understanding the type of requirement that would lead to having such a graph. at first, it seemed like a good idea, but I gave it a thought and can't find any pro arguments for the calculation formula you used, now. furthermore, isn't transactions / time graphed over a period of time more relevant, statistically speaking?
this is how I interpret the example above - the server transactions capability decreases with the number of threads, which is not necessarily true until you reach the server's maximum capability (the load is higher, the response times have increased a little, but the throughput actually increases a lot when the test reaches it's max potential).
i'd appreciate your input.
adrian, This formula reflects my playing with graph in JMeter, seeking more useful graphs. Clean transaction throughput is just "1000 / response time", and adds no value to response time graph. I agree that this graph can be a bit confusing, please make your suggestions.
gave a thought and i know you're looking for ideas and this how I see it:
what i find useful including in reports is a graph similar to the one on the jmeter wiki LogAnalysis?, response time fluctuation over time compared to throughput fluctuation over time (in one graph). it is focused on one test-action at a time (or on several actions with similar response times). this is pretty easy to do in excel or perl, but it takes a lot of work when there are many graphs to create... i find it useful in order to show that increasing the server load causes a direct proportional increase in response times and after that when decreasing the server load the response times also decrease to previous levels, proportionaly.
adrian, 1. There was this "time/throughput" switch in deprecated version 0.0.2 2. I got the "R-time/throughput over time" idea. 3. I did not get ideas about LogAnalysis? and fluctiations, could you give more explanations what do you expect?
about 1, 2 - great.
about the jmeter wiki, this is the one that i found useful in data analysis: http://wiki.apache.org/jakarta-jmeter/LogAnalysis#Summarizing_Huge_Datasets (it graphs the response times and the transactions per second... that way it's easy to see how increasing load affects load times, it's important)
furthermore, when generating graphs, I plot the response times and transactions per second OVER TIME. I do this for 1 sampler, relevant to the test. for example: login takes more than a static page, so it's not relevant to view their average, it's better to view them graphed independently.
was this explanation better than the previous? what is your input on this?
Ok, you can get r157 and see Response Times Over Time plot.
Again, excellent work! Could you add Transaction Throughput vs Time? That would help to test server endurance.
Thanks and Regards,
Stephane
Stephane, I can add it, in the next release
Hi.. the avg throughput values shown in the aggregate report listener for each samples is not same as the avg values for the same samples in the throughput vs threads listener.. e.g for a sample test conducted the throughput value in Aggregate report listener is 1.1/sec and the avg throughput value for the same sample in throughput vs threads listener is 49.3/sec..
please let me know if my interpretation of the graph is wrong..
Hi, Prasanna, Yes, there's duscussion about how to calculate transaction throughput in JMeter Plugins, and current algorithm is really different of native JMeter aggregate reports. I find useful that Throughput is calculated for the whole server load with following statements:
Native aggregate report uses just 2 statement for throughput calculation.
Tested out some of the new listeners and the plugins I haven't checked out yet (I was more interested on the response / throughput over time listeners). Great! I want to congratulate you on your initiative as well. Works fine so far in synthetic benchmarks of the test system, will run it on live application in the future, for further impressions.
This particular listener is giving me very bogus results. If I take the explanation above and use the equation:
<active threads> X 1 second / <1 thread response time>
by taking data from the Active Threads listener and the Response Time listener, I get a completely different plot. Sometimes this listener gives a factor of 10 times higher than the direct calculation.... and it continues to increase with the number of threads, even if the server under test is saturated.
I agree that the chart is confusing. We need to change the equation for it to be more correct
Can I retrieve these graphs after running in non-GUI mode?
yes, either load them in the gui or use command line tool JMeterPluginsCMD
Seems one of the example calculations is off:
20 threads 1000ms / 200ms = 50 transactions per second. If you do the actual calculation, result is still 100. You would need a different response time (400 ms) to produce the desired result of 50 transactions/sec.