posted 06-26-2002 07:43 AM         

In a peak hour of usage, how many times will a particular transactions be performed? How long does this transaction take the average user to complete ? How many transactions does each user perform ?

Using this information you can work out how many virtual users, running at "real user" speed, you will need.

E.G. you expect 1000 transactions in an hour, the transaction will take 6 mins, and each user will perform a single transaction.

To achieve this level of load you need 100 virtual users, repeatedly running the transaction, with think time included so that the transaction time averages 6 minutes.
Each virtual user performs 10 transactions per hour, and the total is 1000, as required.

posted 06-26-2002 09:48 AM         

Also do not confuse the possible number of users with the maximum concurrent users. Quite often the maximum concurrent users count is significantly lower.

Also keep an eye out for your network usage (if you use any Internet pipes), with that amount of users you could easily run out of bandwidth.

------------------
Roland

posted 07-18-2002 12:07 PM         

Consider this example. You need to simulate 100 concurrent users who each spend approximately 6 minutes completing a specific task. If you set up a test where all 100 users log in at the same time and perform the activity at the same time then the server will see unrealistic spikes in activity. If, however, you model realistically, those 100 concurrent users will begin their activity over a 6 minute window and your test will loop several times to get a realistic picture of what, say, 1 hour of 100 concurrent users actually looks like from the server perspective.

Remember, try to design your tests so that the servers can't tell the difference between the simulated users (Virtual Testers) and actual users.

posted 07-18-2002 09:00 PM         

quote:

---

Originally posted by RSBarber:

Remember, try to design your tests so that the servers can't tell the difference between the simulated users (Virtual Testers) and actual users.

---

This could depend on what type of testing that your trying to perform since performance has 4 subsets of test (load, stress, volume and transaction testing). For instance if I am running a Load test, I would want to model the actual users. However if I am running a sustained stress/stability test or running more of a hardware configuration test (where I am comparing a current piece of hardware with a new piece of hardware...I probably would not necessarily model actual users but instead setup a benchmark to compare against. So it really depends on the type of performance testing you are planning on doing. You may find the following thread useful.

http://www.qaforums.com/boards/ubb/Forum15/HTML/000101.html

------------------

posted 07-19-2002 07:30 AM         

quote:

---

Originally posted by Shaky:

This could depend on what type of testing that your trying to perform since performance has 4 subsets of test (load, stress, volume and transaction testing).

---

Many people categorize performance related testing into different "buckets", but that not the point at hand.

I agree with your point about how you model being specific to the type of test you want to execute, however the origional question asked about modeling a specific number of employees, thus focusing on true load testing. Or at least that was my interpretation of the question.

posted 07-19-2002 08:06 PM         

quote:

---

Originally posted by JoeW:

E.G. you expect 1000 transactions in an hour, the transaction will take 6 mins, and each user will perform a single transaction.

To achieve this level of load you need 100 virtual users, repeatedly running the transaction, with think time included so that the transaction time averages 6 minutes.
Each virtual user performs 10 transactions per hour, and the total is 1000, as required.

---

This could help calculate the number of virtual users only if each transaction processing time is constant, always completes in 6 minutes. However, in reality it changes with different number of concurrent users. In such a case how could you get the transaction processing time so as to calculate the number of virtual users to emulate the 1000 transactions in an hour.

posted 07-22-2002 07:46 AM         

quote:

---

Originally posted by roadrunner:
This could help calculate the number of virtual users only if each transaction processing time is constant, always completes in 6 minutes. However, in reality it changes with different number of concurrent users. In such a case how could you get the transaction processing time so as to calculate the number of virtual users to emulate the 1000 transactions in an hour.

---

roadrunner, you open what could become a huge philisophical debate, that I will address only briefly. Probably the most direct answer to your question, is that you couldn't. You would make your best guess then run the test and see if you got it right. Truth is, transactions per time measurements are normally goals, or artificial lines in the sand anyway. Get close, run a test that is a little under and one that is a little over, and you meet the intent of the test. No one really expects a (web-based) system to deliver exactly 1000 transactions per hour every hour.

This opens the point of the value of transaction based modeling for web-based systems. I submit that transactions per time are statistics that result from accurate user community and workload distribution modeling and not the other way around.

posted 07-22-2002 08:56 AM         

The transaction time will be an average, although I would not expect it to increase significantly with concurrent users if you are within expected operational limits. I.E. the transaction time should increase only once you have exceeded the expected peak usage (optimistic I know).

quote:

---

This opens the point of the value of transaction based modeling for web-based systems. I submit that transactions per time are statistics that result from accurate user community and workload distribution modeling and not the other way around.

---

posted 07-22-2002 09:43 AM         

Certainly, this is one of my favorite topics.

The high-level overview is this. With a client server application, the only way to measure user load was by seeing how many transactions occured over a certain period of time, then simulating that load on the back end.

Web-based applications aren't as simple to model for a variety of reasons. First is that HTTP is a connectionless protocol, so the concept of concurrent or active users gets confusing. Add to that the fact that most users will only be accessing a web site/application for a matter of minutes as opposed to hours and the fact that most web sites/applications have significantly more navigation paths, activities and options than the majority of client/server applications and you end up with a significantly more complex model to accurately represent the load the web site/application will see in production.

SO, while one can still model from the "server perspective" with a transactions per time approach then try to map that to user activity, number of users, and user paths. My point is that the transactions per time is actually a side effect of "how many users are doing what, when".

The first three articles in my Rational Developers Network (RDN) series "User Experience not Metrics" actually do a much better job of explaining this concept in detail, as well as describing the "how" to go with the concept.

If you don't have access to RDN and are interested in reading the articles, please email me directly, and I will mail you an MS Word version.

posted 07-23-2002 02:08 AM         

E.G.
Transaction 1 : log on to site, buy a single product, log off.
Transaction 2 : log on, browse 5 products, log off.

Using this high level view, you get understanding and buy-in from the business, and they can set realistic targets for tests (hah - I wish!), and understand the language of the results.

posted 07-23-2002 09:08 AM         

quote:

---

Originally posted by JoeW:
Ahh - I think our definitions of "Transaction" is different. I am thinking of a transaction purely from the user perspective (i.e. "business" transactions).

E.G.
Transaction 1 : log on to site, buy a single product, log off.
Transaction 2 : log on, browse 5 products, log off.

Using this high level view, you get understanding and buy-in from the business, and they can set realistic targets for tests (hah - I wish!), and understand the language of the results.

---

You're absolutely right that this is the approach with which you can communicate to the business. When I first started out with load testing I was more focussed on the more technical definition of transactions until I found out that I had a terrible time communicating this to the business. Only when I started to talk in buisness terms (business transactions), progress could be made.

posted 07-24-2002 05:10 AM         

It is part 4, so there will be some reference to earlier articles, but it still makes sense as a stand alone.

Particlularly after the discussion above, I'd love to hear people's reaction to the article.