posted 12-25-2001 07:59 AM           

quote:

---

Originally posted by uttamg:
I would like to know what are the practical differences between Performance and Stress tests & At what stage of software development we perform these tests. Few practical examples will be highly appreciated.

---

Well, this thread should probably be moved to the Performance forum, where it has been answered in numerous forms and to greater and lesser degrees.

Beyond that, I will say, I presented my own opinion of this here:
http://www.qaforums.com/boards/ubb/Forum15/HTML/000101.html

Moving even beyond that, I will say that "stress testing" should be considered part of performance testing. It is not "Performance and Stress Testing" - it is Performance Testing, of which stress is just one aspect of that testing. (Another aspect of performance testing, for example, is load testing. So it would be proper, in my mind, to say "Stress and Load Testing".) Performance testing can, of course, have a wide scope because you can do performance testing at the unit test level and, in the most popular form, at the system test level.

As examples, you can test the timings of individual modules if you wish at a unit test level. Perhaps this is being done because you have algorithms that are, potentially very time intensive, and so you want to make sure that certain algorithms are more efficient, in terms of computing time, than others. System tests, of course, more tend to take the approach of testing actual scenarios (representative and unrepresentative) within the context of the full application - meaning you are passing transactions back and forth via the front-end of the application.

The reason you do full performance testing in the system test phase is because it is at this phase that most integration and functional testing has already been done. That is important because the nature of the integration and overall functionality can change the parameters of the performance tests in terms of the results. In other words, if a function changes how it works or a new element is integrated into the whole application, this can drastically affect the results you get from performance cycles.

Please note that Performance Testing does not necessarily mean "running tests via a tool." Performance Testing should be thought of as covering three basic things: performance modeling, performance analysis and performance simulation. It is the "performance simulation" that is the direct testing via a tool of some sort. Also keep in mind that there is a distinction between Capacity Planning and Performance Testing, although Performance Testing is certainly one aspect of Capacity Planning.

These are all just some of my thoughts.

posted 12-25-2001 11:57 AM         

Uttam.

posted 12-25-2001 08:29 PM           

quote:

---

Originally posted by uttamg:
I am sorry next time I need to search in the forum's database before I put a general question.

---

You have nothing to apologize for. Sometimes it is not always obvious which threads are relevant to what you want to discuss, particularly because the thread title sometimes does not match what was actually discussed in that thread.

I just more wanted to point you to one post that might be helpful. Also bear in mind that the post referenced is biased to my point of view. There are many other posts on the forums on this topic that will give you other opinions as well.

posted 12-27-2001 02:06 AM         

Can you please elaborate more on the distinction between Capacity Planning and Performance Testing.

Prasad

posted 12-27-2001 04:50 AM           

quote:

---

Originally posted by Prasad:
Can you please elaborate more on the distinction between Capacity Planning and Performance Testing.

---

Well, in general, you are going to hear people say that a performance test (in general terms) is a relative measure of the stability and scalability of a given system under expected user/visitor conditions prior to deployment, which aims to ensure that the software performs in accordance with operational specifications as stated in Service Level Agreements or in operationally agreed upon best practices. This is a very valid definition to a large extent.

But it pays to keep in mind, as I said above, that performance testing is broken up into performance modeling, performance analysis and performance simulation. Thus you can model performance, you can analyze it, and you can simulate it under various conditions.

Capacity planning is often, and correctly, defined as a process of predicting when future load levels will saturate a given system and of determining the most cost-effective way of delaying system saturation as much as possible.

To a large extent, you can look at the two practices as being reactive and proactive elements of the same basic concept but that is actually too simplistic a view, especially since performance testing (in the scope I gave it) is part of capacity planning. For example, during capacity planning you might come up with two things: one is a performance model - which is a given system's representation, used for predicting the values of various performance measures of the system. The other is a prediction model - which is a model that, once calibrated, is used to answer "what if" performance prediction questions.

Basically the idea is that capacity planning is just that - a planning exercise. It is meant to work out the ramifications of a given architecture or infrastructure and its ability to handle expected and unexpected resource saturation at various levels. The idea here is to model out a viable infrastructure, usually via some sort of modeling approach. This is where a large amount of SLA's can be worked out. Performance models may be derived at this point at a high-level. Lower-level performance models come in when the actual application (say, a Web site) is placed within the infrastructure and starts using the resources. Performance analysis is done at various stages to see what is happening: in other words, how is the infrastructure responding to the application? Then performance simulation (such as via automated tools) is done to not only confirm the models and the analysis but perhaps to suggest changes to the models and a more realistic updating of the analysis. Notice how we smoothly moved from capacity planning to performance testing?

Of course, it is possible (and is often the case) to do performance testing practices without any actual capacity planning techniques. It is also possible to do just performance simulation (which is what many people refer to when they say "performance testing") without doing any strict performance modeling.

There is, of course, a lot more to it than this but this is just a general overview.

Did this help?

posted 12-31-2001 03:48 PM         

However, I am not sure if this is the "industry standard" since Bean-test is the only test tool I got experience with.

-jason

posted 01-04-2002 04:33 AM         

There is a difference between stress testing and soak testing.

In soak testing I am looking to test for serveral hours, perhaps to model a full day's estimated load. I am looking at whether the logs, journals, etc are big enough. Is the system stable enough to handle surstained usage? This, with luck, will also provide full sized input in to the overnight batches so that I can gauge the batch window too. True this can all be modelled, and there usualy isn't the time in the test window to allow for this simulation, but if the oppertunity is there I like to run at least one test like this. Warning: what is a typicaly day's usage? This is an important question. I once had to test a payroll system where 1 transaction was never used bar 3 days after the pay-slips were issued, when it was the only transaction used (it was the pay-slip enquiry transaction). My tests had to include both scenarios.

Stress testing is an extension of load testing where I increase the load step by step until some component "breaks", e.g. a log fills, no more spare IP ports etc. The idea is to try to spot the weakest link and to put it under Anne Robinson style scrutany. In part this test can be used to validate the capacity model. Recent events on the UK's 1901 census site shows that you need to know where the breaking point of the system is; if it has been designed to handle 1000 hits per minute will the site fail at 1001 or 10,000? Of course, getting the right answer relies on modelling user activity correctly (are they all really hitting search at the same time?). But test design is another topic!

Hope this helps.