posted 10-06-2002 01:26 PM
I think the following quote represents a tempting, but dangerous ultra-simplicity. One might run off and try to start testing, and soon realize he knows nothing about what he's doing or why:
quote:
Volume Testing = Large amounts of data
Load Testing = Large amount of users
Stress Testing = Too many users, too much data, too little time and too little room
For any testing having characteristics sufficiently distinct to warrant a specific name and category, there are at least 2 important, general things to know about it:
1. The purpose of the testing
2. How the testing is set up and executed
The terse descriptions above describe half the picture, i.e., the characteristics of volume and stress testing. But they explain nothing about the purposes. The purpose or intention behind the different types of testing is usually what drives the characteristics of the test, not the other way around.
"Volume testing" is a term given and described in Glenford Myers' The Art of Software Testing, 1979. Here's his definition: "Subjecting the program to heavy volumes of data. The purpose of volume testing is to show that the program cannot handle the volume of data specified in its objectives" - p. 113.
Myers tries, unsuccessfully in my opinion, to differentiate volume and stress testing as truly distinct:
"Stress testing: subjecting the program to heavy loads or stresses. This should not be confused with volume testing; a heavy stress is a peak volume of data encountered over a short span of time" - pp. 113-114.
In my opinion, volume testing may be intended as a species of performance test, as another poster alluded -- for example, consider performance requirements (by "performance requirements" I mean non-functional requirements in terms of throughput, response time or resource utilization) for a database or network. That is if 'heavy volumes of data' are your concern you're probably not just concerned with volume in itself, because we usually want data to be processed at some rate or stored in some ratio to resource capacity. We're usually concerned with volumes of data in relation to something interesting. So, amount of data per unit of time (throughput) or efficiency and speed of data processing (resource utilization and response time) in relation to "heavy volume" are what we really care about.
On the other hand, it can be done as a stress or reliability test with the purpose of revealing faults such as resource leaks or data corruption, since "heavy volumes of data" is one way to exert stress on a system, and heavy load over a long period of time is a test of reliability. In which case we aren't concerned with or measuring performance metrics.
Finally, there is the relationship of data volume workloads to software or hardware capacity specifications, which would seem to demand functional testing. That is, a "heavy volume of data" (note how this is a relative term) might be exactly what the application is supposed to handle, in which case the test is of whether the application "handles it" successfully or not is simply a verification of functionality. Perhaps in this is way, volume testing can truly be differentiated as a specialized functional test.
In any case, some kind of load generator is required and so is work load characterization, so in my vocabulary (see the Sept., 2002 "Quality Techniques Newsletter" at www.soft.com if interested), volume testing could belong under the broad category of "load test," if anywhere. Right now I am thinking that it is not sufficiently well defined or widely used enough to be established as a distinct type of test (Myers not withstanding). I listed it as an alternate term for "stress test" in my article, but if anyone here has a strong argument for where it should stand in relation to other types of testing and why it deserves its own separate category, I would be very interested to hear it. There is a lot of variety in the literature with respect to different terminology and types of load, performance and functional testing, so don't expect to find a universal answer for volume testing. I suspect the origin of the term is with Myers.
One last thought is that it sounds as though the original poster is unsure about what volume testing is or how to do it:
quote:
Can any body send some info about volume testing [. . .] We would like to do volume testing.
So he should find out what was meant by the source where he got the term in the first place -- was it a manager? Team lead? A software book? A test tool menu item? Again, because much QA terminology is so variable, poorly understood, and liable to misuse (although misuse implies that a proper and stable sense is available to be misused), it might be best to put the term itself on the backburner, and find out what the user of the term meant by it. The risk is possibly to reveal a little ignorance, but at least one would have a clear idea of what is expected. You can always find the proper name for it down the road. :->
Scott Stirling
[This message has been edited by scottqae (edited 10-06-2002).]
posted 10-02-2002 08:28 AM
