posted 07-18-2001 12:04 AM         

My understanding is that inorder to use a regresssion testing tool, the application received (in order to record) should be fairly stable, that means that majority of the bugs should be removed, which implies that the first cycle should be first executed using manual testing and then the automated sequence should start. The second cycle of testing will however be small since recorded tests would be run.

Let me add, the application is handed over to the QA team, as soon as development is over and usually the development team does not perform unit testing, due to time contraints.

Any help would be appreciated
Thanx

posted 07-18-2001 03:58 AM         

When regression testing a stable piece of software, automated testing is faster and more accurate than manual testing (provided it is planned and executed properly).

For a good guide to automated testing I suggest you get Elfreide Dustin's book "Automated Software Testing" and check out some of the white papers available on the web. Here's one to start you off

http://www.stickyminds.com/sitewide.asp?ObjectId=2637&ObjectType=ART&Function=ede tail&tt=nl

posted 07-18-2001 05:04 AM         

Sometime down the rad, you'll get more out of your testing, but it will take some time.

posted 07-18-2001 04:02 PM         

Automated tests require a lot of work to develop, debug, and maintain - they are a software project in themselves with all the attendant software maintenance issues. The payback is limited.

When you have to run the same tests repeatedly and the stimuli and expected results are not going to vary much between test cycles then it makes sense to invest in automating. But this is rarely the case.

Software products are not static - they are constantly being changed/enhanced. Typically, this means test cases must be changed as well. Change implies debugging; over the product's lifecycle you'll find more and more of your time debugging your test scripts....

I've read papers describing test tools that automatically generate test cases based on requirements - the requirements must be translated in a form that are easily parsed and what you get at the other end are test scripts. This sounds great until you look a little closer....

First, you have to faithfully translate the requirements from natural language into something that can be read by a program. Already you can see some problems - errors/ambiguities involved in the translation process. Then there's the matter of requirements management - changes to requirements means changing the tool's input - do you regenerate the test scripts or do you cheat and tweak the scripts manually? Regardless of what you do, you have to still debug the resulting test scripts...

posted 07-18-2001 09:57 PM         

Thanx,
Seema

posted 07-28-2001 10:35 AM         

quote:

---

Originally posted by sahmed:
I have been working in the field of SQA for the last three years. I have, mainly been involved in manual testing. The only tools that I have used are Webload and Winrunner. I have not used winrunner very extensively. From what I have seen, majority of the testing tools are for regression testing and load testing. These days, due to time to market strategies, my company has urged me to use automated tools with the thinking that tools reduce time. Please tell me if this is true and how should i go about it?

My understanding is that inorder to use a regresssion testing tool, the application received (in order to record) should be fairly stable, that means that majority of the bugs should be removed, which implies that the first cycle should be first executed using manual testing and then the automated sequence should start. The second cycle of testing will however be small since recorded tests would be run.

Let me add, the application is handed over to the QA team, as soon as development is over and usually the development team does not perform unit testing, due to time contraints.

Any help would be appreciated
Thanx

---

You might also take a look at www.imbus.de (choose English). This company conducted a one-year study on the ROI of automated software testing. The results are posted on this site.

posted 07-30-2001 05:43 AM         

I have been working with automation tools for just a little over 5 years now. I think there is a huge misconception out there (mostly on the part of management, but quite a few others as well) that you get a tool and you start saving time immediately.

In my limited experience this is just not the case. It takes quite a bit of time and effort to automate. You have to do all the planning that you would normally do, then you have to assess the tests for automation candidates. After that you have to evaluate the candidate tests against the software to determine whether or not the software is stable enough (changing requirements, addressing problems - can conspire to break an automated test case).

Then you have the automation itself. If you are fortunate enough to work with someone who knows the tool then you will probably not experience the frustrations that have caused me many grey hairs

Finally there is the maintenance aspect. Small development changes can impact automated tests in huge ways! Development needs to include test in subsequent code change reviews - so the tester can figure out how the automated tests will need to change.

I am a huge fan of automation (in case this post sounded like I wasn't). I want my tests to be as repeatable as possible. But I do always try to convey to management that, while automated tests take less time to run than manual tests, the time savings is something they should look for in the future.

posted 07-30-2001 07:29 AM         

quote:

---

Originally posted by MelR:

I have been working with automation tools for just a little over 5 years now. I think there is a huge misconception out there (mostly on the part of management, but quite a few others as well) that you get a tool and you start saving time immediately.

In my limited experience this is just not the case. It takes quite a bit of time and effort to automate. You have to do all the planning that you would normally do, then you have to assess the tests for automation candidates. After that you have to evaluate the candidate tests against the software to determine whether or not the software is stable enough (changing requirements, addressing problems - can conspire to break an automated test case).

Then you have the automation itself. If you are fortunate enough to work with someone who knows the tool then you will probably not experience the frustrations that have caused me many grey hairs

Finally there is the maintenance aspect. Small development changes can impact automated tests in huge ways! Development needs to include test in subsequent code change reviews - so the tester can figure out how the automated tests will need to change.

I am a huge fan of automation (in case this post sounded like I wasn't). I want my tests to be as repeatable as possible. But I do always try to convey to management that, while automated tests take less time to run than manual tests, the time savings is something they should look for in the future.

---

Mel,
It is for exactly these reasons why I developed the Automated Software
Testing book. My coauthors and I made a painstaking effort within the
book to articulate such things as the benefits and pitfalls of automated
testing, a structured approach for making the decision to automate, test
automation return on investment analysis, guidance to help overcome misconceptions
about automation on the part of project managers, testers, and development
personnel, the use of a detailed test tool evaluation checklist and a
structured process for introducing automated testing.
The Automated Software Testing book is filled with guidelines to
help the test professional prevent uninformed and rushed decisions concerning the use of test automation.
Thanks,
Elfriede

posted 07-30-2001 10:22 AM         

This is still in the very early stages, but the concept has many people very interested.

posted 07-30-2001 11:47 AM         

quote:

---

Originally posted by Elfriede Dustin:
Mel,
It is for exactly these reasons why I developed the Automated Software
Testing book. My coauthors and I made a painstaking effort within the
book to articulate such things as the benefits and pitfalls of automated
testing, a structured approach for making the decision to automate, test
automation return on investment analysis, guidance to help overcome misconceptions
about automation on the part of project managers, testers, and development
personnel, the use of a detailed test tool evaluation checklist and a
structured process for introducing automated testing.
The Automated Software Testing book is filled with guidelines to
help the test professional prevent uninformed and rushed decisions concerning the use of test automation.
Thanks,
Elfriede

---

Ahh Elfriede,

Surprisingly I have not yet heard from the publisher. Any day now I am expecting a response to my complaint about the cost of these books. I'm just certain that, not only will they discount these publications by 80-90%, they will also send me 10 free copies for taking the time and trouble to clue them in! I also think it's within the realm of possibility that they will offer to fund my retirement if I would only consider a consulting offer with them to review the prices of all future publications.

Mel <---Plans to hold out for a signing bonus as well

posted 07-30-2001 11:48 AM         

quote:

---

Originally posted by Elfriede Dustin:
Mel,
It is for exactly these reasons why I developed the Automated Software
Testing book. My coauthors and I made a painstaking effort within the
book to articulate such things as the benefits and pitfalls of automated
testing, a structured approach for making the decision to automate, test
automation return on investment analysis, guidance to help overcome misconceptions
about automation on the part of project managers, testers, and development
personnel, the use of a detailed test tool evaluation checklist and a
structured process for introducing automated testing.
The Automated Software Testing book is filled with guidelines to
help the test professional prevent uninformed and rushed decisions concerning the use of test automation.
Thanks,
Elfriede

---

Ahh Elfriede,

Surprisingly I have not yet heard from the publisher. Any day now I am expecting a response to my complaint about the cost of these books. I'm just certain that, not only will they discount these publications by 80-90%, they will also send me 10 free copies for taking the time and trouble to clue them in! I also think it's within the realm of possibility that they will offer to fund my retirement if I would only consider a consulting offer with them to review the prices of all future publications.

Mel <---Plans to hold out for a signing bonus as well

posted 08-27-2001 07:14 AM           

quote:

---

Originally posted by MelR:
Any day now I am expecting a response to my complaint about the cost of these books.

---