posted 11-02-2001 07:52 AM         

quote:

---

Originally posted by Elaine:
I am wondering if there is guidelines on when it would be best to start using an automated test tool for regression testing.

Our product is new on the market for about 1 year. I am not sure if there are guidelines on when the best time is to start using an automated test tool for regression testing.

---

If it were me, I would have started at least 1 year ago.

IMHO, you should start automating before the product is complete. And certainly, before the product is on the market.

posted 11-02-2001 12:44 PM         

If its a GUI app, where not only does the Look and feel change, but controls and fields appear/disappear at will, then I'd say wait for things to firm up a bit before doing a lot of your GUI automation. But who is to say that that GUI will not radically change down the line, incurring a hefty maintenance cost?

If it is more along the lines of API testing, I'll say you can start writing your automated tests and create teh framework even before dev has completed its code.

If you have a solid spec, you can also start automation here right away as well. The better defined your final product is, the sooner you can start automation.

But don't lose sight of my number one idea.... it all starts with a good test plan. Without one, automation efforts have a greater chance of being wasted, spinning your wheels in no-mans-land. Define what you need it to do, and why it needs to be done. Then think about how you can implement your solution.

I've been pretty lucky on my latest project where we send an XML stream to another computer based on some well-defined query language. My tests and verification routines were done in a week and a half. Well before I got code from dev. Anytime they make a change in the stored procs, I still can verify that the results returned in the XML stream make sense in less than 5 mins. And this is for several hundred test cases. Powerful indeed. But only because the product is ultra well defined, and I had a kick-a$$ test plan to work from.

Good luck!

posted 11-06-2001 05:54 PM         

quote:

---

Originally posted by jstrazzere:
If it were me, I would have started at least 1 year ago.

IMHO, you should start automating before the product is complete. And certainly, before the product is on the market.

---

Is this realistic?

It seems that even when we just have new features it's just not efficient to try and automate before the feature is released. We will typically test manually the first time around for a couple of reasons:
1. It's more difficult to automate something that does not exists (exaggeration). But really, there's a lag in time before you can start no matter how proactive you are.
2. Automated testing can only find problems you know to look for (regression issues). Manual testing gives you an opportunity to see things you aren't necessarilly looking for.

I guess my advice would be to start when the product is released with the intention of using automated tests to test the second version. I haven't worked on a version 1 project, but I can't imagine you have extra time to get going on automation before it's even out the door. Any thoughts?

------------------

posted 11-07-2001 04:21 AM         

quote:

---

Originally posted by BrianCarroll:

Is this realistic?

It seems that even when we just have new features it's just not efficient to try and automate before the feature is released. We will typically test manually the first time around for a couple of reasons:
1. It's more difficult to automate something that does not exists (exaggeration). But really, there's a lag in time before you can start no matter how proactive you are.
2. Automated testing can only find problems you know to look for (regression issues). Manual testing gives you an opportunity to see things you aren't necessarilly looking for.

I guess my advice would be to start when the product is released with the intention of using automated tests to test the second version. I haven't worked on a version 1 project, but I can't imagine you have extra time to get going on automation before it's even out the door. Any thoughts?

---

Of course it's realistic.
In most cases, software is available for testing before it is complete (and certainly before it reaches the market the first time).

In many environments, builds are done daily - with each build (hopefully) increasing the amount of viable, testable code. It's this increasing amount of code that can be tested with automation.

Proper planning can get the important pieces of software built earlier in the project - allowing time to create and take advantage of test automation.

From your posting, it sounds like you believe that you must choose between automated testing OR manual testing. In my experience, I try to always include BOTH.

If you choose to think of test automation as "something we'll do when we have extra time", then it will never get done. Instead if you think of automation as "something we do so we can do a better job, and save some time later on" then it is more likely to be performed.

I have worked on many Release 1 projects. In virtually all of them, I have been able to take advantage of some test automation long before the product went to market. In fact, I don't know if we could have met our schedules without it.