|
Author
|
Topic: why automate?
|
fk666
Member
Posts: 16
Registered: Sep 2001
|
posted 10-03-2001 02:16 PM
    
Our testing department currently principally carry out manual testing. We have developed some tools ourselves to automate some of our testing, but we don't use any 'off-the-shelf' tools (e.g. Winrunner). There are lots of questions being raised as to why we don't use these. I don't see these tools as the answer to improving our testing, however other people see automated tools as the 'holy grail'. I'm looking to produce a document outlining what testing we do using our in-house tools do, but also the advantages and disadvantages of using off-the-shelf automated tools, and whether or not these would fit in with the way our department and company works. Has anyone done anything similar?
Does anyone know of any resources which could help?
Has anyone been in the same situation and decided that automated tools are not an option?
When do automated tools not help?
Has anyone bought these tools and have them end up as shelfware? Thanks in advance! ------------------
|
Gilbert
Advanced Guru
Posts: 691
Registered: Jul 1999
|
posted 10-03-2001 04:39 PM
If your AUT (Application Under Test) is a typical Client/Server or Web application, NOT something like a Voice Recognition software, and you can visualize that by automating some of your tasks, such as entering the same data over and over to populate your database, you may cut down the time spent from 5 days to 1 hour, for example, why not automate?
If you think that you or someone in your group is capable of learning and using these automated test tools properly and your employer has the budget for automation, why not automate?
Maybe you should consider evaluation of some of these test tools and compare them with the tools you developed in-house!? If you see that the other tools can do more for you, then maybe you should consider switching.
When does automation tool not help?
- when your AUT is written in Visual FoxPro, for example
- when your developers do not understand the importance of having a stable GUI (they keep making many changes in GUI each release to QA)
- when your AUT has a lot to do with Video/Audio stuffs or drawings/bitmaps and time and resources are very limited------------------
|
CrazyQABoy
Advanced
Posts: 121
Registered: Sep 2001
|
posted 10-03-2001 05:12 PM
quote:
Originally posted by Gilbert:
- when your developers do not understand the importance of having a stable GUI (they keep making many changes in GUI each release to QA)
I agree with Gilbert on this issue. This to me is the number one issue when it comes to automating. The slightest change to the interface could mean hours and hours of work changing the automation script. In order to automate you better make sure that the developers are willing to work with you, if not it is hopeless  ------------------
Don He who laughs last...... obviously didn't get the joke!!
|
jstrazzere
Moderator
Posts: 2134
Registered: May 2000
|
posted 10-04-2001 04:22 AM
quote:
Originally posted by fk666:
When do automated tools not help?
- When the users of the tool aren't capable of learning how to use the tool (because they aren't given the time or budget, because they don't have enough expertise, whatever) - When the testers aren't able to test the application well manually, introduction of a tool will often not help - When the testers don't want to use a tool, there is a much lower chance of success ------------------
- Joe (strazzerj@aol.com)
|
testgeek
Advanced Guru
Posts: 829
Registered: Feb 2001
|
posted 10-06-2001 06:13 PM
this article will also offer some insight:Lessons in Test Automation
By Elfriede Dustin
www.stickyminds.com/sitewide.asp?sid=663174&sqry=%2AJ%28MIXED%29%2AR%28createdate%29%2AK%28simplesite%29%2AF%28dustin+automate%29%2A&sidx=2&so pp=10&ObjectId=1802&Function=DETAILBROWSE&ObjectType=ART
[This message has been edited by testgeek (edited 10-06-2001).]
|
AprilS1
Member
Posts: 7
Registered: Oct 2001
|
posted 10-08-2001 11:46 AM
fk666,
I would like to know more about the automation that you have currently done in-house:
- what is your system under test (i.e. n-tier web, n-tier client/server, middleware, etc...)?
- what is your environment (windows, unix, mutli-platform)?
- what does the automation you have written do and what was it developed using?My company is struggling along and won't spend the 4 to 5 grand on a tool. I'm trying to find any alternative solutions. April
|
jgottlieb
Moderator
Posts: 1442
Registered: Apr 2001
|
posted 10-08-2001 01:47 PM
April, are you at the S1 office in Charlotte, NC? I thought about sending my resume there last year...------------------
"C'mon now, you know cheddar doesn't agree with you." - Capt. Archer to Porthos
Jordan Gottlieb
Qualitech Solutions, Inc.
jgottlieb@qualitechsolutions.com
|
jstrazzere
Moderator
Posts: 2134
Registered: May 2000
|
posted 10-09-2001 04:46 AM
quote:
Originally posted by AprilS1:
My company is struggling along and won't spend the 4 to 5 grand on a tool. I'm trying to find any alternative solutions.April
April, You didn't mention the type of test tool you would need. Windows? Web? VB? Java? Many general automation tools are much less expensive than test automation tools, and can provide many of the same benefits at a fraction of the cost. If you cannot spend 4-5K, how much could you spend?
------------------
- Joe (strazzerj@aol.com)
[This message has been edited by jstrazzere (edited 10-09-2001).]
|
sahmed
Advanced
Posts: 173
Registered: Jul 2001
|
posted 10-09-2001 11:27 PM
quote:
Originally posted by CrazyQABoy:
I agree with Gilbert on this issue. This to me is the number one issue when it comes to automating. The slightest change to the interface could mean hours and hours of work changing the automation script.
Most of the time, the application that i get is very unstable, loads of forms crash and etc ... how can u auomate such tests????
------------------
|
QAGirl
Moderator
Posts: 2424
Registered: Aug 2001
|
posted 10-10-2001 06:20 AM
The key to setting up automated tests in unstable environments is not to.  Let me clarify that. If your environment is constantly that unstable, you have other issues that should probably be addressed from a QA standpoint with regard to process, release cycles, etc. If there are only portions of the apps that are unstable, and they become more stable over time, you need to determine at what point they are stable enough to automate.Don, I agree that you need developers to work with you, but the 'slightest' change to a UI should not indicate hours of work, unless it also means a major change in functionality. Define your scripts in small, componantmentalized (is that even a word??) format. Have a log in script, a search script, a script for each small functionality, and string them together. Write Data Driven scripts... use a common GUI map (speaking MI here). There are ways to define your automated scripts so that there is 'less' impact when changes are made that you're not informed of. That being said, there are always risks associated with automation. You *do* need the buy in of development and the rest of the company, because you can design parts of the QA and Dev cycle to work more efficiently with the fact that you're going to automate. ------------------
|
AprilS1
Member
Posts: 7
Registered: Oct 2001
|
posted 10-10-2001 04:37 PM
To jgottlieb, Sorry, but no. S is my last initial and I had already registered here once and can't find the password, so I added the 1 to my new reg:-) I live in Washington State. To jstrazzere, Our application is an n-tier web application running on Win2K, SQL2K, with COM objects developed in VB for most of the business logic. The newest add on application is a Java App running on Win2K with SQL2K backend. They would prefer some sort of automation with $0. Since automation is long term ROI, they aren't interested, they want to see short term ROI. April ------------------
|
dmatharu
New Member
Posts: 5
Registered: Oct 2001
|
posted 10-10-2001 04:47 PM
I read most of the responses. I agree with some but my main concern is that a lot of the people have taken automation as the only thing that will find and cure the software quality.
1st and for most if we are going to run the same tests and the results show that no bugs were detected. Are we to assume that the software has reached zero defect level? Or are we just testing the same piece of code which does not have any bugs?
In my opinion we need to have an intelligent way of using automation. I prefer automating for the regression cycle in that way I know that the bugs are not being reintroduced.
|
dmatharu
New Member
Posts: 5
Registered: Oct 2001
|
posted 10-10-2001 04:48 PM
I read most of the responses. I agree with some but my main concern is that a lot of the people have taken automation as the only thing that will find and cure the software quality.
1st and for most if we are going to run the same tests and the results show that no bugs were detected. Are we to assume that the software has reached zero defect level? Or are we just testing the same piece of code which does not have any bugs?
In my opinion we need to have an intelligent way of using automation. I prefer automating for the regression cycle in that way I know that the bugs are not being reintroduced.
|
jstrazzere
Moderator
Posts: 2134
Registered: May 2000
|
posted 10-11-2001 04:06 AM
quote:
Originally posted by AprilS1:
Our application is an n-tier web application running on Win2K, SQL2K, with COM objects developed in VB for most of the business logic. The newest add on application is a Java App running on Win2K with SQL2K backend. They would prefer some sort of automation with $0. Since automation is long term ROI, they aren't interested, they want to see short term ROI. April
It's sad to see such short-sightedness. Maybe as the organization matures, they'll learn. OK, so they want short-term ROI, but the "I" here is $0? I guess you don't need much "R" to justify that! Well, I suppose the best you can do under the circumstances is to search for freeware. Try to find something QA does repeatedly, that takes a fair amount of time. Try to find something that would be useful to shorten. Then look for a tool to help automate part of it. For example, where I work we often have tests to do that require us to place hundreds or thousands of "orders". If we had to enter them manually, it would take forever. We automated that process so it only takes minutes, with no manual intervention. In your case you could look at macro recorders. Some are free. Some very useful test tools can be built using VB or Java, if they are available and the time and knowledge exists. If you are ready to do load testing, look into something like OpenSTA. If you have any luck with freeware, you may be able to point to your successes and explain to management how much more you could do if they were willing to invest more than $0. Good luck. ------------------
- Joe (strazzerj@aol.com)
|
kMcLeod
Member
Posts: 77
Registered: May 2001
|
posted 10-11-2001 07:08 AM
Hi April,
Now, I have not read the book nor do I use VB for automation, at least not yet , but this may be a place for you to start if you already have VB in your company.
http://www.vbfortesters.com Hope this helps,
Keith ------------------
|
