posted 01-30-2002 02:57 AM         

A) What are the things i should be looking for while doing load testing an app using Load Runner or Silk Performer not yet decided
b) Also i have scripting exp using Silk/WR/QARun Is coding as flexible for L0ad testing as it is for Functionality testing???

To give you a preview I ma supposed to load tets and intranet application where in there will be a solaris server with all client using IE
The middle layer will probably use WebLogic and EJB
and a sybase and or Oracle database
Awaiting lots of inputs

posted 01-30-2002 07:32 AM         

quote:

---

Originally posted by devdreadme:
Hi Guys I am venturing in load Testing Zone for the first time and i want to know

A) What are the things i should be looking for while doing load testing an app using Load Runner or Silk Performer not yet decided

---

This needs to come from project management. What are the expectations of the application? Does it handle the number of users expected? How many users can it handle? Is there memory problems, configuration problems? Does the application respond fast enough? These are the questions load testing can help answer, but the determining factors have to come from the people requesting or making the product.

quote:

---

b) Also i have scripting exp using Silk/WR/QARun Is coding as flexible for L0ad testing as it is for Functionality testing???

---

My only experience is with LoadRunner, so I can only respond for that. LoadRunner uses ansci C so it is really easy to code for if you know C, C++, Java, etc.

Hope this helps.

------------------
Simple minds, Simple thoughts!

posted 02-06-2002 11:36 AM         

Essentially, what you need to do is run the tests and present the findings to the appropriate people. What will then probably happen is that you will be the one driving the requirements and not them.

posted 02-07-2002 09:33 AM         

quote:

---

Originally posted by mgiac76:
That's the same problem we are running into. We already invested in a load running tool, but the main problem is that our lead architects, database developers, and systems people have no idea what they want the benchmarks to be.

Essentially, what you need to do is run the tests and present the findings to the appropriate people. What will then probably happen is that you will be the one driving the requirements and not them.

---

Sorry, I have to disagree here. Requirements for testing should never come from the tester. It needs to come from the people designing the project, the customers, the marketers, etc. They need to set the goals of your testing. If a DBA does not know what to look for in a database to tell if there are problems perhaps they need some trainging. Same with your system administrator, and your network administrator.

The only involvement I like to have in setting the goals of testing is to make sure the goals are written out properly and are well defined. After that, it is my responsibility to create and perform a test or tests that will meet the goals' expectations.

How could I possibly know what to test in an application without any stated goals/requirements?

------------------
Simple minds, Simple thoughts!

posted 02-08-2002 07:37 AM         

quote:

---

Originally posted by JoeW:
It's nice to be given criteria for testing and then just get on with it, but rarely seems to happen! I find you need to provide some kind of guidance as to what tests might be necessary (typical load, stress/break tests etc.). My first approach is always to try to get expected real world transaction rates and use these as the basis of performance testing.

---

I understand this. I have been in the same situation, I think we all have. Part of the problem is educating the proper people about load testing, what it is, what it can do, etc. Then they will be able to be more of a help. Think of it this way, if a person in marketing came to you and asked how what they needed to create a marketing report for a project, would you know what info they need? Probably not. They are in the same boat, so it is part of the job to help them understand what we are doing and what we need from them. Also by you using real-life data to create tests, you are not merely making a test up. You are using data available to make a test. Now you need to find the people who can tell you once you have run the test, whether it was successful or not.

------------------
Simple minds, Simple thoughts!

posted 02-10-2002 11:31 PM         

If you release a report describing what you have tested each week and start showing it to management and other stakeholders then they will poke holes in your assumptions about the requirements and the requirements will eventually emerge from the group of stakeholders.

Write a report once a week saying what you tested and what happened and hold a regular meeting with management and be prepared to be wrong. It's better to be wrong early than wrong too late.

-Richard Perfect.

posted 02-12-2002 07:33 AM         

quote:

---

Originally posted by rperfect:
The thing that I have found helpful when dealing with a lack of requirements is to publish a "Testing Report" report early and on a regular basis.

If you release a report describing what you have tested each week and start showing it to management and other stakeholders then they will poke holes in your assumptions about the requirements and the requirements will eventually emerge from the group of stakeholders.

Write a report once a week saying what you tested and what happened and hold a regular meeting with management and be prepared to be wrong. It's better to be wrong early than wrong too late.

-Richard Perfect.

---

Although I do not agree with this method, I do see how it can be useful. By pointing out the system is flawed to the upper management , hopefully they will work on changing the system. This is risky as well I think. Could show in upper management's eyes that you do not know what you are doing, that this type of testing does not work, or QA does not know what they are doing. Also you waste a lot of time, potentially, testing and writing reports that are basically useless. The best approach is to get the right people together and create a plan and line of communication.

------------------
Simple minds, Simple thoughts!

posted 04-30-2002 11:00 AM         

1. Duration of garbage collection on the WebLogic server should be less than one second. If GC takes more than one second, then you may be adversely affecting overall response times. The server should be tuned to collect more frequently.

2. Overall memory usage on web, app, or database servers should not show an increase over time. This points to a memory leak which may accumulate over time until you get out of memory errors. (Slight memory leaks are not always detectable in a short load test, so a longer duration is often desirable.)

3. At the OS level, TCP close rate should be tuned so that the duration is not too long. A shoter close rate will allow sockets to close soon enough to allow new connections to be made. Typically the default setting is more appropriate for a database, which requires longer connections, than what would be appropriate for an application or web server.

posted 05-01-2002 07:56 AM         

Thanks
Imad

posted 05-01-2002 08:50 AM         

quote:

---

Originally posted by imad:
Now I have a question for Marcy:
I worked with WebObject on a project and it was my first time to deal with it. We had a consultant who was a WebObject expert and he always was pushing for high number of Garbage Collection. He went as high as 20 seconds. So my question is there a standard for setting GC on WebObject because I see a big discrepancy between what you suggested and what he suggested. To be honest with you the Appl was not performing as good as the IIS environment…

Thanks
Imad

---

Imad,

The number of garbage collections is much less critical than the duration of the garbage collection itself. Duration is measured from the time the server begins collecting to the time it finishes collection. This duration should be kept extremely short because while the server is collecting, it basically does not serve any other requests.

For example, if you have collections that last 5 seconds, your response times would increase by at least 5 seconds just at that layer of the architecture.

So, if you find that the duration of your collection is too long, then you do want to tune your server to collect more frequently. This will reduce the amount of memory it has to clean up with each collection, thereby reducing the duration of collection.

I would think that your consultant may have been advocating GC frequency of 20 seconds, but probably not that the actual duration of collection be 20 seconds.

posted 05-01-2002 02:58 PM         

While doing all the test you should keep track of the performance counters like CPU usuage, I/O, Memory usuage on all the servers invloved and through put and hits per second on the web server. Make sure while creating the scenarios build them as close as possible to the real world situation like if it is a e commerce site the load should be scattered and if is a trading application or something then the users should be mostly synchronized before doing the transaction. But always remember no matter how sophesticated your scenarios are its not possible to get them exactly as it works in the real world and also load testing is based on lots of assumptions and considerations.

Regarding the scripting if you worked on WR you should be very comfortable to deal with LR.

Hope this helps and Good Luck with your testing.

------------------
QualityServ