Tag Archives: Testing

[Slide] Automated Web Testing with Selenium for beginners

6 Jun

You can find useful information for this presentation.

1. What is Selenium?

– Test tool for web applications
– Runs in any mainstream browser
– Supports tests in many languages
– Selenese (pure HTML, no backend required)
– Java, C#, Perl, Python, Ruby
– Record/playback (Selenium IDE)
– Open Source with corporate backing
– Lives at selenium.openqa.org

2. Demo

– Record a test in Selenium IDE
– Show same test written in Java

You can download in link

Source from Erik Doernenburg, ThoughtWorks


[Selenium] The Magic of Testing – Presentation

5 Jun

This presentation includes

1. What is Selenium
2. Why Selenium
3. Testing using record/playback and scripting tool
4. Selenium Grid
5. Benefits
6. The Problem
7. Conclusion

Source: http://www.uet.edu.pk/

Download in link

[Ebook] Selenium 1.0 Testing Tools Beginner’s Guide

2 Jun

The Selenium 1.0 Testing Tools Beginner’s guide shows developers and testers how to create automated tests using a browser. You’ll be able to create tests using Selenium IDE, Selenium Remote Control and Selenium 2 as well. A chapter is completely dedicated to Selenium 2. We will then see how our tests use element locators such as CSS, XPath, and DOM to find elements on the page.

Once all the tests have been created we will have a look at how we can speed up the execution of our tests using Selenium Grid.

Download in link


31 May

The testing process has these 4 steps :
• Creating a testplan (if you are using the testplan editor)
• Recording a test frame.
• Creating testcases
• Running testcases and interpreting their results.
Creating a testplan
If the testplan editor is used, the automated testing process is started by creating a testplan. A basic testplan is structured as a hierarchical outline and contains:
• Descriptions of individual tests and groups of tests. As many levels of description can be used.
• Statements that link the test descriptions in the plan to the 4Test routines, called testcases, that accomplish the actual work of testing.
Recording a test frame
Next, record a test frame, which contains descriptions, called window declarations, of each of the GUI objects in your application. A window declaration specifies a logical, cross-platform name for a GUI object, called the identifier, and maps the identifier to the object’s actual name, called the tag. In addition, the declaration indicates the type of the object, called its class.
Creating testcases

The 4Test commands in a testcase collectively perform three distinct actions :
• Drive the application to the state to be tested.
• Verify the state (this is the heart of the testcase).
• Return the application to its original state.
The powerful object-oriented recorder can be used to automatically capture these 4Test commands to interact with the application, or to white the 4Test code manually if one is comfortable with programming languages. For maximum ease and power, these two approaches can be combined, recording the basic testcase and then extending it using 4Test’s flow of control features.
Running testcases and interpreting results
Next, run one or more testcases, either by running a collection of scripts, called a suite, or, if you are using the testplan editor, by running specific portions of the testplan. As each testcase runs, statistics are written to a results file. The results file and its associated comparison tools allow you to quickly pinpoint the problems in your application.
A Test Frame
The test frame is the backbone that supports the testcases and scripts. It is a file that contains all the information about the application’s GUI objects that Silk Test needs when you record testcases. This information minimally consists of a declaration for each GUI object, but can also include any data that you want to associate with each GUI object, as well as any new classes and methods that you want to define.
A window declaration specifies a cross-platform, logical name for a GUI object, called the identifier, and maps the identifier to the object’s actual name, called the tag. Because the testcases use logical names, if the object’s actual name changes on the current GUI, on another GUI, or in a localized version of the application, only the tag in the window declarations need to be changed; don’t need to change any of the scripts. Variables, functions, methods, properties can be added to the basic window declarations recorded by Silk Test.
To record declarations for the main window and menu hierarchy of your application.

1. Start up your application and Silktest.
2. Select File / New. The New dialog appears.
3. Select the Test Frame radio button and click OK. The New Test Frame dialog is displayed, allowing to create a test.
4. Frame file for an application displayed in the Application list box.
5. Select the application fro the Application list box. If a Web application is tested, different fields are seen.
6. Click OK. The new test frame file is created. The file contains the 4Test declarations for the main window and all its menus, as well as a generic declaration that is valid for each of the standard message boxes in the application.

A Test Plan
A Testplan is made up of a large amount of information, a structured, hierarchical outline provides an ideal model for organizing and developing the details of the plan. A testplan consists of two distinct parts
• An outline that describes the test requirements
• Statement that connect the outline to the 4Test scripts and testcases that implement the test requirements.
Using the testplan, we can create and run tests.

To start a new testplan :
• Select File / New
• Select Testplan and click OK.
An empty testplan window opens.

A Test Suite
A Test Suite is a collection of test scripts. Consider a case that we are having a set of script (.t) file. If we want to run these scripts against our application, we have to select the required testcase or we have to run the entire script file. But after the completion of that script file, the user has to manually change that to the next script file to run those testcases available in that script. Instead of that silktest provides a way to continuously select a set of script files and run those script files at-a-stretch. This can be done by creating a new Test Suite file and declare the needed script files in that suite file.
To start a new test suite:

1. Select File / New.
2. Select Test Suite and click OK.
3. In that suite file enter the script file names to run continuously.
4. Save that script file.
5. Compile the script file and run it.
6. Now the process of running the script will not stop after the completion of the first script file, instead of that it will automatically pass to the next script file and run the testcases available there.
Assume a case where there is a folder called silk scripts in c drive with five test script files. Here in the suite file, we are calling all the script files instead of running those script files separately. The suite file will look like as given below :
use c:\silkscripts.\script.t” use c:\silkscripts.\script2t” use “c:\silkscripts./script3.t” use “c:\silkscripts\script4.t” use “c:\silkscripts\5.”

A Test script

A testscript file contains various testcases for various test conditions.

A Testcase
In a script file, a testcase ideally addresses one test requirement. Specifically, a 4Test function that begins with the testcase keyword and contains a sequence of 4Test statements. It drives an application to the state to be tested, verifies that the application works as expected, and returns the application to its base state.
• In the silktest tool, select the File -> Now option the menu bar.
• In the resulting dialog box �New�, there will be options for selecting different kind of files.
• Select the file type 4� test script� option.
• It will open a new script file.
• Before start writing the testcase, declare the necessary file that is to be used in that script file.
• Start with the keyword �testcase� followed by the testcase name. The name of the testcase is whatever as selected by the user. But make sure that by looking at the name of the testcase, the objective of the testcase should be understandable.
• Start the tests from the scratch so that the silktest will start eh application and to the testing from the base state.
• Use necessary conditions / loops if necessary.
• At the end of each and every script, print a statement to know whether the test case has achieved its objective or not. The user can make sure that the particular part of the application is error free by looking at the message you print.
• Try to make the testcase effective and time consuming (say) by keeping the second tests continue from the place where the first test finishes.
• A sample testcase for registering into the yahoo mail.
Testcase registration ()
• Browser.LoadPage “mail.yoo.com”)
• Sign In YahooMail.SetActive ()
• SignIn Yahoo.Mail.objSingIn YahooMail.SignUp.Now.Click()
• Sleep (3)
• WelcomeTo Yahoo.Set Active
• Welcome To yahoo.objWelcomeToYahoo.LastName.SetText(“lastname”)
• Welcome To Yahoo.objWelcomeToYahoo.LanguageConten1.Select(5)
• WelcomeTo Yahoo.objWelcome ToYahoo.ContactMeOccassionally About.Click ()
• Welcome To Yahoo.objWelcome To Yahoo.Submit ThisForm.Click()
• If Registration Success.Exists ()
• Print (“Test Pass”)
• else
• logerror (“Test Fail”)

Silk Test – Introduction

31 May

Silk Test is a tool specifically designed for doing REGRESSION AND FUNCTIONALITY testing. It is developed by Segue Software Inc. Silk Test is the industry’s leading functional testing product for e-business applications, whether Window based, Web, Java, or traditional client/server-based. Silk Test also offers test planning, management, direct database access and validation, the flexible and robust 4Test scripting language, a built in recovery system for unattended testing, and the ability to test across multiple platforms, browsers and technologies.
You have two ways to create automated tests using silktest:

1. Use the Record Testcase command to record actions and verification steps as you navigate through the application.
2. Write the testcase manually using the Visual 4Test scripting language.

1. Record Testcase
The Record / Testcase command is used to record actions and verification steps as you navigate through the application. Tests are recorded in an object-oriented language called Visual 4Test. The recorded testreads like a logical trace of all of the steps that were completed by the user. The Silk Test point and click verification system allows you to record the verification step by selecting from a list of properties that are appropriate for the type of object being tested. For example, you can verify the text is stored in a text field.

2. Write the Testcase manually
We can write tests that are capable of accomplishing many variations on a test. The key here is re-use. A test case can be designed to take parameters including input data and expected results. This “data-driven” testcase is really an instance of a class of test cases that performs certain steps to drive and verify the application-under-test. Each instance varies by the data that it carries. Since far fewer tests are written with this approach, changes in the GUI will result in reduced effort in updating tests. A data-driven test design also allows for the externalization of testcase data and makes it possible to divide the responsibilities for developing testing requirements and for developing test automation. For example, it may be that a group of domain experts create the Testplan Detail while another group of test engineers develop tests to satisfy those requirements.
In a script file, an automated testcase ideally addresses one test requirement. Specifically, a 4Test function that begins with the test case keyword and contains a sequence of 4Test statements. It drives an application to the state to be tested, verifies that the application works as expected, and returns the application to its base state.
A script file is a file that contains one or more related testcases. A script file has a .t extension, such as find .t
Other Segue products

The Silk products include

Sit Test for functional and regression testing
Silk Performer for load and performance simulation
Silk Pilot for functional and regression testing CORBA and EJB servers
Silk Radar for automated defet tracking
Silk Vision for enterprise application health monitoring
Silk Express for a scalability and load testing consulting solution.

Silk Test Features :

Some of the feature of silk test are given below.
• Easy to use interface
• Built in recovery system
• The object oriented concept
• Record & Play
• Multi-kind application testing
• Automatic generation of results
• Browser & Platform independent
• 24 x 365 unattended testing
• Distributed Access to Test Results
• Cross Platform Java Testing
• Testing Across Multiple Browsers and Windows Versions
• Support for HTML, XML, JavaScript, Java, Active X, Windows controls, and Visual Basic.
• Single-recording testing for cross-platform Java testing with the Silk Bean.
• Against Over 35 Databases.
• Link Tester
• Validation of Advanced Database Structure and Techniques
• Creation of
Test Plan
Test Frame
Test Suite
• Integration with other silk products.
Silk Test Architecture
Normal use of an application consists of a person manipulating a keyboard and mouse to initiate application operations. The person is said to be interacting with the GUI (Graphical User Interface). During Silk Test testing, Silk Test interacts with the GUI to submit operations to the application automatically. Thus Silk Test can simulate the actions of a person who is exercising all the capabilities of an application and verifying the results of each operation. The simulated user (Silk test) is said to be driving the application. The application under test reacts to the simulated user exactly as it would react to a human rest. Silk Test consists of two distinct software components that execute in separate processes :
Silk Test host software
The Silk Test host software is the program you use to develop, edit, compile, run and debug your 4Test scripts and test plans. This manual refers to the system that runs this program as the host machine or the Silk Test machine.
The Agent
The 4Test Agent is the software process that translates the commands in your 4Test scripts into GUI-specific commands. In order words, it is the Agent that actually drives and monitors the application you are testing. One Agent can run locally on the host machine. In a networked environment, any number of Agents can run on remote machines. This manual refers to the systems that run remote Agents as target machines. This manual refers to the systems that run remote Agents as target machines. In a client/server environment, Silk Test drives the client application by means of an Agent process running on each application’s machine. The application then drives the server just as it always does. Silk Test is also capable of driving the GUI belonging to a server or of directly driving a server database by running scripts that submit SQL statements to the database. These methods o directly manipulating the server application are intended to support testing in which the client application drives the server.
Limitations of Silk Test:

Some of the limitations of Silk Test are given below:
• SilkTest may not recognize some objects in a window / page due to some technical reasons.
• Silk Test may not recognize some window frames.
• The ‘tag’ value may get changed frequently.
• Sometimes it will be difficult to activate some window.
• It may be necessary to make some modifications if testing should be shifted to other browser / operating system.
• In web based applications, sometimes, silktest will take the links as simple text.
System Requirements :

The minimum requirement, a system needs to run the silk test is given below :

Windows NT, Windows 95, 98 or 2000
Pentium 466 Mhz or better processor (application dependent)
60MB Hard Disk

Supported Environments:

Netscape Navigator 4.x
Internet Explorer 4 and 5
Active X, Visual Basic 5 and 6
Java JDK 1.3
Swing 1.1
Microsoft Web browser control

[Test tool] [Selenium] Selenium Regression Testing Part II – Tips and Tricks

29 May

In my last post, I talked about how you can use Selenium to do real regressions tests for web applications. It’s a great way to automate testing the real user experience, and not just the backend stuff.

(see Luke’s last post here: Web Application Functional Regression Testing Using Selenium)

That said, Selenium is a relatively new technology, and it’s not without its issues. When building your first test you might find a lot of times where it’s a trial and error process. There are so many different ways to do the same test, it can be overwhelming. Often times, one or more of the ways you try won’t work. I’m going to try to list some of the common problems I ran into and tips I found below.
Selenium Commands
this is will block your test proceeding until it finds the identified element. It checks once a second for 30 seconds.
this was the most common way I dealt with ‘ajax’ type interactions where it takes an unknown period of time for something to show up
I also use it generally instead of verifyElementPresent – it does basically the same thing with a little wiggle room
mostly equivalent, but sometimes you need to match the desired event to a javascript handler
try click first. If it doesn’t work the way you want, move on to mouseDown and so on.
important if you use iFrames (modal dialog, etc.)
the selenium selectors only hit the current frame, otherwise you have to select the correct frame
an easy way to get back to the root window is to do selectFrame null
type vs. typeKeys
type fills out an input in code – if it works, use this. You can use this,and then fire a single typeKeys for the last character if you need an event to be triggered.
typeKeys fires the key events on the element
has some idiosyncracies – certain letters (I’m looking at you,‘y’) are reserved to do other special keypresses
necessary if you are using a wysiwyg ‘designmode’ type box instead of a standard input
verifyX vs. assertX
if verify fails, the test continues (and is marked as errored). If assert fails, the test aborts.
Usually verify is better, unless one task blocks a future one from functioning correctly
runScript vs. Eval vs. Expression
runScript inserts the javascript you provide into the current frame/window. Useful for those things selenium doesn’t support – like moving a cursor around and selecting text in a wysiwyg
Eval runs javascript in the context of selenium. Handy for complex checks – use waitForEval (for instance, checking the css background image property of a particular element)
Use this.browserbot.findeElement(“selenium selector”) to find elements the way selenium would
Use window.X To access the current frame/window context objects
Expression is similar to Eval, but uses Selenium’s custom expression format instead of javascript (but you can combine with javascript by using javascript{}
storedVars[‘key’] allows you to get to a variable you created with a Selenium ‘store’ expression
useful for checking stuff in a popup that was initiated
Easiest to get by the html title of the popup, but do a ‘pause’ first to let it load
Selenium Selectors and XPath
In general, be as abstract as possible.
Don’t select individual server generated ids (hand crafted html ids are ok if you don’t expect them to change)
Don’t select on complicated relationships ( /div[0]/div[2]/a[4] ) – your html structure will change and you’ll have to maintain it
Select links by the simple link=text when possible – easy to read/maintain, unlikely to change
Use //that (any decendant) instead of /this/that where possible
. references ‘this’ element. Helps to select something with a particular text: //div[@id=’publish-private-shares’]//p[.=’This is pretty cool.’]
Contains() is useful if you don’t know the exact text (for instance, when an element has multiple css classes): //div[@id=’pageContent’ and contains(@class,’contenteditable’) and h2=’Goals’]/p[1]
Selenium RC
While you can use Selenium IDE to create a c# version of your tests – if you do so, you have two tests to maintain. You can run your ‘selenese’ tests directly with RC, too.
I’ve written a simple csharp console project that automatically finds the correct javapath and fires up the test when you run it. If people ask in the comments, I’ll post it.
Last I checked, Chrome and Safari-Windows don’t work. Chrome is supposed to be fixed in Selenium RC 1.0.4
Sauce RC
This is a great UI to help test multiple browsers, but there are a couple of issues
Firefox works, but only in dual window mode
IE works, but only in single window mode.
The ‘timeout’ setting implies a default timeout per action in your test, but it is actually the timeout for your entire test run. Since it defaults to 30 seconds, you’ll probably want to change it, or else your tests will suddenly die for no reason with no explanation/log.

I’m sure there is probably more I’ve forgotten, so leave a comment if you get stuck and I’ll try to help out if I can.

For some helper classes for Selenium 2 with ASP.NET Forms, see the following post: Simplifying C# Selenium 2 Tests for ASP.NET WebForms


[Test tool] [Selenium] Web Application Functional Regression Testing Using Selenium

29 May

At Foliotek, we use a rapid development methodology. Typically, a new item will go from definition through coding to release in a month’s time (bucketed along with other new items for the month). A bugfix will nearly always be released within a week of the time it was reported. In fact, we are currently experimenting with a methodology that will allow us to test and deploy new items individually as well – which means that a new (small) item can go from definition to release in as little as a week, too.

Overall, this kind of workflow is great for us, and great for our customers. We don’t need to wait a year to change something to make our product more compelling, and customers don’t have to wait a year to get something they want implemented. We also avoid the shock of suddenly introducing a year’s worth of development to all our customers all at once – a handful of minor changes every month (or week) is much easier to cope with.

However, it also means that Foliotek is never exactly the same as it was the week before. Every time something changes, there is some risk that something breaks. We handle this risk in two ways:
We test extremely thoroughly
We fix any problems that arise within about a week (severe problems usually the same day)

At first, we did all testing manually. This is the best way to test, assuming you have enough good testers with enough time to do it well. Good testers can’t be just anyone – they have to have a thorough knowledge of how the system should work,they have to care that it does work perfectly,and they have to have a feel for how they might try to break things. Having enough people like this with enough time to do testing is expensive.

Over time two related things happened. One was that we added more developers to the project, and started building more faster. Two was that the system was growing bigger and more complex.

As more people developed on it and the system grew more complex, our testing needs grew exponentially. The rise in complexity and people developing led to much, much more potential for side-effects – problems where one change affects a different (but subtly related) subsystem. Side-effects by their nature are impossible to predict. The only way to catch them was to test EVERYTHING any time ANYTHING changed.

We didn’t have enough experienced testers to do that every month (new development release) let alone every week (bugfix release).

To deal with that, we started by writing a manual regression test script to run through each week. While this didn’t free up any time overall – it did mean that once the test was written well, anyone could execute it. This was doable, because we had interns who had to be around to help handle support calls anyways – and they were only intermittently busy. In their free time they could execute the tests.

Another route we could have gone would have been to write automated unit tests (http://en.wikipedia.org/wiki/Unit_testing). Basically, these are tiny contracts the developers would write that say something like “calling the Add function on the User class with name Luke will result in the User database table having a new row with name Luke”. Each time the project is built, the contracts are verified. This is great for projects like code libraries and APIs where the product of the project IS the result of each function. For a web application, though, the product is the complex interaction of functions and how they produce an on screen behavior. There are lots of ways that the individual functions could all be correct and the behavior still fails. It is also very difficult to impossible to test client-side parts of a web application – javascript, AJAX, CSS, etc. Unit testing would cost a non trivial amount (building and maintaining the tests) for a trivial gain.

Eventually, we discovered the Selenium project (http://seleniumhq.org/download/). The idea of Selenium is basically to take our manual regression test scripts, and create them such that a computer can automatically run the tests in a browser (pretty much) just like a human tester would. This allows us to greatly expand our regression test coverage, and run it for every single change we make and release.

Here are the Selenium tools we use and what we use them for:
Selenium IDE (http://release.seleniumhq.org/selenium-ide/) : A Firefox plugin that lets you quickly create tests using a ‘record’ function that builds it out of your clicks, lets you manually edit to make your tests more complex, and runs them in Firefox.
Selenium RC (http://selenium.googlecode.com/files/selenium-remote-control-1.0.3.zip): A java application that will take the tests you create with Selenium IDE, and run them in multiple browsers (firefox, ie, chrome, etc). It runs from the command line, so its fairly easy to automate test runs into build actions/etc as well.
Sauce RC (http://saucelabs.com/downloads): A fork of RC that adds a web ui on top of the command line interface. It’s useful for quickly debugging tests that don’t execute properly in non-firefox browsers. It also integrates with SauceLabs – a service that lets you run your tests in the cloud on multiple operating systems and browsers (for a fee).
BrowserMob (http://browsermob.com/performance-testing): An online service that will take your selenium scripts and use them to generate real user traffic on your site. Essentially, it spawns off as many real machines and instances of FireFox at once to run your test – each just as you would do locally – for a fee. It costs less than $10 to test up to 25 “real browser users” – which actually can map to many more users than that since the automated test doesn’t have to think between clicks. It gets expensive quickly to test more users than that.

Selenium is a huge boon for us. We took the manual tests that would occupy a tester for as much as a day, and made it possible to run those same tests with minimal interaction in a half hour or less. We’ll be able to cover more test cases, and run it more – even running them as development occurs to catch issues earlier.

In my next post, I’ll talk about the details of how you build tests, run them, maintain them, etc. with the tools mentioned above. See it here: Selenium Tips and Tricks

Also, for Selenium 2 with ASP.NET Web Forms, see Simplifying C# Selenium 2 Tests for ASP.NET WebForms

From http://www.foliotek.com/devblog/realwebtesting/