![]()
Chapter 1
Preliminary Concerns
The term āsoftware bugā is a common term that even beginning computer users know to be a defect or imperfection in a software application. Software users have become accustomed to finding problems with software. Some problems have workarounds and are not severe, whereas others can be extremely problematic and, in some cases, costly. Sadly, as users, we have come to expect this from software. However, in recent years the quality of software has generally increased as software teams spend countless hours identifying and eliminating these problems before the software reaches the user. The process of identifying these bugs is known as testing.
There are many different types of testing that can be performed on your web applications, including functionality of the application, security, load/stress, compliance, and accessibility testing.
If you are new to testing, donāt worry: we will explain the fundamental concepts and guide you to the correct actions youāll need to consider for each type of testing discipline. If you already have experience with testing applications, then this book will identify the key areas of the ASP.NET family and pair them with the correct approaches and the tools available to successfully test your web application.
This book is not intended to be the definitive guide to any particular type of testing, but a thorough overview of each type of web testing discipline. Its goal is to get you started using best practices and testing tools, and provide you with resources to master that particular testing discipline. Itās our aim as authors to help the reader navigate to a section of this book and learn what and how they should be testing at any point in the development of a web-based application.
Although existing books cover different testing disciplines in depth, this book is unique because it applies todayās best testing approaches to the ASP.NET family, including WebForms, ASP.NET MVC Framework, Web Services, Ajax, Silverlight, and ADO.NET Data Services, ensuring that the key technologies relevant today are able to be tested by the reader.
The History of Testing Tools
Tools for testing have been around for as long as developers have been writing code. In the early years of software development, however, there wasnāt a clear distinction between testing and debugging. At the time, this model worked. Some argue that this model worked because the system was closed; most companies who needed software had the developers on staff to create and maintain the systems. Computer systems were not widespread, even though developers worked very closely with customers to deliver exactly what was required. In the years between 1970 and 1995, computer systems started becoming more popular, and the relationships between developers and customers became distant, often placing several layers of management between them.
What is the difference between debugging and testing you might ask? Testing is the process of finding defects in the software. A defect could be a missing feature, a feature that does not perform adequately, or a feature that is broken. Debugging is the process of first tracking down a bug in the software and then fixing it.
Many of the tools developers used for testing in the early days were internal tools developed specifically for a particular project and oftentimes not reused. Developers began to see a need to create reusable tools that included the patterns they learned early on. Testing methodologies evolved and tools started to become standardized, due to this realization. In recent years, testing methodologies have become their own, very strict computer science discipline.
During the past 12 years, many tools have been developed to help make testing easier. However, itās essential to learn about the past and the tools we had previously before diving into the set of tools we have now. Itās important to notice that the tools tend to evolve as the process evolves.
The term "debugging" was made popular by Admiral Grace Murray Hopper, a woman who was working on a Mark II computer at Harvard University in August 1945. When her colleagues discovered a moth stuck in a relay, and realized it was causing issues with the system, she made the comment that they were ādebugging the system.ā
The sUnit Testing Framework
It is said that imitation is the sincerest form of flattery; that said, most modern unit testing frames are derived from the principals set forth in the sUnit testing framework primary developed by Kent Beck in 1998. Below are just a small number of frameworks which have built upon Beckās original concept.
- sUnit. Created by Kent Beck for Small Talk, sUnit has become known as the āmother of testing frameworks.ā Many popular unit testing frameworks such as jUnit and nUnit are ports of sUnit. The key concepts of the sUnit testing framework were originally published in Chapter 30 of Kent Beckās Guide to Better Smalltalk (Cambridge University Press, 1998).
- jUnit. A port of sUnit for Java created by Kent Beck and Erich Gamma in late 1998. jUnit helped bring automated unit testing into the main stream.
- nUnit. In late 2000, all the great things about jUnit were ported to .NET allowing C# developers to write jUnit style unit tests against their C# code.
- qUnit. This is the unit test running for the jQuery Framework. In May 2008, qUnit was promoted to a top-level application in the jQuery project. qUnit allows web developers to run unit tests on JavaScript.
- WCAT. First included in the IIS 4 resource kit in 1998, the Web Capacity Analysis Tool (WCAT) is a freely distributed command-line tool that allows a server to be configured with agents to perform load/stress testing on websites.
- Web Application Stress Tool. In 1999, Microsoft released a free tool to create GUI browser stress tests. This tool recorded a browser session and scripted the actions into a Visual Basic 6 script that could be modified. Because the tool generated scripts that could be modified, many web developers used the tool not only for stress testing but modified the scripts for user interface functional testing.
- Microsoft Application Center Test. Included in Visual Studio 2001 Enterprise Edition, Microsoft ACT improved upon the Web Application Stress tool. Microsoft ACT provided a schema in which the tests could be distributed among agents for large-scale load testing.
- Framework for Integrated Test (FIT). Created by Ward Cunningham in 2002, FIT is a tool for automated customer tests. Examples of how the software should perform are provided by the customer, and automated Test fixtures are created by the developer. The goal of FIT is to help integrate the work of developers, customers, tests, and analysts.
- Fitnesse. Ported to .NET in 2006 by David Chelimsky and Mike Stockdale, Fitnesse combines a web server, Wiki, and the FIT software acceptance testing framework together. This provides an acceptance testing framework which allows users to define input that can be interpreted by a Test fixture allowing for non-technical users to write tests.
- Watir. In May 2002, the Web Application Testing in Ruby Watir (pronounced āWaterā), a library to automate browser acceptance tests in Ruby, is released.
- Selenium. Selenium provides a suite of tools for automated user interface testing of web applications. In 2004, Jason Huggins of Thoughtworks created the core āJavaScriptTestRunnerā mode for automation testing of a time and expense system.
- WatiN. In May 2006, Watir, the popular browser acceptance testing framework, is ported to .NET as the WatiN (pronounced āWatt inā) project.
- Visual Studio 2005 Test Edition. In 2005, Microsoft released a version of Visual Studio that included a new unit testing framework created by Microsoft called MS Test. Along with this new unit testing framework, what was known previously as Microsoft Application Center Test (Microsoft ACT) was integrated into this version as Web Unit Tests and Web Load Tests.
- Visual Studio 2008 Professional. In 2008, the professional version of Visual Studio 2008 included MSTest.
Testing Terminology
As with many different aspects in programming, testing disciplines have their own unique vocabulary. However, because of the number of terms, the barrier to entry is high and can scare new developers. This section is intended to get the reader up to speed on some common terms that will be used throughout the remainder of this book. The terms shown next are only intended to be a brief explanation. Each term will be discussed thoroughly in their respective chapters.
- Test. A test is a systematic procedure to ensure that a particular unit of an application is working correctly.
- Pass. A pass indicates that everything is working correctly. When represented on a report or user interface (UI), it is represented as green.
- Fail. In the case of a fail, the functionality being tested has changed and as a result no longer works as expected. When represented on a report, this is represented as red.
- xUnit. xUnit refers to the various testing frameworks which were originally ported from sUnit. Tools such as jUnit, qUnit, and nUnit fall into the xUnit family.
- Test Fixture. Test fixtures refer to the state a test must be in before the test can be run. Test fixtures prepare any objects that need to be in place before the test is run. Fixtures ensure a known, repeatable state for the tests to be run in.
- Test Driven Development (TDD). Test Driven Development is an Agile Software Development process where a test for a procedure is created before the code is created.
- Behavior Driven Development (BDD). Building on top of the fundamentals of TDD, BDD aims to take more advantage of the design and documentation aspects of TDD to provide more value to the customer and business.
- Test Double. When we cannot, or choose not, to use a real component in unit tests, the object that is substituted for the real component is called a test double.
- Stub. A test stub is a specific type of test double. A stub is used when you need to replicate an object and control the output, but without verifying any interactions with the stub object for correctness. Many types of stubs exist, such as the responder, saboteur, temporary, procedural, and entity chain, which are discussed in more depth in Chapter 2.
- Mock. Mock objects are also a form of test double and work in a similar fashion to stub objects. Mocks are used to simulate the behavior of a complex object. Any interactions made with the mock object are verified for correctness, unlike stub objects. Mock objects are covered in depth in Chapter 2.
- Fake. Fake objects are yet another type of test doubles. Fake objects are similar to test stubs, but replace parts of the functionality with their own implementation to enable testing to be easier for the method.
- Dummy Objects. Dummy objects are used when methods require an object as part of their method or constructor. However, in this case the object is never used by the code under test. As such, a common dummy object is null.
- Unit Test. A unit test is a method used to verify that a small unit of source code is working properly. Unit tests should be independent of external resources such as databases and files. A unit is generally considered a method.
- Developer Test. This is another term for a unit test.
- Integration Test. This is similar to a unit test; however, instead of being an isolation unit, these test cross-application and system boundaries.
- Functional Test. Functional tests group units of work together to test an external requirement. Testing disciplines such as graphical user interface testing and performance testing are considered functional tests.
- GUI Test. GUI tests test the graphical user interface. GUI tests are considered functional tests. Applications are used to simulate users interacting with the system such as entering text into a field or clicking a button. Verifications are then made based on the response from the UI or system.
- Customer Test. This is another term for an acceptance test.
- System Test. The term system test is a term to indicate an āEnd To Endā test of the system. System tests include unit t...
