· Do stated goals and objectives for software remain consistent will system goals and objectives.
· Have important interfaces to all system elements been described?
· " Is information flow and structure adequately defined for the problem?
· Are diagrams clear? Can each stand alone without supplementary text?
· Do major functions remain within scope and has each been adequately described?
· Is the behaviour of the software consistent with the information it must process and the functions it must perform?
· Are design constraints realistic?
· Have the technological risks of development been considered?
· Have alternative software requirements been considered?
· Have validation criteria been stated in detail? Are they adequate to describe a successful system?
· Do inconsistencies, omissions, or redundancies exist?
· Is the customer contact complete?
· Has the user reviewed the preliminary user's manual or prototype?
· How are planning estimates affected?
· Be on the lookout for persuasive connectors (e.g., certainly, therefore, clearly, obviously, it follows that), and ask 'why?"
· Watch out for vague terms (e.g., some, sometimes, often, usually, ordinarily, most, mostly); ask for clarification.
· Watch lists are given, but not completed, be sure all items are understood. Keys to look for: 'etc., and so forth, and so on, such as."
· Be sure stated ranges don't contain unstated assumptions (e.g., Valid codes range from 10 to 100. Integer? Real? Hex?).
· Beware of vague verbs such as "Handled, rejected, processed, skipped, eliminated.' They can be interpreted in many ways.
· Beware ambiguous pronouns (e.g., The I/O module communicates with the data validation module and its control flag is set. Whose control flag?)
· Look for statements that imply certainty (e.g., always, every, all, none, never), then ask fur proof.
· When a term is explicitly defined in one place, try substituting the definition for other occurrences of the term.
· When a structure is described in words, draw a picture to aid in understanding.
· When a calculation is specified, work at least two examples.
Tuesday, February 05, 2008
Full Lifecycle Testing Concept
What is Software Testing?
Ø Process of identifying defects
• Defect is any variance between actual and expected results
Ø Testing should intentionally attempt to make things go wrong to determine if
• things happen when they shouldn't or
• things don't happen when they should
Types of Testing
Ø Static Testing
Ø Dynamic Testing
Static Testing
Ø Involves testing of the development work products before any code is developed.
Ø Ex
• Plan reviews
• Requirements walkthroughs
• Design or code inspections
• Test plan inspections
• Test case reviews
Dynamic Testing
Ø Process of validation by exercising or operating a work product under scrutiny & observing its behavior to changing inputs or environments
Ø Some representative examples of dynamic testing are
• Executing test cases in an working system
• Simulating usage scenarios with real end-users to test usability
• Parallel testing in a production environment
Purpose of Static Testing
Ø Early removal of the Defects in the software development life cycle
Ø Increases the productivity by shortening the testing lifecycles & reducing work
Ø Increases the Quality of the project deliverables
Static Testing Techniques
Ø Prototyping
Ø Desk checks
Ø Checklists
Ø Mapping
Ø Reviews
Ø Walkthroughs
Ø Inspections
Ø Prototyping
• Prototyping is reviewing a work product model (initial version) defined without the full capability of the proposed final product
- The prototype is demonstrated and the result of the exercise is evaluated, leading to an enhanced design
• Used to verify and validate the User Interface design and to confirm usability
Ø Desk checks
• This technique involves the process of a product author reading his or her own product to identify defects
Ø Checklists
• Checklists are a series of common items, or prompting questions to verify completeness of task steps.
Ø Mapping
• Mapping technique is identification of functions to the specification and to show how that function directly or indirectly, maps to the requirements.
- Used with Test Scripts to Cases, Test Conditions to Test Scripts etc
Ø Reviews
• Reviews are useful mechanism for getting feedback quickly from peers and team members
Ø Walkthroughs
• Walkthroughs are generally run as scheduled meetings and participants are invited to attend.
- The minutes of the meeting are recorded, as are the issues and action items resulting from the meeting.
- Owners are assigned for issues and actions and there is generally follow up done.
Ø Inspections
• Defect detection activity, aimed at producing a "defect free" work product, before it is passed on to the next phase of the development process
Ø Objectives of an Inspection.
• Increase quality and productivity
• Minimize costs and cycle elapsed time
• Facilitate project management
Inspection Process
Ø An inspection has the following key properties
• A moderator
• Definite participant roles
• Author, Inspector, Recorder
• Stated entry and exit criteria
• Clearly defined defect types
• A record of detected defects
• Re-inspection criteria
• Detected defect feedback to author
• Follow-up to ensure defects are fixed / resolved
Inspection Process Results
Ø Major defects
• Missing (M) - an item is missing
• Wrong (W) - an item has been implemented incorrectly.
• Extra (E) - an item is included which is not part of the specifications.
• Issues (I) - an item not implemented in satisfactory manner
• Suggestion (S) - suggestion to improve the work product.
Ø Minor defects
• Clarity in comments and description
• Insufficient / excessive documentation
• Incorrect spelling / punctuation
Testing Techniques
Ø Black Box Testing
• The testers have an "outside" view of the system.
• They are concerned with "what is done" NOT "how it is done.“
Ø White Box Testing
• In the White Box approach, the testers have an inside view of the system. They are concerned with "how it is done" NOT "what is done
Levels of Testing
* Unit testing
* Integration testing
* System testing
* Systems integration testing
* User acceptance testing
Unit Testing
Ø Unit level test is the initial testing of new and changed code in a module.
Ø Verifies the program specifications to the internal logic of the program or module and validates the logic.
Integration Testing
Ø Integration level tests verify proper execution of application components and do not require that the application under test interface with other applications
Ø Communication between modules within the sub-system is tested in a controlled and isolated environment within the project
System Testing
Ø System level tests verify proper execution of the entire application components including interfaces to other applications
Ø Functional and structural types of tests are performed to verify that the system is functionally and operationally sound.
Systems Integration Testing
Ø Systems Integration testing is a test level which verifies the integration of all applications
• Includes interfaces internal and external to the organization, with their hardware, software and infrastructure components.
Ø Carried out in a production-like environment
User Acceptance Testing
Ø Verify that the system meets user requirements as specified.
Ø Simulates the user environment and emphasizes security, documentation and regression tests
Ø Demonstrate that the system performs as expected to the sponsor and end-user so that they may accept the system.
Types of Tests
Ø Functional Testing
Ø Structural Testing
Functional Testing
Ø Audit and Controls testing
Ø Conversion testing
Ø Documentation & Procedures testing
Ø Error Handling testing
Ø Functions / Requirements testing
Ø Interface / Inter-system testing
Ø Installation testing
Ø Parallel testing
Ø Regression testing
Ø Transaction Flow (Path) testing
Ø Usability testing
Audit And Controls Testing
Ø Verifies the adequacy and effectiveness of controls and ensures the capability to prove the completeness of data processing results
• Their validity would have been verified during design
Ø Normally carried out as part of System Testing once the primary application functions have been stabilized
Conversation Testing
Ø Verifies the compatibility of the converted program, data, and procedures with those from existing systems that are being converted or replaced.
Ø Most programs that are developed for conversion purposes are not totally new. They are often enhancements or replacements for old, deficient, or manual systems.
Ø The conversion may involve files, databases, screens, report formats, etc.
User Documentation And Procedures Testing
Ø Ensures that the interface between the system and the people works and is useable.
Ø Done as part of procedure testing to verify that the instruction guides are helpful and accurate.
• Both areas of testing are normally carried out late in the cycle as part of System Testing or in the UAT.
Ø Not generally done until the externals of the system have stabilized.
• Ideally, the persons who will use the documentation and procedures are the ones who should conduct these tests.
Error-Handling Testing
Ø Error-handling is the system function for detecting and responding to exception conditions (such as erroneous input)
Ø Ensures that incorrect transactions will be properly processed and that the system will terminate in a controlled and predictable way in case of a disastrous failure
Function Testing
Ø Function Testing verifies, at each stage of development, that each business function operates as stated in the Requirements and as specified in the External and Internal Design documents.
Ø Function testing is usually completed in System Testing so that by the time the system is handed over to the user for UAT, the test group has already verified that the system meets requirements.
Installation Testing
Ø Any application that will be installed and run in an environment remote from the development location requires installation testing.
• This is especially true of network systems that may be run in many locations.
• This is also the case with packages where changes were developed at the vendor's site.
• Necessary if the installation is complex, critical, should be completed in a short window, or of high volume such as in microcomputer installations.
• This type of testing should always be performed by those who will perform the installation process
Interface / Inter-system Testing
Ø Application systems often interface with other application systems. Most often, there are multiple applications involved in a single project implementation.
Ø Ensures that the interconnections between applications function correctly.
Ø More complex if the applications operate on different platforms, in different locations or use different languages.
Parallel Testing
Ø Parallel testing compares the results of processing the same data in both the old and new systems.
Ø Parallel testing is useful when a new application replaces an existing system, when the same transaction input is used in both, and when the output from both is reconcilable.
Ø Useful when switching from a manual system to an automated system.
Regression Testing
Ø Verifies that no unwanted changes were introduced to one part of the system as a result of making changes to another part of the system.
Transaction Flow Testing
Ø Testing of the path of a transaction from the time it enters the system until it is completely processed and exits a suite of applications
Usability Testing
Ø Ensures that the final product is usable in a practical, day-to-day fashion
Ø Looks for simplicity and user-friendliness of the product
Ø Usability testing would normally be performed as part of functional testing during System and User Acceptance Testing.
Structural Testing
Ø Ensures that the technical and "housekeeping" functions of the system work
Ø Designed to verify that the system is structurally sound and can perform the intended tasks.
The categories for structural testing
Ø Backup and Recovery testing
Ø Contingency testing
Ø Job Stream testing
Ø Operational testing
Ø Performance testing
Ø Security testing
Ø Stress / Volume testing
Backup And Recovery Testing
Ø Recovery is the ability of an application to be restarted after failure.
Ø The process usually involves backing up to a point in the processing cycle where the integrity of the system is assured and then re-processing the transactions past the original point of failure.
Ø The nature of the application, the volume of transactions, the internal design of the application to handle a restart process, the skill level of the people involved in the recovery procedures, documentation and tools provided, all impact the recovery process
Contingency Testing
Ø Operational situations may occur which result in major outages or "disasters". Some applications are so crucial that special precautions need to be taken to minimize the effects of these situations and speed the recovery process. This is called Contingency.
Job Stream Testing
Ø Done as a part of operational testing (the test type not the test level, although this is still performed during Operability Testing).
Ø Starts early and continues throughout all levels of testing.
- Conformance to standards is checked in User Acceptance and Operability testing.
Operational Testing
Ø All products delivered into production must obviously perform according to user requirements. However, a product's performance is not limited solely to its functional characteristics. Its operational characteristics are just as important since users expect and demand a guaranteed level of service from Computer Services. Therefore, even though Operability Testing is the final point where a system's operational behavior is tested, it is still the responsibility of the developers to consider and test operational factors during the construction phase.
Performance Testing
Ø Performance Testing is designed to test whether the system meets the desired level of performance in a production environment. Performance considerations may relate to response times, turn around times (through-put), technical design issues and so on. Performance testing can be conducted using a production system, a simulated environment, or a prototype.
Security Testing
Ø Security of an application system is required to ensure the protection of confidential information in a system and in other affected systems is protected against loss, corruption, or misuse; either by deliberate or accidental actions. The amount of testing needed depends on the risk assessment of the consequences of a breach in security. Tests should focus on, and be limited to those security features developed as part of the system
Stress/ Volume Testing
Ø Stress testing is defined as the processing of a large number of transactions through the system in a defined period of time. It is done to measure the performance characteristics of the system under peak load conditions.
Ø Stress factors may apply to different aspects of the system such as input transactions, report lines, internal tables, communications, computer processing capacity, throughput, disk space, I/O and so on.
Ø Stress testing should not begin until the system functions are fully tested and stable. The need for Stress Testing must be identified in the Design Phase and should commence as soon as operationally stable system units are available.
Bottom-up Testing
Ø Approach to integration testing where the lowest level components are tested first then used to facilitate the testing of higher level components. This process is repeated until the component at the top of the hierarchy is tested.
Test Bed
Ø A test environment containing the hardware, instrumentation tools, simulators, and other support software necessary for testing a system or system component. (2) A set of test files, (including databases and reference files), in a known state, used with input test data to test one or more test conditions, measuring against expected results.
Ø Process of identifying defects
• Defect is any variance between actual and expected results
Ø Testing should intentionally attempt to make things go wrong to determine if
• things happen when they shouldn't or
• things don't happen when they should
Types of Testing
Ø Static Testing
Ø Dynamic Testing
Static Testing
Ø Involves testing of the development work products before any code is developed.
Ø Ex
• Plan reviews
• Requirements walkthroughs
• Design or code inspections
• Test plan inspections
• Test case reviews
Dynamic Testing
Ø Process of validation by exercising or operating a work product under scrutiny & observing its behavior to changing inputs or environments
Ø Some representative examples of dynamic testing are
• Executing test cases in an working system
• Simulating usage scenarios with real end-users to test usability
• Parallel testing in a production environment
Purpose of Static Testing
Ø Early removal of the Defects in the software development life cycle
Ø Increases the productivity by shortening the testing lifecycles & reducing work
Ø Increases the Quality of the project deliverables
Static Testing Techniques
Ø Prototyping
Ø Desk checks
Ø Checklists
Ø Mapping
Ø Reviews
Ø Walkthroughs
Ø Inspections
Ø Prototyping
• Prototyping is reviewing a work product model (initial version) defined without the full capability of the proposed final product
- The prototype is demonstrated and the result of the exercise is evaluated, leading to an enhanced design
• Used to verify and validate the User Interface design and to confirm usability
Ø Desk checks
• This technique involves the process of a product author reading his or her own product to identify defects
Ø Checklists
• Checklists are a series of common items, or prompting questions to verify completeness of task steps.
Ø Mapping
• Mapping technique is identification of functions to the specification and to show how that function directly or indirectly, maps to the requirements.
- Used with Test Scripts to Cases, Test Conditions to Test Scripts etc
Ø Reviews
• Reviews are useful mechanism for getting feedback quickly from peers and team members
Ø Walkthroughs
• Walkthroughs are generally run as scheduled meetings and participants are invited to attend.
- The minutes of the meeting are recorded, as are the issues and action items resulting from the meeting.
- Owners are assigned for issues and actions and there is generally follow up done.
Ø Inspections
• Defect detection activity, aimed at producing a "defect free" work product, before it is passed on to the next phase of the development process
Ø Objectives of an Inspection.
• Increase quality and productivity
• Minimize costs and cycle elapsed time
• Facilitate project management
Inspection Process
Ø An inspection has the following key properties
• A moderator
• Definite participant roles
• Author, Inspector, Recorder
• Stated entry and exit criteria
• Clearly defined defect types
• A record of detected defects
• Re-inspection criteria
• Detected defect feedback to author
• Follow-up to ensure defects are fixed / resolved
Inspection Process Results
Ø Major defects
• Missing (M) - an item is missing
• Wrong (W) - an item has been implemented incorrectly.
• Extra (E) - an item is included which is not part of the specifications.
• Issues (I) - an item not implemented in satisfactory manner
• Suggestion (S) - suggestion to improve the work product.
Ø Minor defects
• Clarity in comments and description
• Insufficient / excessive documentation
• Incorrect spelling / punctuation
Testing Techniques
Ø Black Box Testing
• The testers have an "outside" view of the system.
• They are concerned with "what is done" NOT "how it is done.“
Ø White Box Testing
• In the White Box approach, the testers have an inside view of the system. They are concerned with "how it is done" NOT "what is done
Levels of Testing
* Unit testing
* Integration testing
* System testing
* Systems integration testing
* User acceptance testing
Unit Testing
Ø Unit level test is the initial testing of new and changed code in a module.
Ø Verifies the program specifications to the internal logic of the program or module and validates the logic.
Integration Testing
Ø Integration level tests verify proper execution of application components and do not require that the application under test interface with other applications
Ø Communication between modules within the sub-system is tested in a controlled and isolated environment within the project
System Testing
Ø System level tests verify proper execution of the entire application components including interfaces to other applications
Ø Functional and structural types of tests are performed to verify that the system is functionally and operationally sound.
Systems Integration Testing
Ø Systems Integration testing is a test level which verifies the integration of all applications
• Includes interfaces internal and external to the organization, with their hardware, software and infrastructure components.
Ø Carried out in a production-like environment
User Acceptance Testing
Ø Verify that the system meets user requirements as specified.
Ø Simulates the user environment and emphasizes security, documentation and regression tests
Ø Demonstrate that the system performs as expected to the sponsor and end-user so that they may accept the system.
Types of Tests
Ø Functional Testing
Ø Structural Testing
Functional Testing
Ø Audit and Controls testing
Ø Conversion testing
Ø Documentation & Procedures testing
Ø Error Handling testing
Ø Functions / Requirements testing
Ø Interface / Inter-system testing
Ø Installation testing
Ø Parallel testing
Ø Regression testing
Ø Transaction Flow (Path) testing
Ø Usability testing
Audit And Controls Testing
Ø Verifies the adequacy and effectiveness of controls and ensures the capability to prove the completeness of data processing results
• Their validity would have been verified during design
Ø Normally carried out as part of System Testing once the primary application functions have been stabilized
Conversation Testing
Ø Verifies the compatibility of the converted program, data, and procedures with those from existing systems that are being converted or replaced.
Ø Most programs that are developed for conversion purposes are not totally new. They are often enhancements or replacements for old, deficient, or manual systems.
Ø The conversion may involve files, databases, screens, report formats, etc.
User Documentation And Procedures Testing
Ø Ensures that the interface between the system and the people works and is useable.
Ø Done as part of procedure testing to verify that the instruction guides are helpful and accurate.
• Both areas of testing are normally carried out late in the cycle as part of System Testing or in the UAT.
Ø Not generally done until the externals of the system have stabilized.
• Ideally, the persons who will use the documentation and procedures are the ones who should conduct these tests.
Error-Handling Testing
Ø Error-handling is the system function for detecting and responding to exception conditions (such as erroneous input)
Ø Ensures that incorrect transactions will be properly processed and that the system will terminate in a controlled and predictable way in case of a disastrous failure
Function Testing
Ø Function Testing verifies, at each stage of development, that each business function operates as stated in the Requirements and as specified in the External and Internal Design documents.
Ø Function testing is usually completed in System Testing so that by the time the system is handed over to the user for UAT, the test group has already verified that the system meets requirements.
Installation Testing
Ø Any application that will be installed and run in an environment remote from the development location requires installation testing.
• This is especially true of network systems that may be run in many locations.
• This is also the case with packages where changes were developed at the vendor's site.
• Necessary if the installation is complex, critical, should be completed in a short window, or of high volume such as in microcomputer installations.
• This type of testing should always be performed by those who will perform the installation process
Interface / Inter-system Testing
Ø Application systems often interface with other application systems. Most often, there are multiple applications involved in a single project implementation.
Ø Ensures that the interconnections between applications function correctly.
Ø More complex if the applications operate on different platforms, in different locations or use different languages.
Parallel Testing
Ø Parallel testing compares the results of processing the same data in both the old and new systems.
Ø Parallel testing is useful when a new application replaces an existing system, when the same transaction input is used in both, and when the output from both is reconcilable.
Ø Useful when switching from a manual system to an automated system.
Regression Testing
Ø Verifies that no unwanted changes were introduced to one part of the system as a result of making changes to another part of the system.
Transaction Flow Testing
Ø Testing of the path of a transaction from the time it enters the system until it is completely processed and exits a suite of applications
Usability Testing
Ø Ensures that the final product is usable in a practical, day-to-day fashion
Ø Looks for simplicity and user-friendliness of the product
Ø Usability testing would normally be performed as part of functional testing during System and User Acceptance Testing.
Structural Testing
Ø Ensures that the technical and "housekeeping" functions of the system work
Ø Designed to verify that the system is structurally sound and can perform the intended tasks.
The categories for structural testing
Ø Backup and Recovery testing
Ø Contingency testing
Ø Job Stream testing
Ø Operational testing
Ø Performance testing
Ø Security testing
Ø Stress / Volume testing
Backup And Recovery Testing
Ø Recovery is the ability of an application to be restarted after failure.
Ø The process usually involves backing up to a point in the processing cycle where the integrity of the system is assured and then re-processing the transactions past the original point of failure.
Ø The nature of the application, the volume of transactions, the internal design of the application to handle a restart process, the skill level of the people involved in the recovery procedures, documentation and tools provided, all impact the recovery process
Contingency Testing
Ø Operational situations may occur which result in major outages or "disasters". Some applications are so crucial that special precautions need to be taken to minimize the effects of these situations and speed the recovery process. This is called Contingency.
Job Stream Testing
Ø Done as a part of operational testing (the test type not the test level, although this is still performed during Operability Testing).
Ø Starts early and continues throughout all levels of testing.
- Conformance to standards is checked in User Acceptance and Operability testing.
Operational Testing
Ø All products delivered into production must obviously perform according to user requirements. However, a product's performance is not limited solely to its functional characteristics. Its operational characteristics are just as important since users expect and demand a guaranteed level of service from Computer Services. Therefore, even though Operability Testing is the final point where a system's operational behavior is tested, it is still the responsibility of the developers to consider and test operational factors during the construction phase.
Performance Testing
Ø Performance Testing is designed to test whether the system meets the desired level of performance in a production environment. Performance considerations may relate to response times, turn around times (through-put), technical design issues and so on. Performance testing can be conducted using a production system, a simulated environment, or a prototype.
Security Testing
Ø Security of an application system is required to ensure the protection of confidential information in a system and in other affected systems is protected against loss, corruption, or misuse; either by deliberate or accidental actions. The amount of testing needed depends on the risk assessment of the consequences of a breach in security. Tests should focus on, and be limited to those security features developed as part of the system
Stress/ Volume Testing
Ø Stress testing is defined as the processing of a large number of transactions through the system in a defined period of time. It is done to measure the performance characteristics of the system under peak load conditions.
Ø Stress factors may apply to different aspects of the system such as input transactions, report lines, internal tables, communications, computer processing capacity, throughput, disk space, I/O and so on.
Ø Stress testing should not begin until the system functions are fully tested and stable. The need for Stress Testing must be identified in the Design Phase and should commence as soon as operationally stable system units are available.
Bottom-up Testing
Ø Approach to integration testing where the lowest level components are tested first then used to facilitate the testing of higher level components. This process is repeated until the component at the top of the hierarchy is tested.
Test Bed
Ø A test environment containing the hardware, instrumentation tools, simulators, and other support software necessary for testing a system or system component. (2) A set of test files, (including databases and reference files), in a known state, used with input test data to test one or more test conditions, measuring against expected results.
Software testing methodologies
Doing software testing after coding is like looking for typos once a book has gone to press. Not only is it ineffective, it's costly. The most efficient testing approach applies sound testing practices throughout the entire software lifecycle.
Automated Testing
Automated testing is as simple as removing the "human factor" and letting the computer do the thinking. This can be done with integrated debug tests, to much more intricate processes. The idea of the these tests is to find bugs that are often very challenging or time intensive for human testers to find. This sort of testing can save many man hours and can be more "efficient" in some cases. But it will cost more to ask a developer to write more lines of code into the game (or an external tool) then it does to pay a tester and there is always the chance there is a bug in the bug testing program. Reusability is another problem; you may not be able to transfer a testing program from one title (or platform) to another. And of course, there is always the "human factor" of testing that can never truly be replaced.
Other successful alternatives or variation: Nothing is infallible. Realistically, a moderate split of human and automated testing can rule out a wider range of possible bugs, rather than relying solely on one or the other. Giving the tester limited access to any automated tools can often help speed up the test cycle.
Release Acceptance Test
The release acceptance test (RAT), also referred to as a build acceptance or smoke test, is run on each development release to check that each build is stable enough for further testing. Typically, this test suite consists of entrance and exit test cases plus test cases that check mainstream functions of the program with mainstream data. Copies of the RAT can be distributed to developers so that they can run the tests before submitting builds to the testing group. If a build does not pass a RAT test, it is reasonable to do the following:
Suspend testing on the new build and resume testing on the prior build until another build is received.
Report the failing criteria to the development team.
Request a new build.
Functional Acceptance Simple Test
The functional acceptance simple test(FAST) is run on each development release to check that key features of the program are appropriately accessible and functioning properly on the at least one test configuration (preferable the minimum or common configuration).This test suite consists of simple test cases that check the lowest level of functionality for each command- to ensure that task-oriented functional tests(TOFTs) cna be performed on the program. The objective is to decompose the functionality of a program down to the command level and then apply test cases to check that each command works as intended. No attention is paid to the combination of these basic commands, the context of the feature that is formed by these combined commands, or the end result of the overall feature. For example, FAST for a File/Save As menu command checks that the Save As dialog box displays. However, it does not validate that the overall file-saving feature works nor does it validate the integrity of save files.
Deployment Acceptance Test
The configuration on which the Web system will be deployed will often be much different from develop-and-test configurations. Testing efforts must consider this in the preparation and writing of test cases for installation time acceptance tests. This type of test usually includes the full installation of the applications to the targeted environments or configurations.
Task-Oriented Functional Test
The task-oriented functional test (TOFT) consists of positive test cases that are designed to verify program features by checking the task that each feature performs against specifications, user guides, requirements, and design documents. Usually, features are organized into list or test matrix format. Each feature is tested for:
The validity of the task it performs with supported data conditions under supported operating conditions.
The integrity od the task's end result
The feature's integrity when used in conjunction with related features
Forced-Error Test
The forced-error test (FET) consists of negative test cases that are designed to force a program into error conditions. A list of all error messages that the program issues should be generated. The list is used as a baseline for developing test cases. An attempt is made to generate each error message in the list. Obviously, test to validate error-handling schemes cannot be performed until all the handling and error message have been coded. However, FETs should be thought through as early as possible. Sometimes, the error messages are not available. The error cases can still be considered by walking through the program and deciding how the program might fail in a given user interface such as a dialog or in the course of executing a given task or printing a given report. Test cases should be created for each condition to determine what error message is generated.
Real-world User-level Test
These tests simulate the actions customers may take with a program. Real-World user-level testing often detects errors that are otherwise missed by formal test types.
Exploratory Test
Exploratory Tests do not involve a test plan, checklist, or assigned tasks. The strategy here is to use past testing experience to make educated guesses about places and functionality that may be problematic. Testing is then focused on those areas. Exploratory testing can be scheduled. It can also be reserved for unforeseen downtime that presents itself during the testing process.
Compatibility and Configuration Testing
Compatibility and configuration testing is performed to check that an application functions properly across various hardware and software environments. Often, the strategy is to run the functional acceptance simple tests or a subset of the task-oriented functional tests on a range of software and hardware configurations. Sometimes, another strategy is to create a specific test that takes into account the error risks associated with configuration differences. For example, you might design an extensive series of tests to check for browser compatibility issues. Software compatibility configurations include variances in OS versions, input/output (I/O) devices, extension, network software, concurrent applications, online services and firewalls. Hardwere configurations include variances in manufacturers, CPU types, RAM, graphic display cards, video capture cards, sound cards, monitors, network cards, and connection types(e.g. T1, DSL, modem, etc..).
Documentation
Testing of reference guides and user guises check that all features are reasonably documented. Every page of documentation should be keystroke-tested for the following errors:
Accuracy of every statement of fact
Accuracy of every screen shot, figure and illustration
Accuracy of placement of figures and illustration
Accuracy of every tutorial, tip, and instruction
Accuracy of marketing collateral (claims, system requirements, and screen shots)
Accuracy of downloadable documentation (PDFs, HTML, or test files)
Online Help Test
Online help tests check the accuracy of help contents, correctness of features in the help system, and functionality of the help system.
Install/uninstall Test
Web system often require both client-side and server-side installs. Testing of the installer checks that installed features function properly--including icons, support documentation , the README file, and registry keys. The test verifies that the correct directories are created and that the correct system files are copied to the appropriate directories. The test also confirms that various error conditions are detected and handled gracefully.
Testing of the uninstaller checks that the installed directories and files are appropriately removed, that configuration and system-related files are also appropriately removed or modified, and that the operating environment is recovered in its original state.
User Interface Tests
Easy-of-use UI testing evaluates how intuitive a system is. Issues pertaining to navigation, usability, commands, and accessibility are considered. User interface functionality testing examines how well a UI operates to specifications.
AREAS COVERED IN UI TESTING
Usability
Look and feel
Navigation controls/navigation bar
Instructional and technical information style
Images
Tables
Navigation branching
Accessibility
External Beta Testing
External beta testing offers developers their first glimpse at how users may actually interact with a program. Copies of the program or a test URL, sometimes accompanied with letter of instruction, are sent out to a group of volunteers who try out the program and respond to questions in the letter. Beta testing is black-box, real-world testing. Beta testing can be difficult to manage, and the feedback that it generates normally comes too late in the development process to contribute to improved usability and functionality. External beta-tester feedback may be reflected in a README file or deferred to future releases.
Security Tests
Security measures protect Web systems from both internal and external threats. E-commerce concerns and the growing popularity of Web-based applications have made security testing increasingly relevant. Security tests determine whether a company's security policies have been properly implemented; they evaluate the functionality of existing systems, not whether the security policies that have been implemented are appropriate.
PRIMARY COMPONENTS REQUIRING SECURITY TESTING
Application software
Database
Servers
Client workstations
Networks
Unit Tests
Unit tests are positive tests that evaluate the integrity of software code units before they are integrated with other software units. Developers normally perform unit testing. Unit testing represents the first round of software testing--when developers test their own software and fix errors in private.
Click-Stream Testing
Click stream Testing is to show which URLs the user clicked, The Web site's user activity by time period during the day, and other data otherwise found in the Web server logs. Popular choice for Click-Stream Testing statistics include Keynote Systems Internet weather report , WebTrends log analysis utility, and the NetMechanic monitoring service.
Disadvantage: Click-Stream Testing statistics reveal almost nothing about the user's ability to achieve their goals using the Web site. For example, a Web site may show a million page views, but 35% of the page views may simply e pages with the message "Found no search results," With Click-Stream Testing, there's no way to tell when user reach their goals.
Click-stream measurement tests
Makes a request for a set of Web pages and records statiestics about the response, including total page views per hour, total hits per week, total user sessions per week, and derivatives of these numbers. The downside is that if your Web-enabled application takes twics as many pages as it should for a user to complete his or her goal, the click stream test makes it look as though your Web site is popular, while to the user your Web site is frustrating.
HTML content-checking tests
HTML content checking tests makes a request to a Web page, parses the response for HTTP hyperlinks, requests hyperlinks from their associated host, and if the links returned successful or exceptional conditions. The downside is that the hyperlinks in a Web-enalbled application are dynamic and can change, depending on the user's actions. There is little way to know the context of the hyperlinks in a Web-enabled application. Just checking the links' validity is meaningless if not misleading. These tests were meant to test static Web sites, not Web-enabled application
Web-Enabled Application Measurement Tests
Meantime between failures in seconds
Amount of time in seconds for each user session, sometimes know as transaction
Application availability and peak usage periods.
Which media elements are most used ( for example, HTML vs. Flash, JavaScript vs. HTML forms, Real vs. Windows Media Player vs. QuickTime)
Ping tests
Ping tests use the Internet Control Message Protocol(ICMP) to send a ping request to a server. If the ping returns, the server is assumed to be alive and well. The downside is that usually a Web server will continue to return ping requests even when the Web-enable application has crashed.
Unit Testing
Unit testing finds problems and errors at the module level before the software leaves development. Unit testing is accomplished by adding a small amount of the code to the module that validates the module's responses.
System-Level Test
System-level tests consists of batteris of tests that are designed to fully exercise a program as a whole and check that all elements of the integrated system function properly.
Functional System Testing
System tests check that the software functions properly from end-to-end. The components of the system include: A database, Web-enable application software modules, Web servers, Web-enabled application frameworks deploy Web browser software, TCP/IP networking routers, media servers to stream audio and video, and messaging services for email.
A common mistake of test professionals is to believe that they are conducting system tests while they are actually testing a single component of the system. For example, checking that the Web server returns a page is not a system test if the page contains only a static HTML page.
System testing is the process of testing an integrated hardware and software system to verify that the system meets its specified requirements. It verifies proper execution of the entire set of application components including interfaces to other applications. Project teams of developers and test analysts are responsible for ensuring that this level of testing is performed.
System testing checklist include question about:
Functional completeness of the system or the add-on module
Runtime behavior on various operating system or different hardware configurantions.
Installability and configurability on various systems
Capacity limitation (maximum file size, number of records, maximum number of concurrent users, etc.)
Behavior in response to problems in the programming environment (system crash, unavailable network, full hard-disk, printer not ready)
Protection against unauthorized access to data and programs.
"black-box" (or functional) testing
Black Box Testing is testing without knowledge of the internal workings of the item being tested. The Outside world comes into contact with the test items, --only through the application interface ,,,, an internal module interface, or the INPUT/OUTPUT description of a batch process. They check whether interface definitions are adhered to in all situation and whether the product conform to all fixed requirements. Test cases are created based on the task descriptions.
Black Box Testing assumes that the tester does not know anything about the application that is going to be tested. The tester needs to understand what the program should do, and this is achieved through the business requirements and meeting and talking with users.
Funcional tests: This type of tests will evaluate a specific operating condition using inputs and validating results. Functional tests are designed to test boundaries. A combination of correst and incorrect data should be used in this type of test.
Scalability and Performance Testing
Scalability and performance testing is the way to understand how the system will handle the load cause by many concurrent users. In a Web environment concurrent use is measured as simply the number of users making requests at the same time.
Performance testing is designed to measure how quickly the program completes a given task. The primary objective is to determine whether the processing speed is acceptable in all parts of the program. If explicit requirements specify program performance, then performance test are often performed as acceptance tests.
As a rule, performance tests are easy to automate. This makes sense above all when you want to make a performance comparison of different system conditions while using the user interface. The capture and automatic replay of user actions during testing eliminates variations in response times.
This type of test should be designed to verify response and excution time. Bottlenecks in a system are generally found during this stage of testing.
Stress Testing
Overwhelm the product for performance, reliability, and efficiency assessment; To find the breakpoint when system is failure; to increase load regressively to gather information for finding out maximum concurrent users.
Stress tests force programs to operate under limited resource conditions. The goal is to push the upper functional limits of a program to ensure that it can function correctly and handle error conditions gracefully. Examples of resources that may be artificially manipulated to create stressful conditions include memory, disk space, and network bandwidth. If other memory-oriented tests are also planned, they should be performed here as part of the stress test suite. Stress tests can be automated.
Breakpoint:
the capabilites and weakness of the product:
High volunmes of data
Device connections
Long transation chains
Stress Test Environment:
As you set up your testing environment for a stress test, you need to make sure you can answer the following questions:
Will my test be able to support all the users and still maintain performance?
Will my test be able to simulate the number of transactions that pass through in a matter of hours?
Will my test be able to uncover whether the system will break?
Will my server crash if the load continues over and over?
The test should be set up so that you can simulate the load; for example:
If you have a remote Web site you should be able to monitor up to four Web sites or URLs.
There should be a way to monitor the load intervals.
The load test should be able to simulate the SSL (Secure Server)
The test should be able to simulate when a user submits the Form Data (GET method)
The test should be set up to simulate and authentical the keyword verification.
The test should be able to simulate up to six email or pager mail addresses and an alert should occur when there is a failure.
It is important to remember when stressing your Web site to give a certain number of users a page to stress test and give them a certain amount of time in which to run the test.
Some of the key data features that can help you measure this type of stress test, determine the load, and uncover bottlenecks in the system are:
Amount of memory available and used
The processor time used
The number of requests per second
The amount of time it takes ASP pages to be set up.
Server timing errors.
Load Testing
The process of modeling application usage conditions and performing them against the application and system under test, to analyze the application and system and determine capacity, throughout speed, transation handling capabilities, scalabilities and reliability while under under stress.
This tyoe of test is designed to identify possible overloads to the system such as too many users signed on to the system, too many terminals on the network, and network system too slow.
Load testing a simulation of how a browser will respond to intense use by many individuals. The Web sessions can be recorded live and set up so that the test can be run during peak times and also during slow times. The following are two different types of load tests:
Single session - A single session should be set up on browser that will have one or multiple responses. The timeing of the data should be put in a file. After the test, you can set up a separate file for report analysis.
Multiple session - a multiple session should be developed on multiple browsers with one or multiple responses. The multivariate statistical methods may be needed for a complex but general performance model
When performing stress testing, looping transactions back on themselves so that the system stresses itself simulates stress loads and may be useful for finding synchronization problems and timing bugs, Web priority problems, memory bugs, and Windows problems using API. For example, you may want ot simulate an incoming message that is then put out on a looped-back line; this in turn will generate another incoming message. The nyou can use another system of comparable size to create the stress load.
Memory leaks are often found under stress testing. A memory leak occurs when a test leaves allocated memory behind and does not correctly return the memory to the memory allocation scheme. The test seems to run correctly, but after several iteration available memory is reduced until the system fails.
Peak Load and Testing Parameters:
Determining your peak load is important before beginning the assessment of the Web site test. It may mean more than just using user requests per second to stress the system. There should be a combination of determinants such as requests per second , processor time, and memory usage. There is also the consideration of the type of information that is on your Web page from graphics and code processing, such as scripts, to ASP pages. Then it is important to determine what is fast and what is slow for your system. The type of connection can be a critical component here, such as T1 or T3 versus a modem hookup. After you have selected your threshold, you can stress your system to additional limits.
As a tester you need to set up test parameters to make sure you can log the number of users coming into and leaving the test. This should be started in a small way and steadily increased. The test should also begin by selecting a test page that may not have a large amount of graphics and steadily increasing the complexity of the test by increasing the number of graphics and image requests. Keep in mind that images will take up additional bandwidth and resources on the server but do not really have a large impact on the server's processor.
Another important item to remember is that you need to account for the length of time the user will spend surfing each page. As you test, you should set up a log to determine the approximate time spend on each page, whether it is 25 or 30 seconds. It may be recorded that each user spends at least 30 seconds on each page, and that will produce a heightened response for the server. As the request is queued, and this will be analyzed as the test continues.
Load/Volume Test
Load/volume tests study how a program handles large amounts of data, excessive calculations, and excessive processing. These tests do not necessarily have to push or exceed upper functional limits. Load/volume tests can, and usually must, be automated.
Focus of Load/Volume Tesing
Pushing through large amounts of data with extreme processing demands.
Requesting many processes simulateously.
Repeating tasks over a long period of time
Load/volume tests, which involve extreme conditions, are normally run after the execution of feature-level tests, which prove that a program functions correctly under normal conditions.
Difference between Load and Strees testing
The idea of stress testing is to find the breaking point in order to find bugs that will make that break potentially harmful. Load testing is merely testing at the highest transaction arrival rate in performance testing to see the resource contention, database locks etc..
Web Capacity Testing Load and Stress
The performance of the load or stress test Web site should be monitored with the following in mind:
The load test should be able to support all browser
The load test should be able to support all Web server.
The tool should be able to simulate up 500 users or playback machines
The tool should be able to run on WIndows NT, Linux, Solaris, and most Unix variants.
There should be a way to simulate various users at different connection speeds.
After the tests are run, you should be able to report the transactions, URL, and number of users who visited the site.
The test cases should be asssembled in a like fashion to set up test suites.
There should be a way to test the different server and port addresses.
There should be a way to account for the user's cookies.
Performance Test
The primary goal of performance-testing is to develop effective enhancement strategies for maintaining acceptable system performance. Performance testing is a capacity analysis and planning process in which measurement data are used to predict when load levels will exhaust system resources.
The Mock Test
It is a good idea to set up s mock test before you begin your actual test. This is a way to measure the server's stressd performance. As you progress with your stress testing, you can set up a measurement of metrics to determine the efficiency of the test.
After the initial test, you can determine the breaking point for the server. It may be a processor problem or even a memory problem. You need to be able to check your log to determine the average amount of time that it takes your provessor to perform the test. Running graphics or even ASP pages can cause processor problems and a limitation every time you run your stress test.
Memory tends to be a problem with the stress test. This may be due to a memary leak or lack of memory. You need to log and monitor the amount of disk capacity during the stress test. As mentioned earlier, the bandwidth can account for the slow down of the processing of the Web site speed. If the test hanges and there is a large waiting period, your processor usage is too low to handle the a,ount of stress on the system.
Simulate Resources
It is important to be able to run system in a high-stress format so that you can actually simulate the resources and understand how to handle a specific load. For example, a bank transaction processing system may be designed to process up to 150 transactions per second, whereas an operating system may be designed to handle up to 200 separate terminals. The different tests need to be designed to ensure that the system can process the expected load. This type of testing usually involves planning a series of tests where the load is gradually increased to reflect the expected usage pattern. The stress tests can steadily increase the load on the system beyond the maximum design load until the system fails.
This type of testing has a dual function of testing the system for failure and looking for a combination of events that occur when a load is placed on the server. Stress testing can then determine if overloading the system results in loss of data or user sevice to the customers The use of stress testing is particularly relevant to an ecommerce system with Web database.
Increas Capacity Testing
When you begin your stress testing, you will want to increase your capacity testing to make sure you are able to handle the increased load of data such as ASP pages and graphics. When you test the ASP pages, you may want to create a page similar to the original page that will simulate the same items on the ASP page and have it send the information to a test bed with a process that completes just a small data output. By doing this, you will have your processor still stressing the system but not taking up the bandwidth by sending the HTML code along the full path. This will not stress the entire code but will give you a basis from which to work. Dividing the requests per second by the total number of user or threads will determine the number of transactions per second. It will tell you at what point the server will start becoming less efficient at handling the load. Let's look at an example. Let's say your test with 50 users shows your server can handle 5 requests per seconf, with 100 users it is 10 requests per second, with 200 users it is 15 requests per second, and eventually with 300 users it is 20 requests per second. Your requests per second are continually climbing, so it seems that you are obtaining steadily improving performance. Let's look at the ratios:
05/50 = 0.1
10/100 = 0.1
15/200 = 0.075
20/300 = 0.073
From this example you can see that the performance of the server is becoming less and less efficient as the load grows. This in itself is not necessarily bad (as long as your pages are still returning within your target time frame). However, it can be a useful indicator during your optimization process and does give you some indication of how much leeway you have to handle expected peaks.
Stateful testing
When you use a Web-enabled application to set a value, does the server respond correctly later on?
Privilage testing
What happens when the everyday user tries to access a control that is authorized only for adminstrators?
Speed testing
Is the Web-enabled application taking too long to respond?
Boundary Test
Boundary tests are designed to check a program's response to extreme input values. Extreme output values are generated by the input values. It is important to check that a program handles input values and output results correctly at the lower and upper boundaries. Keep in mind that you can create extreme boundary results from non-extreme input values. It is essential to analyze how to generate extremes of both types. In addition. sometime you know that there is an intermediate variable involved in processing. If so, it is useful to determine how to drive that one through the extremes and special conditions such as zero or overflow condition.
Boundary timeing testing
What happens when your Web-enabled application request times out or takes a really long time to respond?
Regression testing
Did a new build break an existing function? Repeat testing after changes for managing risk relate to product enhancement.
A regression test is performded when the tester wishes to see the progress of the testing processs by performing identical tests before and after a bug has been fixed. A regression test allows the tester to compare expeted test results with the actual results.
Regression testing's primary objective is to ensure that all bugfree features stay that way. In addition, bugs which have been fixed once should not turn up again in subsequent program versions.
Regression testing: After every software modification or before next release, we repeat all test cases to check if fixed bugs are not show up again and new and existing functions are all working correctly.
Regression testing is used to confirm that fixed bugs have, in fact, been fixed and that new bugs have not been introduced in the process, and that festures that were proven correctly functional are intact. Depending on the size of a project, cycles of regression testing may be perform once per milestone or once per build. Some bug regression testing may also be performed during each accceptance test cycle, forcusing on only the most important bugs. Regression tests can be automated.
CONDITIONS DURING WHICH REGRESSION TESTS MAY BE RUN Issu fixing cycle. Once the development team has fixed issues, a regression test can be run t ovalidate the fixes. Tests are based on the step-by-step test casess that were originally reported:
If an issue is confirmeded as fixed, then the issue report status should be changed to Closed.
If an issue is confirmed as fixed, but with side effects, then the issue report status should be changed to Closed. However, a new issue should be filed to report the side effect.
If an issue is only partially fixed, then the issue report resolution should be changed back to Unfixed, along with comments outlining the oustanding problems
Open-status regression cycle. Periodic regression tests may be run on all open issue in the issue-tracking database. During this cycle, issue status is confirmed either the report is reproducible as is with no modification, the report is reproducible with additional comments or modifications, or the report is no longer reproducible
Closed-fixed regression cycle. In the final phase of testing, a full-regression test cycle should be run to confirm the status of all fixed-closed issues.
Feature regression cycle. Each time a new build is cut or is in the final phase of testing depending on the organizational procedure, a full-regression test cycle should be run to confirm that the proven correctly functional features are still working as expected.
Database Testing
Items to check when testing a database What to test Environment toola/technique
Seach results System test environment Black Box and White Box technique
Response time System test environment Sytax Testing/Functional Testing
Data integrity Development environment White Box testing
Data validity Development environment White Box testing
Query reaponse time
The turnaround time for responding to queries in a database must be short; therefor, query response time is essential for online transactions. The results from this test will help to identify problems, such as possible bottlenecks in the network, sspecific queries, the database structure, or the hardware.
Data integrity
Data stored in the database should include such items as the catalog, pricing, shipping tables, tax tables, order database, and customer information. Testng must verify the integrity of the stored data. Testing should be done on a regular basis because data changes over time.
Data integrity tests
Data integrity can be tested as follows to ensure that the data is valid and not corrupt:
Test the creation, modification, and deletion of data in tables as specified in the business requirement.
Test to make sure that sets of radio buttons represent a fixed set of values. You should also check for NULL or EMPTY values.
Test to make sure that data is save to the database and that each values gets saved fully. You should watch for the truncation of strings and that numeric values are not rounded off.
Test to make sure that default values are stored and saved.
Test the compatibility with old data. You should ensure that all updates do not affect the data you have on file in your database.
Data validity
The most common data errors are due to incorrect data entry, called data validity errors.
Recovery testing
The system recovers from faukts and resumes processing within a predefined period of time.
The system is fault-tolerant, which means that processing faults do not halt the overall functioning of the system.
Data recovery and restart are correct in case of automatic recovery. If recovery requires human intervention, the mean time to repair the database is within predefined acceptable limits.
When testing a SQL server
If the Web site publishes from inside the SQL Server straight to a Web page, is the data accurate and of the correct data type?
If the SQL Server reads from a stored procedure to produce a Web page or if the stored procedure is changed, does the data on the page change?
If you are using FrontPage or interDev is the data connection to your pages secure?
Does the database have scheduled maintenance with a log so testers can set changes or errors?
Can the tester check to see how back ups are being handled?
Is the database secure?
When testing a Access database
If the database is creating Web pages from the datbase to a URL, is the information correct and updated? If the pages are not dynamic or Active Server pages, they will not update automatically.
If the tables in the database are linked to another database, make sure that all the links are active and giving reevant information.
Are the fields such as zip code, phone numbers, dates, currency, and social security number formateed properly?
If there are formulas in the database, do they work? How will they take care of updates if numbers change (for example, updating taxes)?
Do the forms populate the correct tables?
Is the database secure?
When test a FoxPro database
If the database is linked to other database, are the links secure and working?
If the database publishes to the Internet, is the data correct?
When data is deployed, is it still accurate?
Do the queries give accurate information to the reports?
If thedatabase performs calculations, are the calculatons accurate?
Other important Database and security feature
Credite Card Transaction
Shopping Carts
Payment Transaction Security
Secure Sockets Layer (SSL)
SSL is leading security protocol on the Internet.
When an SSL session is started, the server sends its publice key to the browser, which the browser uses to send a randomly generated secret key back to the server to have a secret key exchange for that session.
SSL is a protocol that is submitted to the WWW consortium (W3C) working group on security for consideration as a standard security hanhshake that is used to initiate the TCP/IP connection. This handshake results in the client and server agreeing on the level of security that they will use, and this will fulfill any authentication requirements for the connection. SSL's role is to encrypt and decrypt the byte stream of the application protocol being used. This means the all the inofrmation in both the HTTP request and the HTTP response are fully encrypted, including the URL the client is requesting, any submitted form contents (such as credit card numbers), anty HTTP access authorization information (user names and passwords), and all the data returned from the server to the client.
Transport Layer Security (TLS)
TLS is a majo security standard on the internet. TLS is backward compatible with SSL and use Triple Data Encryption Standard (DES) encryption.
Automated Testing
Automated testing is as simple as removing the "human factor" and letting the computer do the thinking. This can be done with integrated debug tests, to much more intricate processes. The idea of the these tests is to find bugs that are often very challenging or time intensive for human testers to find. This sort of testing can save many man hours and can be more "efficient" in some cases. But it will cost more to ask a developer to write more lines of code into the game (or an external tool) then it does to pay a tester and there is always the chance there is a bug in the bug testing program. Reusability is another problem; you may not be able to transfer a testing program from one title (or platform) to another. And of course, there is always the "human factor" of testing that can never truly be replaced.
Other successful alternatives or variation: Nothing is infallible. Realistically, a moderate split of human and automated testing can rule out a wider range of possible bugs, rather than relying solely on one or the other. Giving the tester limited access to any automated tools can often help speed up the test cycle.
Release Acceptance Test
The release acceptance test (RAT), also referred to as a build acceptance or smoke test, is run on each development release to check that each build is stable enough for further testing. Typically, this test suite consists of entrance and exit test cases plus test cases that check mainstream functions of the program with mainstream data. Copies of the RAT can be distributed to developers so that they can run the tests before submitting builds to the testing group. If a build does not pass a RAT test, it is reasonable to do the following:
Suspend testing on the new build and resume testing on the prior build until another build is received.
Report the failing criteria to the development team.
Request a new build.
Functional Acceptance Simple Test
The functional acceptance simple test(FAST) is run on each development release to check that key features of the program are appropriately accessible and functioning properly on the at least one test configuration (preferable the minimum or common configuration).This test suite consists of simple test cases that check the lowest level of functionality for each command- to ensure that task-oriented functional tests(TOFTs) cna be performed on the program. The objective is to decompose the functionality of a program down to the command level and then apply test cases to check that each command works as intended. No attention is paid to the combination of these basic commands, the context of the feature that is formed by these combined commands, or the end result of the overall feature. For example, FAST for a File/Save As menu command checks that the Save As dialog box displays. However, it does not validate that the overall file-saving feature works nor does it validate the integrity of save files.
Deployment Acceptance Test
The configuration on which the Web system will be deployed will often be much different from develop-and-test configurations. Testing efforts must consider this in the preparation and writing of test cases for installation time acceptance tests. This type of test usually includes the full installation of the applications to the targeted environments or configurations.
Task-Oriented Functional Test
The task-oriented functional test (TOFT) consists of positive test cases that are designed to verify program features by checking the task that each feature performs against specifications, user guides, requirements, and design documents. Usually, features are organized into list or test matrix format. Each feature is tested for:
The validity of the task it performs with supported data conditions under supported operating conditions.
The integrity od the task's end result
The feature's integrity when used in conjunction with related features
Forced-Error Test
The forced-error test (FET) consists of negative test cases that are designed to force a program into error conditions. A list of all error messages that the program issues should be generated. The list is used as a baseline for developing test cases. An attempt is made to generate each error message in the list. Obviously, test to validate error-handling schemes cannot be performed until all the handling and error message have been coded. However, FETs should be thought through as early as possible. Sometimes, the error messages are not available. The error cases can still be considered by walking through the program and deciding how the program might fail in a given user interface such as a dialog or in the course of executing a given task or printing a given report. Test cases should be created for each condition to determine what error message is generated.
Real-world User-level Test
These tests simulate the actions customers may take with a program. Real-World user-level testing often detects errors that are otherwise missed by formal test types.
Exploratory Test
Exploratory Tests do not involve a test plan, checklist, or assigned tasks. The strategy here is to use past testing experience to make educated guesses about places and functionality that may be problematic. Testing is then focused on those areas. Exploratory testing can be scheduled. It can also be reserved for unforeseen downtime that presents itself during the testing process.
Compatibility and Configuration Testing
Compatibility and configuration testing is performed to check that an application functions properly across various hardware and software environments. Often, the strategy is to run the functional acceptance simple tests or a subset of the task-oriented functional tests on a range of software and hardware configurations. Sometimes, another strategy is to create a specific test that takes into account the error risks associated with configuration differences. For example, you might design an extensive series of tests to check for browser compatibility issues. Software compatibility configurations include variances in OS versions, input/output (I/O) devices, extension, network software, concurrent applications, online services and firewalls. Hardwere configurations include variances in manufacturers, CPU types, RAM, graphic display cards, video capture cards, sound cards, monitors, network cards, and connection types(e.g. T1, DSL, modem, etc..).
Documentation
Testing of reference guides and user guises check that all features are reasonably documented. Every page of documentation should be keystroke-tested for the following errors:
Accuracy of every statement of fact
Accuracy of every screen shot, figure and illustration
Accuracy of placement of figures and illustration
Accuracy of every tutorial, tip, and instruction
Accuracy of marketing collateral (claims, system requirements, and screen shots)
Accuracy of downloadable documentation (PDFs, HTML, or test files)
Online Help Test
Online help tests check the accuracy of help contents, correctness of features in the help system, and functionality of the help system.
Install/uninstall Test
Web system often require both client-side and server-side installs. Testing of the installer checks that installed features function properly--including icons, support documentation , the README file, and registry keys. The test verifies that the correct directories are created and that the correct system files are copied to the appropriate directories. The test also confirms that various error conditions are detected and handled gracefully.
Testing of the uninstaller checks that the installed directories and files are appropriately removed, that configuration and system-related files are also appropriately removed or modified, and that the operating environment is recovered in its original state.
User Interface Tests
Easy-of-use UI testing evaluates how intuitive a system is. Issues pertaining to navigation, usability, commands, and accessibility are considered. User interface functionality testing examines how well a UI operates to specifications.
AREAS COVERED IN UI TESTING
Usability
Look and feel
Navigation controls/navigation bar
Instructional and technical information style
Images
Tables
Navigation branching
Accessibility
External Beta Testing
External beta testing offers developers their first glimpse at how users may actually interact with a program. Copies of the program or a test URL, sometimes accompanied with letter of instruction, are sent out to a group of volunteers who try out the program and respond to questions in the letter. Beta testing is black-box, real-world testing. Beta testing can be difficult to manage, and the feedback that it generates normally comes too late in the development process to contribute to improved usability and functionality. External beta-tester feedback may be reflected in a README file or deferred to future releases.
Security Tests
Security measures protect Web systems from both internal and external threats. E-commerce concerns and the growing popularity of Web-based applications have made security testing increasingly relevant. Security tests determine whether a company's security policies have been properly implemented; they evaluate the functionality of existing systems, not whether the security policies that have been implemented are appropriate.
PRIMARY COMPONENTS REQUIRING SECURITY TESTING
Application software
Database
Servers
Client workstations
Networks
Unit Tests
Unit tests are positive tests that evaluate the integrity of software code units before they are integrated with other software units. Developers normally perform unit testing. Unit testing represents the first round of software testing--when developers test their own software and fix errors in private.
Click-Stream Testing
Click stream Testing is to show which URLs the user clicked, The Web site's user activity by time period during the day, and other data otherwise found in the Web server logs. Popular choice for Click-Stream Testing statistics include Keynote Systems Internet weather report , WebTrends log analysis utility, and the NetMechanic monitoring service.
Disadvantage: Click-Stream Testing statistics reveal almost nothing about the user's ability to achieve their goals using the Web site. For example, a Web site may show a million page views, but 35% of the page views may simply e pages with the message "Found no search results," With Click-Stream Testing, there's no way to tell when user reach their goals.
Click-stream measurement tests
Makes a request for a set of Web pages and records statiestics about the response, including total page views per hour, total hits per week, total user sessions per week, and derivatives of these numbers. The downside is that if your Web-enabled application takes twics as many pages as it should for a user to complete his or her goal, the click stream test makes it look as though your Web site is popular, while to the user your Web site is frustrating.
HTML content-checking tests
HTML content checking tests makes a request to a Web page, parses the response for HTTP hyperlinks, requests hyperlinks from their associated host, and if the links returned successful or exceptional conditions. The downside is that the hyperlinks in a Web-enalbled application are dynamic and can change, depending on the user's actions. There is little way to know the context of the hyperlinks in a Web-enabled application. Just checking the links' validity is meaningless if not misleading. These tests were meant to test static Web sites, not Web-enabled application
Web-Enabled Application Measurement Tests
Meantime between failures in seconds
Amount of time in seconds for each user session, sometimes know as transaction
Application availability and peak usage periods.
Which media elements are most used ( for example, HTML vs. Flash, JavaScript vs. HTML forms, Real vs. Windows Media Player vs. QuickTime)
Ping tests
Ping tests use the Internet Control Message Protocol(ICMP) to send a ping request to a server. If the ping returns, the server is assumed to be alive and well. The downside is that usually a Web server will continue to return ping requests even when the Web-enable application has crashed.
Unit Testing
Unit testing finds problems and errors at the module level before the software leaves development. Unit testing is accomplished by adding a small amount of the code to the module that validates the module's responses.
System-Level Test
System-level tests consists of batteris of tests that are designed to fully exercise a program as a whole and check that all elements of the integrated system function properly.
Functional System Testing
System tests check that the software functions properly from end-to-end. The components of the system include: A database, Web-enable application software modules, Web servers, Web-enabled application frameworks deploy Web browser software, TCP/IP networking routers, media servers to stream audio and video, and messaging services for email.
A common mistake of test professionals is to believe that they are conducting system tests while they are actually testing a single component of the system. For example, checking that the Web server returns a page is not a system test if the page contains only a static HTML page.
System testing is the process of testing an integrated hardware and software system to verify that the system meets its specified requirements. It verifies proper execution of the entire set of application components including interfaces to other applications. Project teams of developers and test analysts are responsible for ensuring that this level of testing is performed.
System testing checklist include question about:
Functional completeness of the system or the add-on module
Runtime behavior on various operating system or different hardware configurantions.
Installability and configurability on various systems
Capacity limitation (maximum file size, number of records, maximum number of concurrent users, etc.)
Behavior in response to problems in the programming environment (system crash, unavailable network, full hard-disk, printer not ready)
Protection against unauthorized access to data and programs.
"black-box" (or functional) testing
Black Box Testing is testing without knowledge of the internal workings of the item being tested. The Outside world comes into contact with the test items, --only through the application interface ,,,, an internal module interface, or the INPUT/OUTPUT description of a batch process. They check whether interface definitions are adhered to in all situation and whether the product conform to all fixed requirements. Test cases are created based on the task descriptions.
Black Box Testing assumes that the tester does not know anything about the application that is going to be tested. The tester needs to understand what the program should do, and this is achieved through the business requirements and meeting and talking with users.
Funcional tests: This type of tests will evaluate a specific operating condition using inputs and validating results. Functional tests are designed to test boundaries. A combination of correst and incorrect data should be used in this type of test.
Scalability and Performance Testing
Scalability and performance testing is the way to understand how the system will handle the load cause by many concurrent users. In a Web environment concurrent use is measured as simply the number of users making requests at the same time.
Performance testing is designed to measure how quickly the program completes a given task. The primary objective is to determine whether the processing speed is acceptable in all parts of the program. If explicit requirements specify program performance, then performance test are often performed as acceptance tests.
As a rule, performance tests are easy to automate. This makes sense above all when you want to make a performance comparison of different system conditions while using the user interface. The capture and automatic replay of user actions during testing eliminates variations in response times.
This type of test should be designed to verify response and excution time. Bottlenecks in a system are generally found during this stage of testing.
Stress Testing
Overwhelm the product for performance, reliability, and efficiency assessment; To find the breakpoint when system is failure; to increase load regressively to gather information for finding out maximum concurrent users.
Stress tests force programs to operate under limited resource conditions. The goal is to push the upper functional limits of a program to ensure that it can function correctly and handle error conditions gracefully. Examples of resources that may be artificially manipulated to create stressful conditions include memory, disk space, and network bandwidth. If other memory-oriented tests are also planned, they should be performed here as part of the stress test suite. Stress tests can be automated.
Breakpoint:
the capabilites and weakness of the product:
High volunmes of data
Device connections
Long transation chains
Stress Test Environment:
As you set up your testing environment for a stress test, you need to make sure you can answer the following questions:
Will my test be able to support all the users and still maintain performance?
Will my test be able to simulate the number of transactions that pass through in a matter of hours?
Will my test be able to uncover whether the system will break?
Will my server crash if the load continues over and over?
The test should be set up so that you can simulate the load; for example:
If you have a remote Web site you should be able to monitor up to four Web sites or URLs.
There should be a way to monitor the load intervals.
The load test should be able to simulate the SSL (Secure Server)
The test should be able to simulate when a user submits the Form Data (GET method)
The test should be set up to simulate and authentical the keyword verification.
The test should be able to simulate up to six email or pager mail addresses and an alert should occur when there is a failure.
It is important to remember when stressing your Web site to give a certain number of users a page to stress test and give them a certain amount of time in which to run the test.
Some of the key data features that can help you measure this type of stress test, determine the load, and uncover bottlenecks in the system are:
Amount of memory available and used
The processor time used
The number of requests per second
The amount of time it takes ASP pages to be set up.
Server timing errors.
Load Testing
The process of modeling application usage conditions and performing them against the application and system under test, to analyze the application and system and determine capacity, throughout speed, transation handling capabilities, scalabilities and reliability while under under stress.
This tyoe of test is designed to identify possible overloads to the system such as too many users signed on to the system, too many terminals on the network, and network system too slow.
Load testing a simulation of how a browser will respond to intense use by many individuals. The Web sessions can be recorded live and set up so that the test can be run during peak times and also during slow times. The following are two different types of load tests:
Single session - A single session should be set up on browser that will have one or multiple responses. The timeing of the data should be put in a file. After the test, you can set up a separate file for report analysis.
Multiple session - a multiple session should be developed on multiple browsers with one or multiple responses. The multivariate statistical methods may be needed for a complex but general performance model
When performing stress testing, looping transactions back on themselves so that the system stresses itself simulates stress loads and may be useful for finding synchronization problems and timing bugs, Web priority problems, memory bugs, and Windows problems using API. For example, you may want ot simulate an incoming message that is then put out on a looped-back line; this in turn will generate another incoming message. The nyou can use another system of comparable size to create the stress load.
Memory leaks are often found under stress testing. A memory leak occurs when a test leaves allocated memory behind and does not correctly return the memory to the memory allocation scheme. The test seems to run correctly, but after several iteration available memory is reduced until the system fails.
Peak Load and Testing Parameters:
Determining your peak load is important before beginning the assessment of the Web site test. It may mean more than just using user requests per second to stress the system. There should be a combination of determinants such as requests per second , processor time, and memory usage. There is also the consideration of the type of information that is on your Web page from graphics and code processing, such as scripts, to ASP pages. Then it is important to determine what is fast and what is slow for your system. The type of connection can be a critical component here, such as T1 or T3 versus a modem hookup. After you have selected your threshold, you can stress your system to additional limits.
As a tester you need to set up test parameters to make sure you can log the number of users coming into and leaving the test. This should be started in a small way and steadily increased. The test should also begin by selecting a test page that may not have a large amount of graphics and steadily increasing the complexity of the test by increasing the number of graphics and image requests. Keep in mind that images will take up additional bandwidth and resources on the server but do not really have a large impact on the server's processor.
Another important item to remember is that you need to account for the length of time the user will spend surfing each page. As you test, you should set up a log to determine the approximate time spend on each page, whether it is 25 or 30 seconds. It may be recorded that each user spends at least 30 seconds on each page, and that will produce a heightened response for the server. As the request is queued, and this will be analyzed as the test continues.
Load/Volume Test
Load/volume tests study how a program handles large amounts of data, excessive calculations, and excessive processing. These tests do not necessarily have to push or exceed upper functional limits. Load/volume tests can, and usually must, be automated.
Focus of Load/Volume Tesing
Pushing through large amounts of data with extreme processing demands.
Requesting many processes simulateously.
Repeating tasks over a long period of time
Load/volume tests, which involve extreme conditions, are normally run after the execution of feature-level tests, which prove that a program functions correctly under normal conditions.
Difference between Load and Strees testing
The idea of stress testing is to find the breaking point in order to find bugs that will make that break potentially harmful. Load testing is merely testing at the highest transaction arrival rate in performance testing to see the resource contention, database locks etc..
Web Capacity Testing Load and Stress
The performance of the load or stress test Web site should be monitored with the following in mind:
The load test should be able to support all browser
The load test should be able to support all Web server.
The tool should be able to simulate up 500 users or playback machines
The tool should be able to run on WIndows NT, Linux, Solaris, and most Unix variants.
There should be a way to simulate various users at different connection speeds.
After the tests are run, you should be able to report the transactions, URL, and number of users who visited the site.
The test cases should be asssembled in a like fashion to set up test suites.
There should be a way to test the different server and port addresses.
There should be a way to account for the user's cookies.
Performance Test
The primary goal of performance-testing is to develop effective enhancement strategies for maintaining acceptable system performance. Performance testing is a capacity analysis and planning process in which measurement data are used to predict when load levels will exhaust system resources.
The Mock Test
It is a good idea to set up s mock test before you begin your actual test. This is a way to measure the server's stressd performance. As you progress with your stress testing, you can set up a measurement of metrics to determine the efficiency of the test.
After the initial test, you can determine the breaking point for the server. It may be a processor problem or even a memory problem. You need to be able to check your log to determine the average amount of time that it takes your provessor to perform the test. Running graphics or even ASP pages can cause processor problems and a limitation every time you run your stress test.
Memory tends to be a problem with the stress test. This may be due to a memary leak or lack of memory. You need to log and monitor the amount of disk capacity during the stress test. As mentioned earlier, the bandwidth can account for the slow down of the processing of the Web site speed. If the test hanges and there is a large waiting period, your processor usage is too low to handle the a,ount of stress on the system.
Simulate Resources
It is important to be able to run system in a high-stress format so that you can actually simulate the resources and understand how to handle a specific load. For example, a bank transaction processing system may be designed to process up to 150 transactions per second, whereas an operating system may be designed to handle up to 200 separate terminals. The different tests need to be designed to ensure that the system can process the expected load. This type of testing usually involves planning a series of tests where the load is gradually increased to reflect the expected usage pattern. The stress tests can steadily increase the load on the system beyond the maximum design load until the system fails.
This type of testing has a dual function of testing the system for failure and looking for a combination of events that occur when a load is placed on the server. Stress testing can then determine if overloading the system results in loss of data or user sevice to the customers The use of stress testing is particularly relevant to an ecommerce system with Web database.
Increas Capacity Testing
When you begin your stress testing, you will want to increase your capacity testing to make sure you are able to handle the increased load of data such as ASP pages and graphics. When you test the ASP pages, you may want to create a page similar to the original page that will simulate the same items on the ASP page and have it send the information to a test bed with a process that completes just a small data output. By doing this, you will have your processor still stressing the system but not taking up the bandwidth by sending the HTML code along the full path. This will not stress the entire code but will give you a basis from which to work. Dividing the requests per second by the total number of user or threads will determine the number of transactions per second. It will tell you at what point the server will start becoming less efficient at handling the load. Let's look at an example. Let's say your test with 50 users shows your server can handle 5 requests per seconf, with 100 users it is 10 requests per second, with 200 users it is 15 requests per second, and eventually with 300 users it is 20 requests per second. Your requests per second are continually climbing, so it seems that you are obtaining steadily improving performance. Let's look at the ratios:
05/50 = 0.1
10/100 = 0.1
15/200 = 0.075
20/300 = 0.073
From this example you can see that the performance of the server is becoming less and less efficient as the load grows. This in itself is not necessarily bad (as long as your pages are still returning within your target time frame). However, it can be a useful indicator during your optimization process and does give you some indication of how much leeway you have to handle expected peaks.
Stateful testing
When you use a Web-enabled application to set a value, does the server respond correctly later on?
Privilage testing
What happens when the everyday user tries to access a control that is authorized only for adminstrators?
Speed testing
Is the Web-enabled application taking too long to respond?
Boundary Test
Boundary tests are designed to check a program's response to extreme input values. Extreme output values are generated by the input values. It is important to check that a program handles input values and output results correctly at the lower and upper boundaries. Keep in mind that you can create extreme boundary results from non-extreme input values. It is essential to analyze how to generate extremes of both types. In addition. sometime you know that there is an intermediate variable involved in processing. If so, it is useful to determine how to drive that one through the extremes and special conditions such as zero or overflow condition.
Boundary timeing testing
What happens when your Web-enabled application request times out or takes a really long time to respond?
Regression testing
Did a new build break an existing function? Repeat testing after changes for managing risk relate to product enhancement.
A regression test is performded when the tester wishes to see the progress of the testing processs by performing identical tests before and after a bug has been fixed. A regression test allows the tester to compare expeted test results with the actual results.
Regression testing's primary objective is to ensure that all bugfree features stay that way. In addition, bugs which have been fixed once should not turn up again in subsequent program versions.
Regression testing: After every software modification or before next release, we repeat all test cases to check if fixed bugs are not show up again and new and existing functions are all working correctly.
Regression testing is used to confirm that fixed bugs have, in fact, been fixed and that new bugs have not been introduced in the process, and that festures that were proven correctly functional are intact. Depending on the size of a project, cycles of regression testing may be perform once per milestone or once per build. Some bug regression testing may also be performed during each accceptance test cycle, forcusing on only the most important bugs. Regression tests can be automated.
CONDITIONS DURING WHICH REGRESSION TESTS MAY BE RUN Issu fixing cycle. Once the development team has fixed issues, a regression test can be run t ovalidate the fixes. Tests are based on the step-by-step test casess that were originally reported:
If an issue is confirmeded as fixed, then the issue report status should be changed to Closed.
If an issue is confirmed as fixed, but with side effects, then the issue report status should be changed to Closed. However, a new issue should be filed to report the side effect.
If an issue is only partially fixed, then the issue report resolution should be changed back to Unfixed, along with comments outlining the oustanding problems
Open-status regression cycle. Periodic regression tests may be run on all open issue in the issue-tracking database. During this cycle, issue status is confirmed either the report is reproducible as is with no modification, the report is reproducible with additional comments or modifications, or the report is no longer reproducible
Closed-fixed regression cycle. In the final phase of testing, a full-regression test cycle should be run to confirm the status of all fixed-closed issues.
Feature regression cycle. Each time a new build is cut or is in the final phase of testing depending on the organizational procedure, a full-regression test cycle should be run to confirm that the proven correctly functional features are still working as expected.
Database Testing
Items to check when testing a database What to test Environment toola/technique
Seach results System test environment Black Box and White Box technique
Response time System test environment Sytax Testing/Functional Testing
Data integrity Development environment White Box testing
Data validity Development environment White Box testing
Query reaponse time
The turnaround time for responding to queries in a database must be short; therefor, query response time is essential for online transactions. The results from this test will help to identify problems, such as possible bottlenecks in the network, sspecific queries, the database structure, or the hardware.
Data integrity
Data stored in the database should include such items as the catalog, pricing, shipping tables, tax tables, order database, and customer information. Testng must verify the integrity of the stored data. Testing should be done on a regular basis because data changes over time.
Data integrity tests
Data integrity can be tested as follows to ensure that the data is valid and not corrupt:
Test the creation, modification, and deletion of data in tables as specified in the business requirement.
Test to make sure that sets of radio buttons represent a fixed set of values. You should also check for NULL or EMPTY values.
Test to make sure that data is save to the database and that each values gets saved fully. You should watch for the truncation of strings and that numeric values are not rounded off.
Test to make sure that default values are stored and saved.
Test the compatibility with old data. You should ensure that all updates do not affect the data you have on file in your database.
Data validity
The most common data errors are due to incorrect data entry, called data validity errors.
Recovery testing
The system recovers from faukts and resumes processing within a predefined period of time.
The system is fault-tolerant, which means that processing faults do not halt the overall functioning of the system.
Data recovery and restart are correct in case of automatic recovery. If recovery requires human intervention, the mean time to repair the database is within predefined acceptable limits.
When testing a SQL server
If the Web site publishes from inside the SQL Server straight to a Web page, is the data accurate and of the correct data type?
If the SQL Server reads from a stored procedure to produce a Web page or if the stored procedure is changed, does the data on the page change?
If you are using FrontPage or interDev is the data connection to your pages secure?
Does the database have scheduled maintenance with a log so testers can set changes or errors?
Can the tester check to see how back ups are being handled?
Is the database secure?
When testing a Access database
If the database is creating Web pages from the datbase to a URL, is the information correct and updated? If the pages are not dynamic or Active Server pages, they will not update automatically.
If the tables in the database are linked to another database, make sure that all the links are active and giving reevant information.
Are the fields such as zip code, phone numbers, dates, currency, and social security number formateed properly?
If there are formulas in the database, do they work? How will they take care of updates if numbers change (for example, updating taxes)?
Do the forms populate the correct tables?
Is the database secure?
When test a FoxPro database
If the database is linked to other database, are the links secure and working?
If the database publishes to the Internet, is the data correct?
When data is deployed, is it still accurate?
Do the queries give accurate information to the reports?
If thedatabase performs calculations, are the calculatons accurate?
Other important Database and security feature
Credite Card Transaction
Shopping Carts
Payment Transaction Security
Secure Sockets Layer (SSL)
SSL is leading security protocol on the Internet.
When an SSL session is started, the server sends its publice key to the browser, which the browser uses to send a randomly generated secret key back to the server to have a secret key exchange for that session.
SSL is a protocol that is submitted to the WWW consortium (W3C) working group on security for consideration as a standard security hanhshake that is used to initiate the TCP/IP connection. This handshake results in the client and server agreeing on the level of security that they will use, and this will fulfill any authentication requirements for the connection. SSL's role is to encrypt and decrypt the byte stream of the application protocol being used. This means the all the inofrmation in both the HTTP request and the HTTP response are fully encrypted, including the URL the client is requesting, any submitted form contents (such as credit card numbers), anty HTTP access authorization information (user names and passwords), and all the data returned from the server to the client.
Transport Layer Security (TLS)
TLS is a majo security standard on the internet. TLS is backward compatible with SSL and use Triple Data Encryption Standard (DES) encryption.
Guidelines for Test Case Structure and Writing
Guidelines for Test Case Structure and Writing
In general all test scenarios will be contained in a single test case document by type. If these test cases become too large, then test cases can be split into smaller documents. The goal is to have a manageable list of test cases for tracking and reporting purposes.
• Preconditions must be available for the test case
• All data requirements for the test cases have been identified.
• All the interface requirements for the test cases are identified
• Main Flow (which is denoted by "M" in place of "Z") - The main flow should outline the simplest happy path of the use case, it should avoid optional steps in main flow until the simple case has been executed. Main flow 01 typically will be the simplest case.
• All the alternate scenarios are addressed in test cases and denoted by AF01, AF02…etc in the test cases.
• All the Business Rules are addressed in Test case/ Design Steps
• All the Error Messages are addressed in the design steps
• All the interface related test cases are available.
PN: For interface related test cases we only check the feedback coming from the interface and feedback going into the interface
• When a use case is calling another use case only conditions required to fulfill the first use case will be tested in the second use case. Ex. Use case A calls a screen in use case B. In that screen user needs to fill up a form, click continue button to come back to use case A. Here the user will not do details validation checking of use case B.
• Negative Flows (denoted by "N") - The negative test cases, flows that return error responses
• User Interface (denoted by "U") - This test cases validates the user interface, this checks that the expected fields and field values are correct according to the Use cases, validated via the DB views or other external client systems. The following standard checklist will be used to write UI relates test cases:
UI Testing Checklist:
1. Check that mandatory fields cannot be left blank before moving on the process.
2. "For Date fields a Mask should be present to indicate one of the valid input formats :
-
dd/mm/yyyy (e.g. 12/01/2006)
-
dd/Mmm (e.g.""12/Jan"")
-
Mmm/yyyy (e.g. ""Jan/2006"")"
3. Ensure javascript causes the / separators to be automatically added for date fields.
4. "Date fields accept only valid dates.
5. Check 29th Febuary for a leap year (2008) and non leap year.
6. Invalid dates include : 99/99/9999, 29/02/2005 or 31/04/2005
7. Today, previous & future dates are allowed unless otherwise specified.
8. All valid dates are allowed unless working days has been specified."
9. For Time fields a Mask should be present to indicate the valid input format which is hh:mm.
10. "Time fields accept only valid times of day.
11. Seconds are not required unless specified."
12. "Amount fields should only accept numerical values. (Numeric with 2 decimal places).
13. If no decimal point is entered it is assumed to be a whole number (i.e .00).
14. This format must be automatically enforced when the field is exited (ie the decimal places are put in for you)
15. Negatives and zero amounts are not allowed unless specified.
16. Ensure the maximum length is checked. It is assumed to be 16 (9,999,999,999.99) unless specified otherwise."
17. Ensure javascript causes the - separators to be automatically added for sort codes.
18. "Numeric fields (e.g. account number, sort code, telephone number) only accept numeric data. Should be positive, none zero unless specified otherwise.
19. Check minimum value, maximum value, negative value and zero."
20. "Rate fields (e.g. interest rate) only accept numeric data.
21. The % symbol can be entered but is ignored.
22. Should be positive, none zero unless specified otherwise.
23. Check minimum value, maximum value, negative value and zero."
24. "Text fields will accept all forms of text (Including alphanumeric, carriage return etc.) up to specified limit for the field. If the content is significant, a range of valid values will be tested.
25. Ensure fields hold values up to the maximum possible value. Always use capital Ws as these are the largest characters, and enter random alphanumeric characters (especially carriage return). Ensure that text can be entered in any case."
26. Radio buttons are mutually exclusive.
27. Tick box can be disabled as well as enabled and are not mutually exclusive.
Error testing:
28. Enumerated values - test values which are not legal for this operation
29. Numeric values - include just > maximum, just <>
30. "For text, include (as appropriate) invalid lengths, invalid contents
31. Supply null values (e.g. null strings) where these are invalid
32. Provoke all possible error responses / diagnostics where achievable using just input values
33. Ensure all errors are reported by either a javascript popup (where local validation is possible), or appear alongside an exclamation icon at the top of the page (where server validation is required).
34. Ensure null values can be entered where these are valid values."
Saturday, January 05, 2008
Why do you need self motivation?
- You cannot always rely on others to encourage you, if you have positive friends who are always there when you need them then you are indeed lucky and very much in the minority. If you are lonely or have few friends when you face any difficulties in your life you must rely on your own motivation to get you through. Lack of self motivation at that time could lead to depression.
- You need self motivation to achieve because if you don't encourage yourself to accept opportunity and challenge who will?
- To plan and find direction in your life
- To take up a new activity, hobby or challenge
- To be enthusiastic about life and living
- To have the courage to see things through despite setbacks or negative comments from others
- Focus on what you really enjoy doing, maybe on something you want to take up or on a hobby you've always wanted to devote more time to. What's stopping you? Think about giving it priority to start doing what you love doing.
- Make a list of things you'd like to improve on and how you're going to do it
- Review all the successes you've enjoyed in every area of your life, totally forget any negatives, just positive successes here!
- Start an exercise program - force yourself to do it, it'll make you feel much more positive.
- Contact a positive friend and have a chat.
- Read inspiring books that will help heal your mind and improve your attitude.
- This is not an exhaustive list because self motivation is so important in every aspect of your life. The very fact that you are here searching for answers and trying to improve yourself shows that you are motivated.
Do you lack self motivation ?
How can you become motivated again? Good question!
Here are
some suggestions:
- Focus on what you really enjoy doing, maybe on something you want to take up or on a hobby you've always wanted to devote more time to. What's stopping you? Think about giving it priority to start doing what you love doing.
- Make a list of things you'd like to improve on and how you're going to do it
- Review all the successes you've enjoyed in every area of your life, totally forget any negatives, just positive successes here!
- Start an exercise program - force yourself to do it, it'll make you feel much more positive.
- Contact a positive friend and have a chat.
- Read inspiring books that will help heal your mind and improve your attitude.
Monday, December 31, 2007
TOD
Keeping yourself motivated can be challenging. How often do you encourage yourself? Motivation can come in many forms, but the best kind of motivation is when you motivate yourself. Here are some simple self-motivation tips you can start using today
• Give yourself an incentive
• Think positive
• Acknowledge your success thus far
• Make it fun
Attitudes Can Shape Your Life
We all have experienced the effects of a negative attitude. It’s rotten, and it eats at your mind, heart and soul. A negative attitude is dangerous and it eats at you slowly and quietly. Sometimes you don’t even realize what a negative attitude you have, until it’s pointed out to you.
Power of Positive Thinking
Do you practice positive thinking everyday? The art of positive thinking is a simple one, but one that is practiced and learned with time. Test yourself to see where your positivity lies.
• Give yourself an incentive
• Think positive
• Acknowledge your success thus far
• Make it fun
Attitudes Can Shape Your Life
We all have experienced the effects of a negative attitude. It’s rotten, and it eats at your mind, heart and soul. A negative attitude is dangerous and it eats at you slowly and quietly. Sometimes you don’t even realize what a negative attitude you have, until it’s pointed out to you.
Power of Positive Thinking
Do you practice positive thinking everyday? The art of positive thinking is a simple one, but one that is practiced and learned with time. Test yourself to see where your positivity lies.
Friday, November 30, 2007
What is SRS ? How it is different from URS
SRS is nothing but a Software Requirement Specification. Is a document.This document contains a software specifications. Using this document only the Developer can develop the Software.
The term URS is nothing but a User Requirement Specification.
Both SRS and URS is same.
The term URS is nothing but a User Requirement Specification.
Both SRS and URS is same.