Defects Life Cycle

Defect Tracking Management

When a tester finds a defect in software or application defect is opened / created in test management tool like Quality Center, Test Director or IBM Rational. Defects can be manged as follows:

Ø  Tester opens the defect in QC and it will be in ‘NEW’ Status.

Ø  BA reviews the defect whether it’s a “Valid” Defect or “Invalid” Defect.

Ø  In Case of Valid Defects BA makes the Status as “Open” and Assigned to Development Lead.

Ø  Development Lead then changes the status to ‘Assign’ and Assigns to relevant developer.

Ø  Developer will fix the defect and change the Status of Defect to “Fixed”.

Ø  Lead will change the status of Defect to “Delivered to Test”.

Ø  Tester will now test the Defect in “Ready to Test” status.

Ø  If the Defect is fixed then the status will be changed to “Closed”

Ø  If the Defect is not fixed then the status will be changed to “Re-Open” and Step 5 to Step 8 is followed again.

Ø  In case of Invalid Defects BA will change the status to “Rejected” mentioning the reason      for Reject.

Ø  Rejected defect will be verified by tester/test lead. Based on the result defect status will be changed either to ‘OPEN’ or ‘CANCELED’ status

Ø  If the status is changed to “Open” it will be assigned back to BA and If BA is convinced with new comments in defect and also the ‘OPEN’ status, BA can assign it to Dev team/GL, if still they consider it as ‘Reject’ BA need to discuss it with team/GL before rejecting – this is to avoid number of defect reopening iterations

Ø  Dev needs to change defects status to ‘Fixed’ once they fix and required verification is done from their side. All the ‘Fixed’ defects needs to be ‘Delivered to Testing’ environment by Dev after end of each cycle, defects which are Critical/Show Stoppers can be delivered to QA environment at any point of time as hot fix by BA coordinating it with Testing team.

Ø  All the ‘Delivered to Testing’ defects will be verified by QA team during next cycle execution or during dedicated defect verification duration.

Ø  Verified defects will either be re ‘OPEN’ or ‘Closed’ based on the results

Ø  At any case BA can’t change the ‘Severity/Priority’ of the defect by their own, CDM can ask the QA team to change the ‘Severity/Priority’ of the defects by providing required info to testing team.

Ø  If BA rejects defect saying duplicate, duplicate defect number need to be added to comments area of rejected defect

Defect Life Cycle flowchart:

Ø  Fixes of Defects from UAT/Production can be verified by testing team on request/requirement [As a good methodology fixes of production defects need to be verified by testing team before fix moves to production environment]

Ø  All the ‘Deferred’ defects need to be considered for implementation in future versions as version mentioned in JIRA by BA, Testing team will help sending reminders to BA in appropriate versions

Ø  No defect should go unhandled by saying its defect but functionality is same in production or existing in production so NOT FIXED – if it is identified as defect – it needs to be fixed.

Traceability Matrix

Traceability Matrix from Software Testing perspective

This is a very helpful topic for test leads and managers (also for junior testers if they want to grow).

Traceability Matrix is used in entire software development life cycle phases:

1.     Risk Analysis phase

2.     Requirements Analysis and Specification phase

3.     Design Analysis and Specification phase

4.     Source Code Analysis, Unit Testing & Integration Testing phase

5.     Validation – System Testing, Functional Testing phase

Topics:

• What is Traceability Matrix from Software Testing perspective? (Point 5)
• Types of Traceability Matrix
• Disadvantages of not using Traceability Matrix
• Benefits of using Traceability Matrix in testing
• Step by step process of creating an effective Traceability Matrix from requirements. Sample formats of Traceability Matrix basic version to advanced version.

In Simple words - A requirements traceability matrix is a document that traces and maps user requirements [requirement Ids from requirement specification document] with the test case ids. Purpose is to make sure that all the requirements are covered in test cases so that while testing no functionality can be missed.

This document is prepared to make the clients satisfy that the coverage done is complete as end to end, this document consists of Requirement/Base line doc Ref No., Test case/Condition, and Defects/Bug id. Using this document the person can track the Requirement based on the Defect id

Note – We can make it a “Test case Coverage checklist” document by adding few more columns. We will discuss in later posts

Types of Traceability Matrix:

• Forward Traceability – Mapping of Requirements to Test cases
• Backward Traceability – Mapping of Test Cases to Requirements
• Bi-Directional Traceability - A Good Traceability matrix is the References from test cases to basis documentation and vice versa.

Why Bi-Directional Traceability is required?

Bi-Directional Traceability contains both Forward & Backward Traceability. Through Backward Traceability Matrix, we can see that test cases are mapped with which requirements.

This will help us in identifying if there are test cases that do not trace to any coverage item— in which case the test case is not required and should be removed (or maybe a specification like a requirement or two should be added!). This “backward” Traceability is also very helpful if you want to identify that a particular test case is covering how many requirements?

Through Forward Traceability – we can check that requirements are covered in which test cases? Whether is the requirements are coved in the test cases or not?

Forward Traceability Matrix ensures – We are building the Right Product.

Backward Traceability Matrix ensures – We the Building the Product Right.

Traceability matrix is the answer of the following questions of any Software Project:

• How is it feasible to ensure, for each phase of the SDLC, that I have correctly accounted for all the customer’s needs?
• How can I certify that the final software product meets the customer’s needs? Now we can only make sure requirements are captured in the test cases by traceability matrix.

Disadvantages of not using Traceability Matrix [some possible (seen) impact]:

No traceability or Incomplete Traceability Results into:

1. Poor or unknown test coverage, more defects found in production 

2. It will lead to miss some bugs in earlier test cycles which may arise in later test cycles. Then a lot of discussions arguments with other teams and managers before release.

3. Difficult project planning and tracking, misunderstandings between different teams over project dependencies, delays, etc

Benefits of using Traceability Matrix

• Make obvious to the client that the software is being developed as per the requirements.
• To make sure that all requirements included in the test cases
• To make sure that developers are not creating features that no one has requested
• Easy to identify the missing functionalities.
• If there is a change request for a requirement, then we can easily find out which test cases need to update.
• The completed system may have “Extra” functionality that may have not been specified in the design specification, resulting in wastage of manpower, time and effort.

Steps to create Traceability Matrix:

1. Make use of excel to create Traceability Matrix:

2. Define following columns:

Base Specification/Requirement ID (If any)

Requirement ID

Requirement description

TC 001

TC 002

TC 003.. So on.

3. Identify all the testable requirements in granular level from requirement document. Typical requirements you need to capture are as follows:
Used cases (all the flows are captured)
Error Messages
Business rules
Functional rules
SRS
FRS
So on…

4. Identity all the test scenarios and test flows.

5. Map Requirement IDs to the test cases. Assume (as per below table), Test case “TC 001” is your one flow/scenario. Now in this scenario, Requirements SR-1.1 and SR-1.2 are covered. So mark “x” for these requirements.

Now from below table you can conclude –

Requirement SR-1.1 is covered in TC 001

Requirement SR-1.2 is covered in TC 001

Requirement SR-1.5 is covered in TC 001, TC 003 [Now it is easy to identify, which test cases need to be updated if there is any change request].

TC 001 Covers SR-1.1, SR, 1.2 [we can easily identify that test cases covers which requirements].

TC 002 covers SR-1.3.. So on..

Requirement ID	Requirement description	TC 001	TC 002	TC 003
SR-1.1	User should be able to do this	x
SR-1.2	User should be able to do that	x
SR-1.3	On clicking this, following message should appear		x
SR-1.4			x
SR-1.5		x		x
SR-1.6				x
SR-1.7			x

This is a very basic traceability matrix format. You can add more following columns and make it more effective:

ID, Assoc ID, Technical Assumption(s) and/or Customer Need(s), Functional Requirement, Status, Architectural/Design Document, Technical Specification, System Component(s), Software Module(s), Test Case Number, Tested In, Implemented In, Verification, Additional Comments,

Software Testing Life Cycle (STLC)

Software testing life cycle identifies what test activities to carry out and when (what is the best time) to accomplish those test activities. Even though testing differs between organizations, there is a testing life cycle.

Software Testing Life Cycle consists of six (generic) phases:

·         Test Planning,

·         Test Analysis,

·         Test Design,

·         Construction and verification,

·         Testing Cycles,

·         Final Testing and Implementation and

·         Post Implementation.

Software testing has its own life cycle that intersects with every stage of the SDLC. The basic requirements in software testing life cycle is to control/deal with software testing – Manual, Automated and Performance.

Test Planning

This is the phase where Project Manager has to decide what things need to be tested, do I have the appropriate budget etc. Naturally proper planning at this stage would greatly reduce the risk of low quality software. This planning will be an ongoing process with no end point.

Activities at this stage would include preparation of high level test plan-(according to IEEE test plan template The Software Test Plan (STP) is designed to prescribe the scope, approach, resources, and schedule of all testing activities. The plan must identify the items to be tested, the features to be tested, the types of testing to be performed, the personnel responsible for testing, the resources and schedule required to complete testing, and the risks associated with the plan.). Almost all of the activities done during this stage are included in this software test plan and revolve around a test plan.

Test Analysis

Once test plan is made and decided upon, next step is to delve little more into the project and decide what types of testing should be carried out at different stages of SDLC, do we need or plan to automate, if yes then when the appropriate time to automate is, what type of specific documentation I need for testing.

Proper and regular meetings should be held between testing teams, project managers, development teams, Business Analysts to check the progress of things which will give a fair idea of the movement of the project and ensure the completeness of the test plan created in the planning phase, which will further help in enhancing the right testing strategy created earlier. We will start creating test case formats and test cases itself. In this stage we need to develop Functional validation matrix based on Business Requirements to ensure that all system requirements are covered by one or more test cases, identify which test cases to automate, begin review of documentation, i.e. Functional Design, Business Requirements, Product Specifications, Product Externals etc. We also have to define areas for Stress and Performance testing.

Test Design

Test plans and cases which were developed in the analysis phase are revised. Functional validation matrix is also revised and finalized. In this stage risk assessment criteria is developed. If you have thought of automation then you have to select which test cases to automate and begin writing scripts for them. Test data is prepared. Standards for unit testing and pass / fail criteria are defined here. Schedule for testing is revised (if necessary) & finalized and test environment is prepared.

Construction and verification

In this phase we have to complete all the test plans, test cases, complete the scripting of the automated test cases, Stress and Performance testing plans needs to be completed. We have to support the development team in their unit testing phase. And obviously bug reporting would be done as when the bugs are found. Integration tests are performed and errors (if any) are reported.

Testing Cycles

In this phase we have to complete testing cycles until test cases are executed without errors or a predefined condition is reached. Run test cases --> Report Bugs --> revise test cases (if needed) --> add new test cases (if needed) --> bug fixing --> retesting (test cycle 2, test cycle 3….).

Final Testing and Implementation

In this we have to execute remaining stress and performance test cases, documentation for testing is completed / updated, provide and complete different matrices for testing. Acceptance, load and recovery testing will also be conducted and the application needs to be verified under production conditions.

Post Implementation

In this phase, the testing process is evaluated and lessons learnt from that testing process are documented. Line of attack to prevent similar problems in future projects is identified. Create plans to improve the processes. The recording of new errors and enhancements is an ongoing process. Cleaning up of test environment is done and test machines are restored to base lines in this stage.

Software Testing Life Cycle
Phase	Activities	Outcome
Planning	Create high level test plan	Test plan, Refined Specification
Analysis	Create detailed test plan, Functional Validation Matrix, test cases	Revised Test Plan, Functional validation matrix, test cases
Design	test cases are revised; select which test cases to automate	revised test cases, test data sets, sets, risk assessment sheet
Construction	scripting of test cases to automate,	test procedures/Scripts, Drivers, test results, Bug reports.
Testing cycles	complete testing cycles	Test results, Bug Reports
Final testing	execute remaining stress and performance tests, complete documentation	Test results and different metrics on test efforts
Post implementation	Evaluate testing processes	Plan for improvement of testing process