How Good Is Your Software QA Process?

How Good Is Your Software QA Process?

Software testing is invaluable in providing your end-users with a fully functional, scalable, and accessible product and a seamless, bug-free user experience. But only when it’s done right. Take our quiz to check if your software quality assurance process is up to global standards and is sufficient to fully cover all aspects of your product development.
1/15

1/15. What is the size of your company?

2/15. What software development methodology do you follow?

3/15. What is your current development and QA phase?

4/15. Do you measure the KPIs of your QA Team? How?

Some of the things you should consider to improve your software QA process significantly:

  • If you measure your team’s performance by the number of bugs raised by the team, they will raise a bug for every misplaced pixel. Modifying this to use the number of bugs above a specified severity level will get your misplaced pixels classified as urgent.
  • Using the number of bugs tested or cleared as a KPI tends to encourage testers to be less thorough and focus on checking that the bug as described is fixed without touching anything that might affect their statistics.
  • Any metric you use will be heavily dependent on the context – a tester working with a developer who unit tests thoroughly, makes sure his integration is clean and checks that he’s doing the right thing, will not find many bugs in that developer’s work. A project that’s had its timeline cut to meet an external deadline will generate more bugs and have more bugs reach the customer than one that’s been well-planned and executed without major time or resource stress.
  • Not one metric can capture the complexities of software testing. Who is doing a better job: a tester who works through a complex new feature and finds a dozen of severe bugs in a week, or the one who works through a complex new feature and finds one catastrophic bug in that week? The answer is neither: both testers are finding the right information, but the information they find differs because they’re working in different areas with different developers.

5/15. Do you have enough software testing resources for your product?

Software testing is a very important part of product development. A primary purpose of testing is to detect software failures so that defects may be discovered and corrected. Testing cannot establish that a product functions properly under all conditions, but only that it does not function properly under specific conditions. The scope of software testing often includes the examination of code as well as the execution of that code in various environments and conditions as well as examining the aspects of code: does it do what it is supposed to do and does it do what it needs to do. In the current culture of software development, a software testing organization may be separate from the development team.

6/15. What is the ratio of QA engineers to developers?

Whilst ratios are always useful and may be viewed as an expedient and simple way to arrive at an estimate, there is a significant risk when one organization uses another organization’s ratios and then applies these to their project without regard to differences in technology, software quality assurance process maturity, and skill levels.

Part of the problem with identifying a reasonably accurate ratio is not just the reasons above but also the lack of data collected by organizations on this information. Thus, for many, it is not an exact figure that is sought but an assurance that the testing estimates are reasonably correct and there is a degree of confidence that the software testing can be undertaken within the project time frame with the staff and budget allocated to this function.

7/15. Is there a defined quality assurance process document in place and being used?

Testing is a process rather than a single activity. This process starts with test planning, then designing test cases, preparing for the execution, and evaluating the status until the test closure. So, we can divide the activities within the fundamental quality assurance process into the following basic steps:
1) Planning and Control
2) Analysis and Design
3) Implementation and Execution
4) Evaluating exit criteria and Reporting
5) Test Closure activities

8/15. Do you have test documentation such as a test plan, test cases, checklists, etc?

Proper documentation is a crucial part of your software QA process. A test plan outlines the common strategy that will be applied to test an application, the resources that will be used, the test environment in which testing will be performed, and the schedule of testing activities along with the limitations. Typically, Software Quality Assurance Team Lead is in charge of writing a Test Plan.

A test case is a complexity of inputs, series of steps, and conditions that can be used during the QA process. The key point of this activity is to find out whether the software is successful in terms of its functionality and other aspects. There are various types of test cases such as logical, functional, error, negative test cases, physical test cases, UI test cases, etc. that could be used for the execution of smoke testing or regression testing.

Checklist – is a list of tests that should be run in a definite procedure. It helps to understand if testing is fully run and how many of those tests failed. It also helps to better structure the testing process, by listing all the features that need to be tested. Test order in the checklist may be strict as well as random.

Availability of the test documentation types (test plan, test cases, checklists) on the project depends on the budget, QA team size, development process, etc.

9/15. Do you use any test management system (TMS)?

Test management tools are used to store information on how testing is to be done, plan testing activities, and report the status of software quality assurance activities. Test management tools are very useful for smoke testing as well as regression testing when the execution test set should be defined. The tools have different approaches to testing and thus have different sets of features. Generally, they are used to maintain and plan manual testing, run or gather execution data from automated tests, manage multiple environments, and enter information about found defects. Test management tools offer the prospect of streamlining the QA process and allow quick access to data analysis, collaborative tools, and easy communication across multiple project teams. Many test management tools incorporate the requirements helping to streamline the test case design. Tracking of defects and project tasks are done within one application to further simplify the testing.

10/15. Do you use any bug tracking system (BTS)?

A bug tracking system or defect tracking system is a software application that keeps track of reported software bugs in software development projects and it helps to better structure your software QA process. It may be regarded as a type of issue tracking system.

Many bug tracking systems, such as those used by most open-source software projects, allow end-users to enter bug reports directly. Other systems are used only internally in a company or organization doing software development. Typically bug tracking systems are integrated with other project management software.

A bug tracking system is usually a necessary component of a professional software development infrastructure, and consistent use of a bug or issue tracking system is considered one of the “hallmarks of a good software team”.

11/15. Do you have an isolated deployment environment for testing?

Isolated Test Environment is an exact copy from your production environment but with only the bare minimum needed to test in it. It is also in a separate network and the isolation aspect means that this separate network can not connect to the production network and vice versa. Because the networks can’t see each other, you can use the same IP addresses in both environments and this, of course, means that you don’t have to customize your servers in the test environment. There is no chance that a test user will accidentally contact a production database or mail server.
Ideally, all testing activities should be done in the isolated test environment as to not affect the development team and ongoing development process.

12/15. Do you use any quality monitoring tools for your product?

Monitoring the health and uptime of your web server or web host is an important part of the software QA process. You want to make sure your website is always available to your users, and that your website isn’t slow. Server monitoring tools allow you to gather and analyze data about your web server.
Part of the problem is that different operating systems, browsers, connection speeds, devices, and locations mean that any number of combinations could be interacting with your application. Local testing is useful but fails to identify the vast amount of variables that users can encounter when trying to use your app.

13/15. Do you have automated tests implemented and running?

End-to-end testing is a software testing methodology to test an application flow from start to end. The purpose of this testing is to simulate the real user scenario and test the system and its components to make sure it integrates well with other elements and provides the necessary level of data integrity.

It is performed from start to finish under real-world scenarios, like communication of the application with hardware, network, database, and other applications.

The main reason for carrying out this testing is to determine various dependencies of an application as well as ensuring that accurate information is communicated between various system components. It is usually performed after the completion of functional and system testing of any application.

14/15. Who is involved in the testing of your product?

Software testing can be done by a dedicated Software Quality Assurance team that can be on-site or external (outsourced or outstaffed). Depending on the project-specific and budget, testing also can be provided by stakeholders, managers, developers.
If the product is available for the users, beta-testers can be involved in the testing activities as well.

15/15. Do you have a clear Quality Assurance reporting process in place? What does it look like?

A test report is an organized summary of testing objectives, activities, and results. It is created and used to help stakeholders (product manager, analysts, testing team, and developers) understand product quality and decide whether a product, feature or defect resolution is on track for release. Test reporting should help you understand the achieved value of testing.

You have completed the quiz!

While we are getting ready with your test results, could you kindly share your email address with us. We gather this information exclusively for statistical and communicative purposes and have no intention to misuse any data you provide to us. To be sure, you may check our Privacy Policy.

Protected by Google reCAPTCHA
Privacy Policy and Terms of Service apply.