How to Choose a Software Testing Company: Engineering Firms vs. QA Agencies

Software works, until it doesn’t. Or it works… but not exactly how you want it to work. Or it works… but is a pain to support and maintain services. Or it simply isn’t sufficiently user-friendly. Lots of issues may arise, ruin your brilliant idea, mess up your neat architecture, and tank your profits. There are several ways you can mitigate the risks of low-quality products and do it efficiently and on budget. The most common solution is to outsource some business processes.

For example, you can outsource software testing activities. Software testing services differ: it’s possible to outsource the whole testing process to an independent Quality Assurance agency or a software development company. Those two approaches are different and have their advantages and disadvantages. Let’s take a look at them in this article.          

For more than 10 years I’ve been involved in the software creation process. Having delivered dozens of flawless software products, I can explain why software testing is important: the investment you make in software testing is the key to customers satisfaction. 

Capers Jones is my role model in this arena. An American specialist in software engineering methodologies, Jones said: “High-quality software is not expensive. High-quality software is faster and cheaper to build and maintain than low-quality software, from initial development all the way through total cost of ownership.”

To give you an idea of what software quality assurance involves, consider this analogy. Suppose you go out to your favorite restaurant and order a ribeye. In order for your meal to be as yummy as possible, the steak should be:
1. Properly aged
2. Cooked exactly how you like it 
3. Delivered to your table within an acceptable time frame
4. Presented in an attractive manner.

Similarly, when you order a piece of software, the product must: 

• Be free of bugs
• Have all the features implemented according to the documentation
• Delivered within the timeframe required for product launch
• Presented in an accurate, attractive, user-friendly manner

Can you be satisfied only with the first parameter? Of course not. You wouldn’t order a nice steak without first giving the server your requirements, right? That’s a surefire way to be disappointed with your meal. Making sure the end product meets your expectation is the goal of Quality Assurance (QA). For regular people, Quality Assurance is ambiguous. It sounds fancy, but what is it really? People hear QA and they think testing or testers. But there’s a big difference between software testing and Quality Assurance. Let us first understand why quality assurance is so important. QA is not just plain monkey testing; it covers more than the routine process of clicking on pop-up menus and checking line widths in installation windows.

QA is a systematic process that includes a thorough analysis of product requirements, planning, and organization of testing procedures, test management, selection of proper testing tools, developing test cases and quality requirements, crafting “how-to” suggestions for developers, getting constant feedback from both sides (developers and clients), and so on. The goal of QA is far more complex than simply eliminating mistakes; the objective is for the product to reach the level we call fit for purpose. 

It seems logical that the sooner you start testing a product, the sooner you’ll finish, right? Not completely. The reality of testing is a bit more complex. Starting early can result in higher quality and the lower the cost of bugs, I recommend integrating Software QA stages of the software development process as soon as possible. In a perfect world, QA starts when the project gets underway from the planning and architecture stage all the way to the final release and beyond. [NOTE: Some developers have no QA strategy at all and rely solely on unit and system tests drawn up by the developers themselves. I strongly recommend against this approach as it can waste a great deal of time and money.)

Integrating QA testing processes into the development process itself must be done properly. Here’s a checklist: 

1. Make sure that QA best practices are followed
2. All necessary testing levels and types are planned out in advance
3. Establish the product release testing process 
4. Discuss the reporting process with the client and make sure all engineers are up to speed on all of the above. 

It’s also crucial to organize communication procedures and have project leads check iteratively to ensure there is no tension between the development and QA testing teams. Any issues that arise should be communicated in a polite manner so there are no barriers for productive work.

Now, let’s highlight some key points on why software quality assurance is important. 

• Suppose a rare software bug in your payment system prevents you from getting paid for the services or goods and the payment goes to a random account. Basically, you’ve just made a charitable donation. Alas, your business cannot thrive if you’re giving to charity.
• Money saver. Preventing a bug is cheaper than fixing a bug. Mending a problem here can cause a problem over there, and another somewhere, and before you know it, you’ve got a real mess. The cost of a bug (whatever it is) in the production stage is much higher than if it gets discovered and fixed earlier in the development cycle. 
• Reputation protection. One lame version of a product can damage a company’s reputation and put all further versions on the “do not buy” list. Some of us remember those disastrous smartphones by a popular brand that will remain unnamed. Their phones have gotten better over time but many consumers never purchased their products again. As with relationships, first impressions count. 
• Quality control and overall management. The documentation and specification on your project may contain all the requirements, metrics, and testing plans, but who will control their applications? You definitely need a professional.
  

Integrating software QA with development from the very beginning of every development cycle ensures the developers cooperate with QA engineers and work together to decide on any future features in terms of both implementation and testing. Fortunately, this type of collaboration doesn’t require QA and development professionals to work in the same building. Waverley’s QA engineers have deep experience working remotely. We’ve proven that it’s possible to successfully collaborate with our developers regardless of where those developers are located. We’ve honed our communication processes and have a variety of tools so the process is seamless for everyone involved.  

There are a variety of approaches to outsourced software testing. You can hire an in-house tester, work with freelance quality assurance engineers, contract a software testing agency, or you can order quality assurance services from a full-cycle software engineering company like Waverley.

For some projects, a blended model works best, such as when an in-house team works closely with an outsourced software testing team that is essentially an extension that puts more eyes on the project and/or adds a specialist you don’t have in-house. If you are currently having your product implemented in-house or by a software company, you’re probably curious if software testing outsourcing is right for you. Let’s take a look at the advantages. 

There are dozens of reasons why companies choose to outsource their software or application testing tasks.

• No need for a full QA team. Perhaps the product is too small to warrant hiring a QA team. You may be limited in the number of new hires you can bring onboard. For a lot of companies, software projects are rare or infrequent so they simply don’t need an in-house QA team. A company can get greater efficiency at a lower cost by outsourcing those aspects of a project that are not directly related to the company’s core business.
• A temporary or ongoing demand for more technical resources than the client’s company has (mobile platforms, cloud storage, or testing tools available).
• A need to speed up testing, especially of deliverables, when their human resources are limited. It is especially helpful to have teams working in different time zones when time is of the essence. A product can literally get tested around the clock. 
• Need for a larger talent pool of engineers with vast experience testing all kinds of software applications.
• Many projects require testing without bias; fresh eyes catch more inaccuracies.
• The cost of testing usually goes down if the service is obtained from Central and Eastern Europe, or India and Asia.
  Outsourced software testing saves the added overhead cost for additional office space, insurance, office supplies, as well as the recruitment process which takes time and money. 
• Taking into account everything mentioned above, mitigation of risks and a chance for your home teams to focus on other tasks such as planning, implementing final touches, or marketing.     

Now let’s look at how to choose a software testing partner for your business. The most efficient approach to software testing outsourcing is to start before implementation and find out whether your software development vendor can also be your software testing partner and deliver a full cycle of QA activities. This is the best case scenario because your QA vendor already has deep knowledge of your product.  This will play to your advantage on many levels if you want to establish a long-lasting relationship with the vendor. If you have a local team, another option is to use in-house human and tech resources and share them with the engineering vendor.   

Alternatively, you can look for the top software testing companies that deal exclusively with QA and testing. Independent software testing companies treat testing as a service model (TaaS). From 2009, when this term was introduced for the first time, TaaS has attracted attention in both academic and industry communities. This model cuts the expense of costly testing labs, in-house QA teams, meetings, heated discussions over the documentation with the team of developers, etc.

Typically TaaS increases testing speed, which is another benefit. TaaS can provide extra resources to complement your existing QA team, which means you only pay for what you need. Many companies offer a menu of services to choose from such as: quality issue analysis, mocking real-world users’ environments, vulnerability prognosis, and Cloud resources, to name a few.

Now we come to another option, which is really the “perfect world” scenario when it comes to assuring the highest possible standard for product testing and that’s to partner with a strong software developing company that also offers TaaS. Better yet, a strong software company with an impressive track record and Top Ten rating on GitHub. 

Working with a top-tier Software development company that provides full-cycle support from the idea stage to the maintenance stage avails you of a broad spectrum of talent and expertise. The type of collaborative partnership we establish with our clients makes them secure in the knowledge that we are in this together. We have the tech professionals you need and are experts at the very services you are looking to outsource–from skilled QA engineers, long-lasting tech support, dedicated managers, to high quality of software testing documentation.    

Next, let’s drill down on the options above, explain how to choose a software testing company, and look into reasons to choose one over the other. 

Independent QA outsourcing agencies provide testing services outside the development team. This may sound weird at first. Software testing and development go together like peanut butter and jelly, right? Yes, however, there are good reasons why companies choose an independent software testing company for TaaS:

• They have a small project and employing a full development + QA team doesn’t make sense.
• They have no clear vision on the proper way to perform testing.
• The development phase is in progress and they need to set up QA as soon as possible.
• There are obvious quality issues in the production environment, tons of bugs during acceptance testing, end users that rate the product from 1 to 2 stars, but never 4 and 5, etc. 
• Testing cycles are short, irregular, and/or don’t demand a deep-dive into the product as there is no concern about licensing (Clouds, tools, hardware, software, or professionals).
• The cost of testing is predictable. 
• Standardized bug reporting, test management tools and centralized testing environment can save up to 20% AND guarantee repeatability and stable quality. 
• They want to have remote access to the accumulated data stored in one location.

Along with the obvious advantages, there can be serious no-nos. They are:

• Surprise! Security issues with sensitive data you cannot risk losing. No real influence or control over the course of testing. For sure, the majority of the issues will have been discussed before testing begins, but it’s helpful to observe your software or application testing process with your own eyes;
• Standardized QA protocols or tests may not fit your product and independent software testers might not have enough knowledge to maximize quality by digging deeper;
• For tailored products in super specific areas, independent software testing companies may not be able to give the best advice on the product, i.e.: potential  cost savings, depth of documentation detailing, legal proceedings, end-user suitability, etc;
• Heavy TaaS may require intensive communication between the testing team and a Cloud support team (often external), which often requires an additional level of data sharing;
• Quality may vary from country to country due to cultural differences. Some countries don’t hold the same standards and work ethic, which can make implementing satisfactory communication difficult, especially if their work expectations are relatively low.
• After releasing the product and getting feedback from clients, you might keep paying for extra rounds of tests.

Obviously, you must be incredibly careful when choosing a software testing outsourcing agency that suits your needs. Some points to think about are whether you have:
1. A clear vision of which software testing services you need (functional testing /security testing /structural tests / accessibility testing / regression testing / usability testing / integration testing / penetration testing / performance testing / load testing / mobile testing / test automation)
2. Defined expectations in terms of whether you’re looking for long or short-term cooperation, beta testing or ongoing QA, what are the needed tests , Cloud resources, potential data security issues, etc.
3. A set of criteria that fits your needs (company size, location, budget, cultural background, communication style, testing culture, test strategy they propose, what are their testing specialties, industry experience – e.g. erp testing, ecommerce testing services)

Let’s look into some of these points. 

• Depending on the size of the product, you can feel fairly comfortable outsourcing testing tasks to an agency either fully or partially. Long-term, heavy, slow, or unique projects may require other options. 
• Decide whether you really need TaaS with powerful Cloud capacity or if a small focus group of  software testers is sufficient. Also, is the agency flexible with assigning more people to your product once it gets rolling, or are they rigid on the size of the performing team?  
• How sensitive is your data and are you willing to allow another level of data sharing (in case of third-party support teams)? Are you clear how security issues will be dealt with in the contract?   
• Do you need a full QA protocol early in the process, or just final-stage tests, or vulnerability tests? Is it enough to go with the white-box software and application testing? 
• Check feedback and reviews from other customers about how great their experience was. 
• Will a different time zone be a problem? Will the agency’s cultural peculiarities interfere with the quality you are expecting? What’s the management hierarchy in the agency?     
• How easy is the communication especially in the situation of the lockdown (thanks, the VIRUS). Can the agency work properly after having moved all its routines to the “work-from-home” mode? How is the process organized?

You can find a heap of charts and shortlists on the best software testing outsourcing agencies. Compare them to see which companies stand out year after year. They will be the best candidates for the job. 

Have you heard about ‘shift-left’ testing? I bet you have. This popular paradigm suggests that we incorporate testing ’ab ovo’, at the very beginning of the project, implementing QA processes throughout the development cycle. Here are some of the key benefits of early and continuous testing:

1. QA engineers can work hand-in-hand with developers and make suggestions for improvements during the development in order to avoid surprises after the developers have finished their piece.
2. When cross-functional teams work together they can catch discrepancies and failures early.
3. Testers and developers quickly become one team, which increases productivity.

Developers also perform some forms of testing before releasing the code to their testers, i.e. testing basic functionality, reviewing the code, doing static code analysis; doing unit tests, etc.

Another approach is what’s called shift-right testing, wherein a big portion of testing is done while already in production (although, it doesn’t mean you don’t test along the way). In real life, we typically supply our customers with deliverables at a steady pace. Here is where the concept of a continuoustesting sometimes called continuous development, or infinite development, testing, and delivery. Iterations are straightforward, consisting of  ready-to-go chunks of your product. This is a more efficient process than waiting until the end to do the final test. After releasing one piece and receiving feedback, QA helps make improvements in the code, which expands the next iteration and makes testing methods more precise.

Ideally, the chosen QA experts and your developers start to collaborate in the early stages and work cooperatively “till death do them part”. Unfortunately, in many cases, this can create an infinite loop that ends up eating up your in-house testing team’s resources. However, there is a solution. 

Let’s talk about what happens if you employ a QA/testing team from a full-cycle software engineering company like Waverley. First, what is a full-cycle software engineering company? It’s a company capable of developing a product from end-to-end: from the idea validation and discovery stage (with in-house business analysts), through software architecture (with user experience designers and solutions architects), implementation (with mobile/web engineers, designers, DevOps and Data Science experts), all the way to testing (in-house QA teams), support and post-development maintenance (Level 2 & 3 support engineers).

Partnering with a large software testing outsourcing firm differs from working with a testing agency in many ways. Let’s explore the downsides first. Surprisingly, they are the same as for independent quality assurance companies.

• Security. The software development company has access to your product on a deeper level than an independent testing contractor would. This allows them to gain a better understanding of your product. If you work with an honest, mature company and sign a detailed non-disclosure agreement, your intellectual property will be fully protected.
• Control. If all aspects of your product development depend on one external company that has good knowledge of your product, security issues will inevitably arise. These can easily be mitigated if you are actively involved in the process as say, the acting CTO. Another option is to hire a CTO in-house to oversee the development & testing, if you are not technical enough yourself. You may want to combine a team from an outsource company and your in-house team to have a better grip on the process.
• Communication. This concern needs to be addressed by both the QA company and the software product development company. Different time zones and/or cultural peculiarities can hinder successful cooperation. Check whether the quality standards of a candidate firm correspond with your idea of quality. Make sure that it’s technically feasible to hold regular meetings.

Now let’s look at the upsides: 

• Large companies have considerable experience testing different software aspects (structural, functional, performance testing, accessibility testing, load testing, mobile testing, penetration testing, integration testing, security testing, automation testing) both in the automated and manual field. Testers in such companies communicate with developers on a daily basis and understand the development and testing processes more deeply. 
• They also have a larger variety of test equipment, techniques, and testing tools: the same set a big company would use for its own needs. They are also able to work on a  wider range of supported mobile platforms and fulfill both testing and development tasks.
• Development, DevOps, and Design Departments are within reach. They can be very helpful to bolster in-deep testing. Testers are able to get advice and help on extended debugging, UX/UI, making builds, etc.
• If a client decides to expand or enhance their product, they can use the whole scope of services, including sales and marketing departments.  
• With a bigger pool of QA engineers, there will be no need to recruit additional team members (which can take up to 2 months) in the midst of testing the product.
• Better control. Dedicated QA managers will work closely to address your specific needs, and focus on getting to the finish line through a smooth process because they are interested in you as a client in the long run.  
• Reputation and communication. With a big software company that has an established reputation in software development, you can expect similar standards with TaaS. And because they are accustomed to working around the globe, their communication strategies and routines have proven effective over time. This means they habitually communicate effectively so it’s a no-brainer to keep you in the loop on where your project stands and what you can expect from week to week. company will be better prepared for the unexpected. Testers are trained and ready to work on any issues that arise whether you need them onsite with your development team, working remotely, or even from home. A big software company is more likely to have an emergency reaction plan than a small or highly tailored firm.

In general, a big software company has more at-the-ready resources–both human and technical–to maintain the momentum on your project and carry it through to completion.

People aren’t perfect, but your software can be. Software errors can cost you your reputation, money, sometimes even human lives. I’d like to tell you a story about one superman who saved the world by doing literally nothing.  

On Sept. 26, 1983, Lt. Col. Stanislav Petrov was a duty officer at a bunker outside Moscow that monitored the Soviet Union’s early-warning satellite system. One of the satellites signaled that the United States had launched five ballistic missiles at the USSR. The tension between the two countries was so high at the time, Petrov could easily have believed that the signal was accurate. But he smelled a rat. First of all a mere five missiles were not enough to launch a real nuclear attack. Secondly, he’d been sensing that the Soviet’s early-warning system was “raw” so he reported a false alarm and took no further action. Upon further investigation, it was discovered that the satellite had picked up the sun’s reflection off the cloud tops and interpreted that as a missile launch.

So, you may want to consider the following when deciding to have TaaS implemented by a full-cycle software company.

• Before you start looking for TaaS, make sure your in-house Dev and QA teams have a clear vision of what they think is essential in order to get the best result including testing tools, time pressure, types of needed tests, or other extra services, etc. 
• Consider established software testing providers with experience in testing and development. Is it important for you to partner with a company that presents little risk and has great mobility?
• Look at your scope and budget and compare them with the capabilities of your potential software testing partner. Are you interested in web/mobile application testing, security testing, on-demand test automation?  And what about testing your data storage,  hardware, network services, or computational software testing?  
• Analyze if a provider can ramp up their human and tech resources to accommodate your needs. Do you think having a variety of specialists at hand will benefit your product? 
• Are the provider’s employees ready to travel to your location if necessary? How much interaction are you planning to have with a provider’s testing team? A big company tends to work quite closely with a client to make the process transparent.

Having reviewed all the pros and cons, I want to share our experience in TaaS. Waverley is a full-cycle software development company. We have a Development department, QA teams, Design groups, and other divisions. Because of the depth of our experience, we have powerful testing labs for desktop, web, mobile, hardware, cloud, and network applications, providing both manual and automation testing services. Clients’ data isn’t disclosed to third-party teams because we have enough resources to fulfill both testing and support tasks, locally or in a cloud. From the very beginning, Waverley professionals are trained to work remotely but within reach of our clients and each other. We frequently go on business trips to ensure we are on the same page with our clients. 

Close interaction between the teams guarantees that our QA engineers can seek help and get extra hands from other teams and departments. This affords us a deeper understanding of the internal implementation of a feature, which helps to build more detailed test cases and more accurate bug reports. Proximity to software engineers facilitates test automation if a tester lacks experience with ad hoc automation testing. It also becomes easier to engage developers to act as testers in case of white-box testing, which increases quality and lowers time-to-hire and the overall time it takes to complete the testing. In cloud services or application testing, it’s essential to involve DevOps engineers as well. It’s become a recent trend that DevOps engineers act as QAs in terms of continuous testing and monitoring.

In addition, our company has established an Emergency Response Plan which contains a list of practical actions on many issues (including but not limited to the current situation of lockdown and shelter-in-place modes). Our engineers are mobile and able to access the company’s resources remotely. This allows for a smooth transition in any unexpected situation and ensures deliverables will continue to be on time.

***
We wish you the best in choosing a software testing partner that can meet and exceed your expectations, fulfill all your business needs, will fit like a glove with your software development team so you can create a reliable, quality product your users will love.