August 14, 2007

The QA Bottleneck  Comments 

Filed under: Engineering Tools — admin @ 4:11 pm

In new markets a bottleneck can develop in the Quality Assurance department.  This post discusses why this occurs and outlines potential solution areas.

The Cost of a Problem 

First it is important to look at how the costs to fix a problem depend significantly on in what phase the problem is discovered.  The conventional wisdom has been that the cost to fix a problem goes up ten times for each stage in of the development process from engineering to the customer. 


This makes intuitive sense.  In engineering a fix may involve a simple re-compile/re-test.  Only one functional group in the company has been affected.  In the best case only a single person is affected.


Once a product has been released to QA there has already been a tremendous investment made in integration, and unit-testing.  In QA there needs to be a test added to the regression to cover the newly discovered error and the previous testing re-run. 


Once a product that has been released to manufacturing the costs again go up dramatically.  The costs can include re-fabrication of a semiconductor device, reworking inventory, replacing inventory, and even re-tooling the manufacturing line in some cases.


Again the costs increase dramatically again once a product is released to customers.  The costs now are both increased expenses and lost revenue.  The expenses include support time handling multiple versions, recall costs, and replacement costs.  However, the impact to customer satisfaction, reputation, and market share may swamp the expenses.  All of these costs are magnified by the size of the installed base.


(Note that all the costs of previous stages will be incurred for issues that are discovered in a later stage.  For example, an error that is discovered by a customer will still have manufacturing costs, QA costs, and engineering costs.)


The Pressure on QA 

This cost structure already puts a lot of pressure on QA to avoid the costs of problems getting to manufacturing or to the customer.  In a new and growing market there are several factors that can combine to create a bottleneck in QA.


Among the factors that increase the load on QA are:  

1) The level of quality that is demanded rises. 

What was acceptable when a product was new and unique becomes unacceptable as more alternatives become available. 


2) The need to test with other products can increase dramatically

As a market grows, this interoperability testing can become a significant part of the QA effort.


3) The average user sophistication becomes lower. 

The early adopters may be experts that can work around certain defects and still get value.  As a market moves into the early majority this will no longer be the case.           


4) The new product may be finding new uses

As the product proliferates or as the price drops, the product may be penetrating new classes of customers.  Suddenly many new application scenarios need to be tested.

Potential Solution Areas 

Clearly from looking at the costs the solution has to come during the engineering or QA phases.  Releasing products prematurely just become ticking time bombs ready to explode at the wrong moment.  Actions such as outsourcing customer support are just stop gap measures that do not address the root cause.


There are many tools that help to automate the QA effort.  These can be invaluable.  They can both decrease the labor costs and increase the effectiveness of testing. 


The best solutions are the ones that can reach all the way to the engineering phase.  They typically involve changes to fundamental processes.   Depending on the kind of issues a company has it could include: getting customer feedback earlier in the process, more unit testing in engineering, eliminating fuzzy handoffs, and clearer release criteria.  As with most organizational changes they can be better implemented when there are tools that support and reinforce the new behaviors that are desired.


Rick Denker

Packet Plus™, Inc.

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a comment