August 23, 2007

Wireless Test Environments  Comments 

Filed under: Engineering Tools — admin @ 12:25 pm

It is important to understand the strengths and weaknesses of the range of wireless test environments.  Test environment refers to the setup or environment into which the device being tested and the test equipment are placed.  The four primary types of environments are: 

Faraday Cages

Faraday cages are usually large, hand-constructed, copper mesh wrapped boxes or rooms.  Because of the expense of their construction, they are typically found in the labs of large equipment manufacturers, where they are shared for testing and quality assurance.  Because Faraday cages assure a fairly noise- and interference-free environment, they are good for a wide variety of individual product tests, especially for antennas.  However, test configurations of more than a few devices can quickly congest traffic in a cage.  In addition, there may not be enough distance in the cage to test effects such as multi-path or diversity. 

Test Boxes, or RF Chambers

RF Chambers are metal boxes with absorbing material lining the inside to dampen interference.  They provide a controlled environment for much lower cost than a Faraday cage.  Typically, the DUT is placed into the test chamber, and probes are used to couple signals to/from the DUT through cables to an external test system.  In some cases the DUT and the test equipment are placed within the same test chamber, at which point this approximates a Faraday cage.  At some point, it ceases to be practical to use chambers as opposed to a Faraday cage.  Moreover, because spatial information is lost, some equipment cannot be tested in a chamber, e.g., smart antennas. 

Multiple sizes of chambers may be required for proper testing in a fully-enclosed environment.  The lower limit on the size of the chamber is dictated by the distance at which the RF near-field transitions to the RF far-field.  Objects – including the walls of the chamber itself – that are placed closer than this distance to the unit under test have a significant impact on the radiation pattern and efficiency of the unit; hence it is necessary to ensure that the chamber dimensions are greater than this distance, otherwise the test results may prove to be either irreproducible or erroneous. 

Cabled

Cabled tests simply substitute a wired connection for the wireless connection, bypassing the antennas and directly connecting two pieces of equipment.  As a result, cabled tests are inexpensive and easy to configure, and provide good isolation from interference.  They are not limited to small configurations, like cages and chambers.  However, because of the lack of interference, their results in configurations are idealized toward better performance than would actually occur in the randomness of an open air environment.  In addition, properly performing cabled testing relies on the DUT itself being well-shielded, which may not always be the case in consumer or low-end enterprise equipment.  In addition, equipment with integral antennas (where the antenna cannot be disconnected to gain access to a connector or other attachment point for a cable) cannot be tested using this method. 

Open air

Open air is the only test environment that truly matches the way the customer will use the equipment.  Like cabled environments, open air has no size limitations or limits on the number of pieces of equipment in a configuration.  For some tests, it is ideal because it can test both the antenna and the protocol effects.  Also it is the only solution for certain location-dependent tests. 

Open air test environments can be separated into indoor and outdoor.  Indoor environments are normally actual buildings, usually with furnishings and other accoutrements characteristic of typical office buildings.  Outdoor environments are usually open spaces without obstructions, such as would be found at an antenna range.  Of these two, the indoor environment is of the most interest, as it most closely approximates the conditions under which the equipment is expected to function.  Outdoor environments are used for applications such as characterizing antenna patterns, setting baselines for range and rate, etc.

Summary of test environments

Complete testing requires a combination of test environments; a one-size-fits-all environment does not exist for wireless testing.  Ideally, test equipment should be able to accommodate all environments.  Click on the thumbnail below to see a summary table.   

 test-environments-figure.gif

Rick Denker

Packet Plus™, Inc

August 14, 2007

The QA Bottleneck  Comments 

Filed under: Engineering Tools — admin @ 4:11 pm

In new markets a bottleneck can develop in the Quality Assurance department.  This post discusses why this occurs and outlines potential solution areas.

The Cost of a Problem 

First it is important to look at how the costs to fix a problem depend significantly on in what phase the problem is discovered.  The conventional wisdom has been that the cost to fix a problem goes up ten times for each stage in of the development process from engineering to the customer. 

 

This makes intuitive sense.  In engineering a fix may involve a simple re-compile/re-test.  Only one functional group in the company has been affected.  In the best case only a single person is affected.

 

Once a product has been released to QA there has already been a tremendous investment made in integration, and unit-testing.  In QA there needs to be a test added to the regression to cover the newly discovered error and the previous testing re-run. 

 

Once a product that has been released to manufacturing the costs again go up dramatically.  The costs can include re-fabrication of a semiconductor device, reworking inventory, replacing inventory, and even re-tooling the manufacturing line in some cases.

 

Again the costs increase dramatically again once a product is released to customers.  The costs now are both increased expenses and lost revenue.  The expenses include support time handling multiple versions, recall costs, and replacement costs.  However, the impact to customer satisfaction, reputation, and market share may swamp the expenses.  All of these costs are magnified by the size of the installed base.

 

(Note that all the costs of previous stages will be incurred for issues that are discovered in a later stage.  For example, an error that is discovered by a customer will still have manufacturing costs, QA costs, and engineering costs.)

cost-of-change.JPG 

The Pressure on QA 

This cost structure already puts a lot of pressure on QA to avoid the costs of problems getting to manufacturing or to the customer.  In a new and growing market there are several factors that can combine to create a bottleneck in QA.

 

Among the factors that increase the load on QA are:  

1) The level of quality that is demanded rises. 

What was acceptable when a product was new and unique becomes unacceptable as more alternatives become available. 

 

2) The need to test with other products can increase dramatically

As a market grows, this interoperability testing can become a significant part of the QA effort.

 

3) The average user sophistication becomes lower. 

The early adopters may be experts that can work around certain defects and still get value.  As a market moves into the early majority this will no longer be the case.           

 

4) The new product may be finding new uses

As the product proliferates or as the price drops, the product may be penetrating new classes of customers.  Suddenly many new application scenarios need to be tested.

Potential Solution Areas 

Clearly from looking at the costs the solution has to come during the engineering or QA phases.  Releasing products prematurely just become ticking time bombs ready to explode at the wrong moment.  Actions such as outsourcing customer support are just stop gap measures that do not address the root cause.

           

There are many tools that help to automate the QA effort.  These can be invaluable.  They can both decrease the labor costs and increase the effectiveness of testing. 

 

The best solutions are the ones that can reach all the way to the engineering phase.  They typically involve changes to fundamental processes.   Depending on the kind of issues a company has it could include: getting customer feedback earlier in the process, more unit testing in engineering, eliminating fuzzy handoffs, and clearer release criteria.  As with most organizational changes they can be better implemented when there are tools that support and reinforce the new behaviors that are desired.

 

Rick Denker

Packet Plus™, Inc.