Across the wireless ecosystem, new product development teams for chipset vendors, device and infrastructure OEMs, and carriers face a host of challenges. Fueling the fire is the proliferation of multi-antenna (MIMO) designs, which intensifies the complexity when testing of today’s OFDM-based mobile devices.

To shorten development times, it’s essential to incorporate real-world test methodologies when comparing performance results from the beginning to the end of the complete design cycle for a wide variety of engineering teams and labs (including chipset development, handset development, network interoperability, and acceptance testing).

Pre-Deployment Testing With Field Conditions

The ability to accurately predict—in the lab—the performance a user will experience in the field offers a multitude of benefits

• Testing against field conditions early in the design cycle to discover field-related performance concerns as early as possible

• Benchmarking against the “golden routes” used by carriers to verify and validate device performance

• Sharing test conditions between partners to accelerate collaboration and reduce the time to troubleshoot or debug problems

• Regression testing of new hardware, firmware, or software revisions

• Performance benchmarking of possible firmware upgrades

Final pre-deployment testing often consists of device validation, performed in actual mobile-device usage field conditions. However, device field testing across diverse conditions of terrain, population density, physical location, and motion is extremely time-consuming and costly. It also lacks repeatability. 

The imperative to test early in the design cycle across as many challenging conditions as possible suggests that emulating the field environment and field testing early in the design process can save significant design time and resources. In all cases, tools that facilitate collaboration via a common platform and test environment can reduce test and development time and cost.

Virtual Field Testing in the Lab

When modeling the customer use case, the device in the test lab should be exposed to the same changing environment experienced by a customer in the field. A traditional laboratory device-validation methodology uses channel-emulation technology with industry-approved and standardized channel models.

Statistical representations of channel conditions are very useful in representing generalized RF environments needed for testing and certification. However, they don’t capture the unique and specific conditions experienced by a device as it physically moves through an actual wireless network.

This is addressed by virtual field testing in the lab, or “Field-to-Lab” testing, which replays real-world channel conditions (collected from drive testing) in a network testbed (Fig. 1). It thus effectively bridges the gap between laboratory and field-measured results.

By combining test methodologies including conducted test, in-lab replay of drive tests, and over-the-air performance testing, product-development teams can create a comprehensive environment in the lab that’s as realistic as possible. Subsequently, R&D and quality-assurance engineers can test products in the lab using both standard models and real-world conditions from actual field data collected from locations of interest. This drastically reduces the need for resource-intensive field testing. Such tests hold even greater value since they also can be performed across the full product-development cycle.

Field-to-lab testing integrates actual drive-test scan logs into the testbeds used in carrier and equipment vendors’ labs. They can more effectively test devices against field data collected over “golden drive test routes” or at key locations of interest in the actual network, where performance is critical.  

Field-to-lab testing typically involves three phases:  

• Collecting field data

• Conditioning the data for playback

• Playback of the field data on a channel emulator  

Accordingly, the key components of a field-to-lab methodology include a data source such as commercially available scanner or proprietary logging tool, a tool to map and filter scanner results to a subset suitable for playback on a channel emulator, and a software tool that will replay the filtered data on the channel emulator.

After commercial drive-test tools collect the device and network performance data, field-to-lab test methodologies can condition, save, and reuse it as many times as required with many different devices. To collect field data, field-to-lab solutions come with both standard and custom drive-testing software to allow the reuse of drive-test data. Because the tests are virtual, test labs can concatenate multiple drive-test logs and perform virtual field tests across multiple geographies without ever starting a vehicle.

Drive-test data, which is converted into a playback file for a given number of basestations, can be filtered, smoothed, and filled to ensure realistic playback conditions. Then the data is played back on the channel emulator.

A playback control tool configures the channel emulator and streams the playback conditions to the channel emulator’s signal path running between the mobile device and the basestation. This offers the ability to characterize expected performance, reproduce and verify field issues, benchmark interoperability with other devices, and optimize performance algorithms. Consequently, wireless devices are more thoroughly tested before ever being deployed in the field, and there’s considerably improved post-deployment troubleshooting and resolution.

Field-To-Lab Testing In Action

In the following case study, a leading handset manufacturer initially utilized field-to-lab testing to reduce software-development time, as well as improve time-to-market and product quality. After integrating the field-to-lab methodology into the handset manufacturers’ 3G test program, the company found that it could test a “crossing cell” scenario (as observed during drive test) and play it back in the lab to pre-qualify products against a particularly challenging field scenario. The “crossing cell” scenario describes the situation where the pilots from two cells cross over, typically with the power from one cell decreasing and the power from the other cell increasing.

Testing against a known scenario was a good start. However, the test team used a field-to-lab solution to further exaggerate that setting. Thus, the handset manufacturer could now test products against the “crossing cell” scenario as well as confirm that the design margin was sufficient to handle a range of field conditions. Software changes necessary for additional performance margin were implemented and then regression-tested against multiple scenarios using end-to-end testbed management capability.

While using field-to-lab testing, the handset manufacturer identified additional problems that could also be addressed by the field-to-lab methodology. In particular, the manufacturer was able to use field-to-lab to investigate a particular dropped call scenario in a carrier’s 3G network. 

The collected data showed a higher rate of dropped calls in one urban environment. The handset vendor postulated that the pilot from adjacent cells might appear long enough to trigger a handoff but would then disappear, ultimately resulting in a dropped call. This “rising pilot” was apparent in the drive-test logs (Fig. 2). However, prior to using field-to-lab testing, it couldn’t be easily replicated in the lab to perform more detailed analysis (Fig. 3).

After mapping a rising pilot captured in the drive-test logs, both the carrier and the handset vendor were able to use the same playback file to rapidly test their devices against the drive-test profile and identify the specific situations that led to a dropped call. Testers could then exaggerate the timing and magnitude of the rising pilot to establish the required device performance margins necessary to avoid a higher dropped call rate, delivering valuable feedback to network planners and chipset suppliers (Fig. 4).  

The improved ability to rapidly test many devices against multiple drive-test profiles saved both the handset manufacturer and carrier a significant amount of time and money. Furthermore, field-to-lab testing is performed in the privacy and security of a company’s own labs. No doubt, then, it provides the enhanced security needed when designing and developing groundbreaking new mobile devices versus drive testing.

The Value of “Common Ground”

As the wireless industry introduces advanced new products, field testing has, and will continue to be, particularly important. Field-to-lab testing accelerates performance testing, benchmarking, and troubleshooting of mobile devices against real-world conditions, while avoiding the substantial cost and time required for actual drive testing.  

However, the value of a field-to-lab test methodology goes beyond the accurate mapping of current drive-test profiles to the field. It also offers a common platform for sharing test environments (that capture challenging radio conditions) across the wireless ecosystem (Fig. 5). The sharing of playback files and test results up and down across the ecosystem achieves faster troubleshooting and lower development cost.

By establishing “common ground,” operators and device vendors can share and compare results generated from common drive-test scenarios using the exact same playback files, ensuring true apples-to-apples comparisons. Common ground ensures a rapid feedback loop and a powerful collaborative test environment to perform more accurate and efficient testing, enhance product quality, and substantially reduce time-to-market.

 

Caption: Erik Org, senior marketing manager at Azimuth Systems, brings more than 15 years of experience in improving corporate performance and value. His positions have been with both Fortune 500 companies as well as startups in the wireless communications field. At BitWave Semiconductor, he built a successful track record of driving sales, evangelizing new technology, and managing the overall marketing program. Prior to BitWave, he worked for TVM Capital, an early stage technology investor, and earlier, for Motorola and Qualcomm. He holds a BSEE from RPI and a MBA from Columbia Business School.