Debugging embedded SoC systems

Ever-advancing semiconductor technology has pulled together many system design elements that were once separate devices. Processors merged with peripherals. Next they absorbed memory. Eventually, analog components came on-chip.

Article Tools

Now, rising interest in wireless connectivity is starting to pull RF into the aptly named system-on-chip (SoC).

Digital meets analog

For the first time, RF designers working on embedded systems need to join their digital logic and software counterparts to program and debug these chips. Understanding the tools and techniques of SoC, debugging will help the collaboration run more smoothly and may suggest ideas for adding debug structures to the RF section.

When systems needed an entire printed circuit board (PCB), design teams had a wide range of tools available for debugging. Logic analyzers, in-circuit emulators and oscilloscopes provided insight into the operation of hardware and software design. To connect these tools to the system, designers used probes, connection to test posts, chip clips or chip sockets.

As silicon became more highly integrated, such test tools began to lose their applicability. This was especially true in the debugging of software as processors began to include peripherals and operating memory on-chip. Without the interface lines to connect with, tools could no longer provide insight into the processor's operation. In-circuit emulators began using shadow devices or special processors with internal signals brought out to in/out (I/O) pins to maintain access, but such measures became difficult to implement reliably. Too many processor variations were coming to market too quickly for the emulators to keep pace.

As single chips began to encompass more and more design elements, the problem got worse. Most of the system signals now remained entirely on-chip, where they were hidden from probes. Nor could they be brought out to I/O pins for access. Transistor density increased, but I/O pad density could not increase because pad size was set by the mechanical constraints of lead-frame attachment. So the ratio I/O pins-to-chip content continued to decline and the inner workings of these chips became increasingly inaccessible from the outside. Some form of built-in test that used as few I/O pins as possible was needed.

Scan techniques open SoC window

The answer to this dilemma rose from a surprising source: PCB designers. While chips were getting more comprehensive, packaging was shrinking and traces were becoming thinner. As a result, the probing of boards to test circuit continuity became increasingly difficult. An ad hoc group of companies, the Joint Test Action Group (JTAG), banded together in the 1980s to develop a solution to the problem. The answer JTAG came up with was the technique of boundary scan, and the JTAG specification was born.

Now known as IEEE Std. 1149.1, the JTAG specification describes the operation of an on-chip test circuit. The circuit uses a scan register, shown in Figure 1, that interrupts a signal line of interest. The register can read the data on the line and pass it on or insert its own test signal onto the line. Scan registers connect to form a chain that loops through the chip. This chain forms a serial shift register, allowing the JTAG controller to shift data and test signals into and out of the device one bit at a time. The entire test interface uses four I/O pins, yet can access all signal lines with scan registers.

The original intent of the JTAG plan was to include scan registers in a chip's I/O lines, so that signals leaving the chip could be set to a known state, and signals entering the chip could be captured and read. By comparing the test output of one chip with the input data of another, test engineers could determine if a printed circuit board's traces and solder joints had problems.

The JTAG specification allows the scan register to work with more than just I/O pins; it can be applied anywhere in the logic. Inserting a scan register into a signal line allows designers to push data into and read results from that signal line no matter where it is within the chip. The hidden interior of the SOC opens up.

The use of scan registers to look inside an SOC has value for the hardware design engineer as a test and monitoring tool. The scan chain is able, on command, to take a snapshot of the logic's operation. Accessing that information, however, requires shifting the data through the entire chain. Depending on the length of the chain, this could take several dozen to several thousand clock cycles. The application resource manager (ARM) processor, for example, has a scan chain 100 to 200 bits in length. Such prolonged serial access eliminates the possibility of performing any kind of real-time logic tracing, though single-step or reduced speed operation can be monitored.

Scan for software debug

The software design engineer can make great use of scan technology for debugging. Proper placement of a scan chain in the SoC design can give software designers external access to system memory as well as processor registers. With access to these resources, software designers can control and monitor the system software's operation. Given the increasing role of software in SoC designs, this access is essential to SoC debugging.

The JTAG scan technology is not the only form of on-chip software debug used in SoC designs. Other proprietary methods include background debug mode (BDM) and on-chip emulation (OnCE). Unlike JTAG, which provides general-purpose access to on-chip logic, these techniques focus on software needs. They use a command set rather than pure bit manipulation to manipulate the processor and memory. But like JTAG, they all use a serial interface for reading from and writing to the chip and can be used to perform essential debugging functions.

One essential debugging function is the ability to read from and write to system memory. This capability allows the designer to insert test code, patch system code or alter blocks of data that the processor will use in its operation. A related attribute, the ability to read and write processor registers provides a finer level of monitoring and control. A processor's registers include internal memory locations that hold instruction operands, the location of the next instruction to be executed and control parameters. Having direct access to registers, as well as system memory, gives developers control over everything from individual instructions to entire programs.

Access to memory is only part of the on-chip debug's essential capabilities. There also needs to be some form of processor control. The ability to run and halt processor execution and to set breakpoints in the code is fundamental to software debugging (see sidebar - Software Debug Basics), and scan-based testing provides them. These capabilities let developers execute blocks of code at the system's normal speed, then freeze everything and examine the results. In many cases, on-chip debugging circuits also allow single-step execution of code so that developers can watch execution line-by-line. A few go so far as to include the ability to trace program execution on a real-time basis.

A logistics nightmare

A significant challenge in using any of these on-chip debug schemes lies in handling the logistics of moving the data and instructions in and out of the SoC. Even the simple act of loading a register requires:

  • Shifting the data into the scan chain by the correct number of bits to reach the appropriate scan register.
  • Issuing a command to transfer the data into the scan register.
  • Issuing a second command to transfer the data into the processor's register.

For those systems with different scan chains for hardware test and software debugging, the additional step of selecting the appropriate scan chain adds to the effort.

Recovering data from the processor's register uses similar steps, but in the opposite order. The data come out serially, along with everything else in the scan chain. Interpreting the data requires decoding the serial stream based on each bit's position in the chain. This is difficult enough when the chain is long, but it can become even more complicated. For one thing, no two designs have the same internal chain; no standard exists for the order and content of scan chains. Further, many chip design teams leave the addition of the scan chain to automatic place and route tools. As a result, different revisions of the same design may have different scan chains. Yet loading and interpreting data using scan-based debug depends on having an accurate map of the chain's elements.

Fortunately, the industry has created software tools for use with scan-based debugging that manage and automate all this complexity. These tools reside on a host computer that connects to the target SoC through its JTAG, BDM or other test port. The host manages the test port, controls the scan clocking, and automatically maps test signals and data from bit position to logical function within the SoC, completely masking the tool's complex serial access method.

Presenting the data in a useful form is an important part of the host's task. Software, as seen at the processor level, is simply a pattern of binary values. For developers to understand the significance of the information, they need to relate the data back to the original source code. The host system makes this correlation and uses a graphical user interface (GUI) to present the information and provide tool controls to the designer. The result is much easier to use and frees the developer to use the tool to its maximum capability.

Many levels of capability and ease-of-use attributes are available among commercial scan-based debugging tools. The simplest tools provide basic control of the test port and will read and write data to the scan chain. More complete implementations include links to software development tools also running on the host. For example, Figure 2 shows the read-out of a processor's register contents, obtained from an SoC through a JTAG port. This display is identical to the one the designer sees in more conventional debugging setups.

Probe-enhanced, scan-based tools

Newer debugging tools use a probe between the host and target to speed the whole operation. The older host-only tools used the host's parallel port to read and drive the test lines. Clocking data into and out of the chain required the host to “wiggle” one output line to serve as a clock. This placed considerable overhead on the tool's operation and severely limited the speed at which the tool could access the port.

Probes serve to speed up the data stream. They typically offer high-speed transfers to the host, using Ethernet, USB or RS-232 links. On the target side, probes use specialized logic to control and clock the scan chain, speeding operation to the maximum the chip can support. Modern probes can drive the test port at clock speeds ranging from 10 kHz to 50 MHz. In comparison, the ARM processor's test port only runs at 10 MHz.

Probes also off-load port-intensive tasks from the host. For example, probes can be commanded to collect a sequence of system snapshots to form a program trace, and then deliver the accumulated results to the host. The probe can also automate repetitive tasks. Some probes, for example, can automatically run diagnostics on the SoC's built-in memory without host assistance.

The high rate of interaction that probes bring to scan-based debugging has made a major step in bringing software debugging for SoC designs up to the level enjoyed by board-level designs. Especially when integrated with software development tools, probes and scan-based debugging provide a full range of debug capability.

For example, scan testing can support source-level debugging. Rather than simply display data in the processor's native machine code, the host and probe can correlate processor instructions with the software in the high-level language the designers use.

Scanning debuggers

Scan-based debug can even handle designs with multiple processors on-chip, a configuration common in cellular telephony and wireless LAN SoCs. High-tech probes, combined with debuggers, can provide insight into and control over several on-chip processors simultaneously. As shown in Figure 3, the host tool can display each processor's data in its own window for easier interpretation. The tool can also coordinate the operation of multiple processors, halting them all simultaneously when one reaches a breakpoint in its code. It can also coordinate with other system events, allowing a trigger condition elsewhere in the system to halt the processors for examination of code status.

The added utility of a probe also means that software debugging can proceed with the system configured for normal operation rather than a special test condition. With debugging techniques such as software monitor programs, some system resources are used to hold the test routine. The techniques also force software to run out of RAM rather than system memory to be able to set breakpoints in the code. The use of scan-based testing and a probe allows debugging without using system resources. Further, probes can include routines for on-chip programming of system Flash memory holds the operational software, so patches and breakpoints can be set in the program memory rather than operating out of RAM.

Advantage: software + hardware

Probe-enhanced scan-based techniques have made most of the system software designer's traditional tools available for SoCs. Source-level debugging, multiprocessor capability, external triggering and GUI-based host software are all available using the scan-test window into the SoC. It's a narrow and limited window, but still a window. Without it, SoC debugging would be nearly impossible.

About the authors

Michael Johnson is a product marketing manager for Green Hills Software. He has six years of experience in software programming, four of those in embedded real-time design.

Neil Puthuff is a corporate field applications engineer for Green Hills Software. He has 19 years of circuit and software design experience and one U.S. patent.

Software Debug Basics

The software developer faces immense challenges in SoC design. Virtually every aspect of system behavior is software-controlled, run by programs thousands of instructions long and executing at a rate of millions of instructions per second. Further, these instructions are basic: fetch and store information, modify information using math or logic operations, compare two pieces of information and select what step in the program to execute next. Everything the developer wants to accomplish must be made from these few building blocks.

To make this task more manageable, developers often write their programs in a high-level language, such as C++, creating what is called source code. The high-level language allows developers to write programs using more powerful and complex instructions than the basic ones the processor understands. They then use a compiler to convert programs in this high-level language into machine language, the pattern of binary numbers that the processor uses. When developers need precision control, however, they may write in assembly language, a mnemonic representation of machine language. In either form, or in combination, the developer creates source code and uses development tools to convert it to object code, the binary number pattern that resides on the SoC as its program.

Unlike hardware, there is no practical way for the software developer to fully simulate the design. To debug this program, the developer must run it on the SoC and observe the results. The trouble is, it's running too fast to see anything as it happens. To make a program's behavior more observable, developers use a debugging tool.

Debugging tools come in many forms. The simplest monitor programs are pieces of code that let the developer use the system processor to watch and control its own program execution and provide basic access to system resources. A more hardware-intensive tool, the in-circuit emulator replaces the system processor with one that will execute code while providing the tool with program execution data. Whatever form the tool takes, however, it generally provides the ability to read and write system memory and processor registers, and to control processor execution through single-stepping and breakpoints.

Single-stepping, which is restricting the processor to execute only one program instruction before halting, allows developers to examine in detail the system's response to a program step. While useful for finding many types of errors, single-stepping has the drawback of slowing system operation to a crawl. Not only is this tedious for developers looking for one wrong instruction in a thousand, it can actually create problems. For many SoCs, correct operation means the system must respond to external signals within a limited time. Single-step program execution violates those “real-time” limits, and the resulting system failure can mask program errors.

Breakpoints are roadblocks the developer can place in the program to stop execution at a specific step or system event. The developer can then examine the system in detail immediately after a problem occurs, or just before a suspected problem area. This approach has the advantage of allowing long segments of code to execute at normal speed, but it can still violate real-time constraints.

Some debugging tools also have the ability to make a record of program execution, as well as some system parameters, for later examination. This record, or program trace, allows the developer to see events as they happened in real-time, without violating real-time constraints. It is, however, limited in the amount of system information it can gather. Program traces are essential for finding problems that result from missed timing, as well as problems that only show up when the system is running without interruptions.

The program information that any of these methods gather has the drawback of being in machine form rather than in the high-level language with which the developer designed the program. In high-level development tool suites, the debugging tool will correlate the gathered information with the original source code, allowing the developer to examine program execution in the original high-level language. The most capable tool suites combine editors, compilers, assemblers, debuggers and operating system software in an integrated package so that the tools share information and let the developer effortlessly move from one to another as needed.

Want to use this article? Click here for options!
© 2010 Penton Media Inc.


Acceptable Use Policy blog comments powered by Disqus


Most Popular Stories

Resources

Special Coverage

CTIA Wireless IT & Entertainment 2008

Read the latest from the show...