In the past tutorial, we have established how to integrate two pieces of code together, exemplifying a boot-loader and firmware interaction. The difference to the previous scenario is that in a test-suite, you actually need to maintain a symmetric interaction among those two different pieces of code. There are several approaches how this can be achieved.
- Explicitly pin each data-structure to specific locations in the memory. So each of the code pieces knows where to look for the others code and data.
- Pin an entry point of the firmware to a specific memory section that registers tests in another shared memory section.
- Pin an entry point of the firmware to a specific memory section that registers tests in a data-structure provided by the test-suite.
Looking at the different options it becomes apparent, why we talk about “grey-box testing”. In all cases we need to know some memory sections within the other pieces of code. The approaches differ by the number of memory sections required to be pinned. In addition, you might want to load tests dynamically through the boot loader requiring additional pinned sections.
Step One: How to Customize the Locations of Code and Data
In principle you want to place individual data structures and code into labelled memory sections, defining the location and label in the linker script and referencing label in GCC for the linker. The attribute-section paradigm allows us to do so. Suppose we want to load a function called test into a block of memory at an absolute address. First, we need to define this memory section in the linker script as follows:
MEMORY
{
ram : ORIGIN = 0x10200000, LENGTH = 1M
}
SECTIONS
{
.text :
{
*(.text)
*(.rodata*)
} > ram
.data :
{
*(.data)
} > ram
.bss :
{
*(.bss)
} > ram
__TESTS__ 0x10300000:
{
*(__TESTS__)
}
}
Second, we need to reference this section in the code. This is done by declaring an attribute in the function’s specification as follows:
void __attribute__ ((section ("__TEST_INIT__"))) init_tests() {
...
}The listing shows that the function is indeed stored at that particular location.
Sections:
Idx Name Size VMA LMA File off Algn
...
3 __TEST_INIT__ 00000030 10400000 10400000 00006000 2**1
...
Disassembly of section __TEST_INIT__:
10400000 :
...
Likewise global data structures and variables can be stored at such particular locations.
typedef struct {
void (*test1_fun)();
} TestFixture;
TestFixture __attribute__ ((section("__TESTS__"))) tests;
Step Two: Designing the Tests
We proposed three options for the test integration in the introduction.
Option One: (Naïve) Tell everything
The first naïve approach is to actually pin each testable primitive and global data structure to a particular memory region. In this scenario, all entry points to these primitives and global data structures are declared in the linker script of the test code. The linker script of the firmware declares all of those sections and the primitives are pinned to these sections using the attribute-section paradigm. This approach reduces the overhead of implementing tests vastly, since all locations are defined and no futher registration of the firmware with the tests is needed. Expected results can be directly checked against the data structures. However, maintaining the linker scripts in-sync, handling fragmentation of the firmware code (i.e., huge gaps between the declared sections) and changing the firmware code incur at significant overhead.
Option Two: Let the Firmware Register with a Global Data Structure
This approach is geared to minimize the overhead of maintaining the memory locations. In this scenario the firmware voluntarily registers with the test code, placing the information into a shared data-structure. In this scenario, two locations need to be shared across the firmware and the test-code.
- The location of the registration routine that is to be implemented by the firmware code.
- The location of the global data structure that contains the test information.
In addition the specification of the test data structure as well as the registration interface need to be shared among the two pieces of code. This can be achieved by sharing a common header file.
This scenario is useful when the test information is known beforehand. It also enables testing slightly modified versions of the firmware because the test code does not need to be aware of the location of the primitives or data structures. The firmware voluntarily provides this information through the registration routine. Both linker scripts define the section of the global data structure. Both linker scripts define the sections of the global data structure and registration routine. In order to avoid adverse effects the test code may prevent explicit writing to the registration routine. Here the test linker script:
MEMORY
{
sram : ORIGIN = 0x10200000, LENGTH = 1M
}
__FIRMWARE_ = 0x10100000;
/* firmware's test registration routine */
__REGISTER_TEST__ = 0x10300000;
SECTIONS
{
...
/* shared test data goes here */
__TEST_DATA__ 0x10400000:
{
*(__TEST_DATA__)
}
}
And the firmware linker script looks like this:
MEMORY
{
sram : ORIGIN = 0x10100000, LENGTH = 1M
}
SECTIONS
{
...
/* firmware's test registration routine */
__REGISTER_TEST__ 0x10300000:
{
*(__REGISTER_TEST__)
}
/* shared test data goes here */
__TEST_DATA__ 0x10400000:
{
*(__TEST_DATA__)
}
}
The shared header among the firmware and the test code defining the structure of the shared data structure and the registration interface:
typedef struct {
void (*firmware_fun)();
} TestFixture;
TestFixture __attribute__ ((section("__TEST_DATA__"))) gTestFixture;
extern void __REGISTER_TEST__();
The test code inside the bootloader simply registers with the firmware and invokes the required primitives of the firmware:
int main(void)
{
debug_puts("Inside boot-loader test suite!\r\n");
/* register test structure */
__REGISTER_TEST__();
/* execute a firmware function to test */
gTestFixture.firmware_fun();
debug_puts("Inside boot-loader again!\r\n");
return 0;
}
The location of the registration code inside the firmware is pinned as follows:
void __attribute__ ((section ("__REGISTER_TEST__"))) test_register() {
gTestFixture.firmware_fun = myprimitive;
}
Merging the test suite with the firmware and executing it in the coldfire simulator, as described in Part one of this tutorial, yields the following output. The test code can be obtained from option2.zip (see attachments below).
Use CTRL-C (SIGINT) to cause autovector interrupt 7 (return to monitor)
Loading memory modules...
Loading board configuration...
Opened [/usr/local/coldfire/share/coldfire/cjdesign-5307.board]
Board ID: CJDesign
CPU: 5307 (Motorola Coldfire 5307)
unimplemented instructions: CPUSHL PULSE WDDATA WDEBUG
69 instructions registered
building instruction cache... done.
Memory segments: dram timer0 timer1 uart0(on port 5206)
uart1(on port 5207) sim flash sram
!!! Remember to telnet to the above ports if you want to see any output!
Hard Reset...
Initializing monitor...
Enter 'help' for help.
dBug> dl merged.s19
Downloading S-Record...
Done downloading S-Record.
dBug> go 0x10200000
... telnet on uart0
Inside boot-loader test suite!
Inside firmware primitive!
Inside boot-loader again!
Option Three: Only Register with the Firmware
This option obviates the use of a shared data structure and may be used in the case where the test code has access to enough memory to allocate its own data structures. Usually, the constraints on boot loaders and such testers are relatively low that this is not an option. In this case we only have to share parts of the test data structure and the specification of the registration interface. The only difference to the previous example is that now the registration routine takes a pointer to the test structure provided by the test code. In addition only the prefix of the structure needs to be identical across the two pieces of code. For example the test code may choose to store test results in the structure that are hidden from the firmware. Let’s look at an example. As follows the specification of the test structure for the bootloader. Notice the removal of the pinned global variable and the additional value.
typedef struct {
void (*firmware_fun)();
int someTestingValue;
} TestFixture;
extern void __REGISTER_TEST__(TestFixture *tests);
And here the specification of the test structure for the firmware:
typedef struct {
void (*firmware_fun)();
} TestFixture;
This time the test definition structure is allocated by the bootloader and passed in as parameter to the firmware. The example code is included in option3.zip (see attachments below). The interaction with the simulator is identical to the previous example.
Step Three: Dynamic Tests
In many cases the space constraints for the test code are limited. So flashing an entire precompiled suite of tests may be impossible or undesirable. In order to overcome this issue you may want to consider dynamic tests. In this scenario only individual tests are uploaded through the boot loader and executed against the firmware. This approach can be combined with all of the above methods. In addition to the specified memory sections required by the test procedure (see Step two) you also need to define a section that holds the dynamic code. In the boot loader this section is referenced as array to store and replace the code and as function pointer to execute the test. The following example shows this with an already written array that is stored at the location of the test-code. The example code of the test is shown as follows:
/* dynamically loaded structure */
unsigned char __attribute__ ((section("__TEST_CODE__")))code [] = {
...
};
...
extern int __TEST_CODE__();
extern unsigned char * code;
...
int main(void)
{
debug_puts("Inside boot-loader test suite!\r\n");
/* perform test from loaded array */
__TEST_CODE__();
debug_puts("Inside boot-loader again!\r\n");
return 0;
}
To build the test code, you link it using a script that places the text, data and bss segment at the location of the test-code; or pin the test function explicitly to the section of the test code of the boot loader. The former option is useful, when the tests consist of several subroutine calls. The latter is useful for unit tests consisting of a single function call, having no global data structures. In addition, you might want to consider allocating a separate stack inside the test routine to avoid corruption.
The array containing the test cases can be created from the SREC-S19 file of the compiled test-case. The python script srec_to_c.py included in the sample code performs that conversion for continous S19 files.
Discussion
In this tutorial, we have shown how to leverage the boot-loader-firmware-paradigm introduced in the previous part of this tutorial to perform dynamic firmware testing. It is up to the software engineer to select the degree to which the firmware has to interact with the tests to register with the test-suite.
In addition to the procedures shown, you may want to consider using your host systems timers to check the progress of executed tests. If the firmware does not register with the tests properly or a test becomes stalled, the boot loader can be able to recover itself using a timer interrupt.
A substantial risk using these approaches is that the firmware and the bootloader still share the same address space. You may want to consider introducing explicit checks that ensure that the firmware does not touch boot loader code (i.e., through heap operations) and vice-versa.
The srec_to_c.py script performs the transformation of the test-case’s SREC/S19 files to the array. You can modify this script to create binary images that are uploaded through your devices interface.
Sample Code:
