SW Test Methods in VHDL
When testing SW there are two methods that are primarily used to handle dependencies that are not needed for a test: stubbing and mocking. For embedded SW the dependencies are usually on HW or the MCU’s Hardware Abstraction Layer (HAL). For application SW the dependencies are usually one of these: the file system, the internet, or other applications. In short, these dependencies are usually removed and their interface is fulfilled by another, more effective and simpler mechanism. In VHDL the HW interface is nicely abstracted through ports and testbenches without ports are standard industry practice for simulating the effects of HW stimulus on a design. When dealing with the VHDL counterpart of a file system; memory access, mocking may speed up simulations by providing a memory model that is effective for simulations. Finally, VHDL also interfaces with other applications: processing units that either run SW (softcore-CPUs) or are designed to infer information about their inputs in a hard-to-predict manner; AI accelerators or Tensor Processing Units (TPUs). In this post I’ll mention stubbing briefly and then cover how to mock a processing unit in VHDL.
Table of Contents
Stubbing Architectures
Stubbing is when you remove the internals of a dependency, leaving it empty or making it’s functionality static (e.g. methods returning constants).
Providing an empty architecture for an entity when simulating an unrelated part of a top-level design may decrease simulation time and setup effort. This also provides a way to force values onto signals which are left dangling once the internals of an architecture are removed.
I’ll not go into details on this method since it is trivial to provide an empty architecture, however the same principle applies to replacing the architecture as for the mocked architectures below.
Mocking Architectures with Shared Variables
Mocking is when you remove the internals of a dependency and replace it with something simpler, which you have more control over. Mocks are dynamic, as opposed to the static nature of stubs.
The access_type_pkg is a generic package which takes record definitions for the in-ports and the out-ports of a block as generic parameters.
I’ve named the methods of the access_t type to coincide with the expected usage in testbenches, the get_outputs and set_inputs are intended to serve as the mechanism internal to the mock, while get and set are meant to be used in a testbench that makes use of the mock.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
package access_type_pkg is
generic(type T_in; type T_out);
type access_t is protected
procedure set(val: T_out);
impure function get return T_in;
-- NOTE: For internal use only,
-- i.e. don't use these in a testbench.
impure function get_outputs return T_out;
procedure set_inputs(val: T_in);
end protected;
end package;
package body access_type_pkg is
type access_t is protected body
variable outputs: T_out;
variable inputs: T_in;
procedure set(val: T_out) is
begin
outputs := val;
end procedure;
impure function get return T_in is
begin
return inputs;
end function;
procedure set_inputs(val: T_in) is
begin
inputs := val;
end procedure;
impure function get_outputs return T_out is
begin
return outputs;
end function;
end protected body;
end package body;
Two caveats: You’d probably want to add some synchronization for setting the internal variables if using this approach in multiple processes, and it should be possible to implement the same pattern without using VHDL-2008 features such as generic packages and protected types.
Now we just need to instantiate the generic package, instantiate a shared variable, connect the shared variable to ports inside a mock architecture of the CPU, instrument the CPU through the shared variable in a testbench, and finally make sure that our mocked architecture gets used and not the real architecture. It’s best to show these steps in an example, which is provided hereafter.
Example
Let’s assume we have a design that looks like the one shown in Listing 2 below. The design instantiates a CPU and in the hw_if block it maps GPIOs, that SW will have control over, to pins on the chip (FPGA or ASIC). It is this hw_if block that we are really interested in testing here, and since it has a dependency on the CPU’s ports, mocking the CPU frees us from running SW on the CPU in simulations when the only implementation that interests us is outside the CPU.
Note that the CPU is instantiated by using a component declaration, this enables us to swap it out with a configuration in the test bench. However, if your design instantiates a block that you want to mock with an entity instantiation then you’ll have to make sure to only compile the mock architecture and not the real architecture, if the entity and real architecture are defined in the same file then you’ll have to redefine the entity too.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
library ieee;
use ieee.std_logic_1164.all;
use work.cpu_types_pkg.all;
entity example_dut is
port(
rst_n: in std_logic;
clk: in std_logic;
some_input: in std_logic;
some_output: out std_logic
);
end entity;
architecture rtl of example_dut is
component riscv_soc_wrapper is
port(
inputs: in cpu_in_ports_t;
outputs: out cpu_out_ports_t
);
end component;
signal inputs : cpu_in_ports_t := CPU_IN_PORTS_INIT;
signal outputs : cpu_out_ports_t := CPU_OUT_PORTS_INIT;
begin
hw_if: block
begin
inputs.i_rst <= not rst_n;
inputs.i_clk <= clk;
inputs.i_gpio(0) <= some_input when outputs.o_gpio_dir(0) = '0' else 'X';
some_output <= outputs.o_gpio(1) when outputs.o_gpio_dir(1) = '1' else 'Z';
end block;
cpu: riscv_soc_wrapper
port map(
inputs => inputs,
outputs => outputs
);
end architecture;
The CPU itself is not important for this discussion, but here is the empty wrapper. In a real implementation a CPU would be instantiated in the wrapper’s architecture and ports mapped as appropriate.
1
2
3
4
5
6
7
8
9
10
11
12
13
use work.cpu_types_pkg.all;
entity riscv_soc_wrapper is
port(
inputs: in cpu_in_ports_t;
outputs: out cpu_out_ports_t
);
end entity;
architecture rtl of riscv_soc_wrapper is
begin
-- Real imlementation, not important for this demonstration.
end architecture;
The ports of the wrapper are two separate records, one for input and another for output, these will be used when instantiating a specific instance of the generic access_type_pkg.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
-- Loosely based on riscv soc:
-- https://github.com/sergeykhbr/riscv_vhdl/blob/master/rtl/work/riscv_soc.vhd
library ieee;
use ieee.std_logic_1164.all;
package cpu_types_pkg is
type cpu_in_ports_t is record
i_rst :std_logic;
i_clk :std_logic;
i_gpio :std_logic_vector(11 downto 0);
end record;
type cpu_out_ports_t is record
o_gpio :std_logic_vector(11 downto 0);
o_gpio_dir :std_logic_vector(11 downto 0); -- '1' for output, '0' for input
end record;
constant CPU_OUT_PORTS_INIT: cpu_out_ports_t := (
o_gpio => (others => '0'),
o_gpio_dir => (others => '0')
);
constant CPU_IN_PORTS_INIT: cpu_in_ports_t := (
i_rst => 'Z',
i_clk => 'Z',
i_gpio => (others => 'Z')
);
end package;
Now let’s take a look at a testbench and see how we want to instrument the CPU to check the HW Interface logic. Here we instantiate the design under test with a component instantiation, to allow for configuration at the bottom of the testbench. Then we have a mysterious object, cpu_access, which acts as the SW that will eventually run on the CPU. Checks of values can be performed as usual, here with VUnit’s check library.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
library ieee;
use ieee.std_logic_1164.all;
library vunit_lib;
context vunit_lib.vunit_context;
library osvvm;
context osvvm.OsvvmContext;
use work.cpu_types_pkg.all;
use work.mock_cpu_pkg.cpu_access;
entity example_tb is
generic (runner_cfg : string);
end entity;
architecture test of example_tb is
component example_dut is
port(
rst_n: in std_logic;
clk: in std_logic;
some_input: in std_logic;
some_output: out std_logic
);
end component;
signal rst_n: std_logic := '1';
signal clk: std_logic := '0';
signal some_input: std_logic := '0';
signal some_output: std_logic := 'Z';
constant CLOCK_PERIOD : time := 1 sec/100e6;
begin
CreateClock(clk, CLOCK_PERIOD);
CreateReset(
Reset=>rst_n,
ResetActive=>'0', -- active low
Clk=>clk,
Period=>2*CLOCK_PERIOD,
tpd=>CLOCK_PERIOD
);
dut_instance : example_dut
port map(
rst_n => rst_n,
clk => clk,
some_input => some_input,
some_output => some_output
);
main : process is
variable inputs: cpu_in_ports_t;
variable outputs: cpu_out_ports_t := CPU_OUT_PORTS_INIT;
begin
test_runner_setup(runner, runner_cfg);
while test_suite loop
WaitForToggle(rst_n); -- high to low
WaitForToggle(rst_n); -- low to high
if run("check_inputs") then
inputs := cpu_access.get;
check_equal(inputs.i_rst, '0', "Reset received");
elsif run("apply_outputs") then
outputs.o_gpio(1) := '1';
cpu_access.set(outputs);
WaitForClock(clk, 1);
check_equal(some_output, 'Z');
outputs.o_gpio_dir(1) := '1';
cpu_access.set(outputs);
WaitForClock(clk, 1);
check_equal(some_output, '1');
end if;
end loop;
test_runner_cleanup(runner);
end process;
end architecture;
configuration example_tb_mock_cpu of example_tb is
for test -- arch
for dut_instance: example_dut
use entity work.example_dut(rtl);
for rtl -- arch
for cpu: riscv_soc_wrapper
use entity work.riscv_soc_wrapper(mock);
end for;
end for;
end for;
end for;
end configuration;
The generic package needs to be instantiated and when doing so we need to provide the appropriate input and output record types. Then the shared variable can be declared based on the specific package, mock_cpu_type_pkg in this case.
1
2
3
4
5
6
7
8
use work.cpu_types_pkg.all;
package mock_cpu_type_pkg is new work.access_type_pkg
generic map (T_in => cpu_in_ports_t, T_out => cpu_out_ports_t);
package mock_cpu_pkg is
shared variable cpu_access: work.mock_cpu_type_pkg.access_t;
end package;
Finally, we hook into the shared variable in a mock architecture of the CPU wrapper entity, using the internal get and set methods to update the global state.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
library ieee;
use ieee.std_logic_1164.all;
use work.cpu_types_pkg.CPU_OUT_PORTS_INIT;
use work.mock_cpu_pkg.cpu_access;
architecture mock of riscv_soc_wrapper is
begin
main: process(inputs.i_clk) is
begin
if rising_edge(inputs.i_clk) then
if inputs.i_rst = '1' then
cpu_access.set(CPU_OUT_PORTS_INIT);
else
cpu_access.set_inputs(inputs);
outputs <= cpu_access.get_outputs;
end if;
end if;
end process;
end architecture;
Here is the VUnit script for completedness, notice that there is no need to have the CPU entity declared in a separate file from it’s architecture and removing the CPU architecture from the source list when compiling for these tests. This is the approach that would be needed if the CPU and dut instances had not been instantiated as components.
1
2
3
4
5
6
7
8
#!/usr/bin/env python3
from vunit import VUnit
vu = VUnit.from_argv()
vu.add_osvvm()
lib = vu.add_library("lib")
lib.add_source_files("*.vhd")
vu.main()
Conclusion
By mocking CPUs for simulation you get full control over the CPUs inputs and outputs, which provides a way to simulate SW behaviour towards the SW/FW interface. The advantage of this approach as opposed to stubbing and forcing signals is that you can hook up Verification Components from within the CPU and keep complex mimicking of SW behaviour contained within the mock architecture. This mimicking of SW behaviour might include state machines or main-loops with execution times that are much slower than the clock-cycle.
Comments? You are welcome to start a discussion on Github.