Verilog RTL verification
CoCoTB based verification: A small example is getting bigger
CoCoTB: A Simple DUT and (almost) complete randomized, self-checking Testbench/Test with (some) coverage reporting
Introduction
In a last blog post dedicated to CoCoTB and titled "CoCoTB based verification: A small example(https://asicstoic.blogspot.com/2023/06/cocotb-based-verification-small-example.html#comment-form) there is an introduction to CoCoTB (Coroutine-based Co-simulation Test Bench), an open-source Python library used for hardware design verification and testing. The post provides a small example of a CoCoTB testbench/test implemented using the icarus simulator and explains CoCoTB’s key concepts and features.In the last post there is also an explanation of the directory structure and Makefile to facilitate running the simulation . All of this is not going to be repeated in this blog post.
In this blog post files involved are:
DUT module with handshake controls on both input data and output data side (SimpleDUT_hs.v),
wrapper module (SimpleDUT_hs_test.v) for collecting a VCD file, and
Python/CoCoTB testbench/test (SimpleDUT_hs_test.py).
Here is a source code of the blog post files
SimpleDUT_hs.v (The DUT)
As the DUT in the previous blog post, the new DUT (SimpleDUT_hs.v) module has a single data 4-bit input port (din_i) and a corresponding 4-bit output data port (dout_o). The data functionality of the DUT is straightforward, as dout_o is always equal to din_i.
However, what sets this version of the DUT apart is the addition of handshake control signals, which streamline the flow of data to and from the DUT.
To facilitate the transfer of input data, we have incorporated the en_din_i DUT input port and the rdy_din_o DUT output port. Similarly, for sending out the DUT result data, we employ the rdy_dout_i DUT input port and the en_dout_o DUT output port. These handshake control signals establish a seamless and coordinated data exchange process and the DUT has no internal data storage capability.
Despite the inclusion of these handshake control signals, the basic functionality of the DUT remains as before straightforward. To repeat once again The DUT output port, dout_o, always mirrors the value of the DUT input port, din_i.
The handshake protocol employed in this module is fairly simple. When a producer of data has a data ready for transmission, it assigns "en_*" to 1. On the other hand, when a consumer is prepared to receive the data, it responds by assigning "rdy_*" to 1.
In one pair of handshake control signals: "rdy_*" cannot transition to 1 unless its corresponding "en_*" is already 1. When a producer has data ready for transmission ("en_*" = 1) the data transmission is stalled until corresponding consumer becomes ready to accept the data ( "rdy_*" change from 0 to 1 and stay 1 one clock cycle ). The reason for this , as mentioned before, is that the DUT has no internal data storage capability.
The other assumptions about DUT:
It is fully synchronous design on it’s main ( and only ) system clock: DUT input port: CLK
Power On reset (DUT input port: RST_N) is active low
handshake control signals and in and out data bus values are also synchronous to DUT main system clock.
As in the previous blog post there is a “wrapper” for the DUT (SimpleDUT_hs_test.v) just to add a capability to collect VCD file (SimpleDUT.vcd) to later display waveforms of verification/simulation run:
A Python/CoCoTB testbench for the DUT verification integrated with a self-checking/randomized test ( with some coverage reporting capability): SimpleDUT_hs_test.py
SimpleDUT_hs_test.py functionality
The code begins by importing necessary modules and libraries for the implementation of a cocotb-based testbench. It includes modules for clock management, triggers, bus drivers, result handling, randomization, warnings, and coverage analysis.
Main test function: basic_count and SimpleDUTDriver
Lines 137-until the end of the file(line 170) present the basic_count test function, which includes the initialization, clock generation, reset control, stimulus generation, and output verification using the defined coroutines. Coverage reporting and exporting are performed at the end of the test.
Lines 47-52 define a bus driver class called SimpleDUTDriver responsible for driving signals to the DUT (Device Under Test).
coroutines for stimulus generation and output verification
The both coroutines (stimulus and check_output) are defined with the async def syntax, indicating that it can be paused and resumed asynchronously.
Also the both coroutines take the same four parameters each :
dut (representing the DUT instance),
stimulus_complete (an event object to signal stimulus completion),
stimulus_data (a deque object to store stimulus values), and
number_of_transfers (indicating the number of transfers to be performed)
number_of_transfers value is also randomized on line 162 in the range of 32 to 64: number_of_transfers = random.randint(32, 64))
asynchronous coroutine named stimulus
Lines 54-99 present an asynchronous coroutine named stimulus and here is what it does.
generates randomized stimulus values for two DUT input ports:
4 bit data din_i and
en_din_i part of corresponding handshake control for transfer input data ( din_i)
also it follows a handshake protocol by checking and ensuring the readiness of the rdy_din_o signal before driving inputs.
updates coverage points for various signals, and
logs the stimulus values.
updates coverage points(din_i and en_din_i ) and logs the stimulus values.
asynchronous coroutine named check_output
The check_output coroutine is responsible for verifying the correctness of the DUT's output by comparing it with the expected values stored in the FIFO queue. It also updates coverage points associated with the output signal and logs relevant output information for analysis and debugging purposes.
The verification process begins with a loop that iterates for the randomized number of transfers (defined by the number_of_transfers variable). This loop is responsible for checking each output value against the expected value stored in the FIFO queue (stimulus_data).
A waveform detail of the verification run
Simulation log file
rm -rf sim_build
make sim MODULE=SimpleDUT_hs_test TOPLEVEL=SimpleDUT_hs_test
make[1]: Entering directory '/< your path >/tests'
rm -f results.xml
make -f Makefile results.xml
make[2]: Entering directory '/< your path >/tests'
make[2]: Warning: File '/< your path >/tests/../hdl/SimpleDUT_hs.v' has modification time 9182 s in the future
mkdir -p sim_build
/usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s SimpleDUT_hs_test -f sim_build/cmds.f -g2012 /< your path >/tests/../hdl/or_gate.v /< your path >/tests/wrappers/or_test.v /< your path >/tests/wrappers/ifc_test.v /< your path >/tests/../hdl/ifc_or.v /< your path >/tests/../hdl/FIFO1.v /< your path >/tests/../hdl/FIFO2.v /< your path >/tests/../hdl/SimpleDUT.v /< your path >/tests/wrappers/SimpleDUT_test.v /< your path >/tests/../hdl/SimpleDUT_hs.v /< your path >/tests/wrappers/SimpleDUT_hs_test.v /< your path >/tests/../hdl/count_up.v /< your path >/tests/wrappers/count_up_test.v
rm -f results.xml
MODULE=SimpleDUT_hs_test TESTCASE= TOPLEVEL=SimpleDUT_hs_test TOPLEVEL_LANG=verilog \
/usr/bin/vvp -M /home/tilic/tomislav/cocoTB_tutorial/venv/lib/python3.11/site-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp
-.--ns INFO gpi ..mbed/gpi_embed.cpp:105 in set_program_name_in_venv Using Python virtual environment interpreter at /home/tilic/tomislav/cocoTB_tutorial/venv/bin/python
-.--ns INFO gpi ../gpi/GpiCommon.cpp:101 in gpi_print_registered_impl VPI registered
0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable)
0.00ns INFO cocotb Running tests with cocotb v1.8.0 from /home/tilic/tomislav/cocoTB_tutorial/venv/lib/python3.11/site-packages/cocotb
0.00ns INFO cocotb Seeding Python random module with 1687860737
0.00ns INFO cocotb.regression Found test SimpleDUT_hs_test.basic_count
0.00ns INFO cocotb.regression running basic_count (1/1)
3.00ns INFO cocotb.SimpleDUT_hs_test Running stimulus...
3.00ns INFO cocotb.SimpleDUT_hs_test Checking output...
7.00ns INFO cocotb Stimulus value: 7 added in fifo on position 1
19.00ns INFO cocotb Output expected value: 7 removed fifo on position 1
26.00ns INFO cocotb Stimulus value: 2 added in fifo on position 2
28.00ns INFO cocotb Output expected value: 2 removed fifo on position 2
35.00ns INFO cocotb Stimulus value: 12 added in fifo on position 3
47.00ns INFO cocotb Stimulus value: 9 added in fifo on position 4
51.00ns INFO cocotb Output expected value: 12 removed fifo on position 3
62.00ns INFO cocotb Stimulus value: 5 added in fifo on position 5
68.00ns INFO cocotb Output expected value: 9 removed fifo on position 4
71.00ns INFO cocotb Stimulus value: 9 added in fifo on position 6
77.00ns INFO cocotb Output expected value: 5 removed fifo on position 5
92.00ns INFO cocotb Stimulus value: 4 added in fifo on position 7
98.00ns INFO cocotb Output expected value: 9 removed fifo on position 6
102.00ns INFO cocotb Stimulus value: 8 added in fifo on position 8
118.00ns INFO cocotb Output expected value: 4 removed fifo on position 7
121.00ns INFO cocotb Output expected value: 8 removed fifo on position 8
135.00ns INFO cocotb Stimulus value: 0 added in fifo on position 9
138.00ns INFO cocotb Stimulus value: 1 added in fifo on position 10
144.00ns INFO cocotb Output expected value: 0 removed fifo on position 9
162.00ns INFO cocotb Stimulus value: 7 added in fifo on position 11
166.00ns INFO cocotb Output expected value: 1 removed fifo on position 10
171.00ns INFO cocotb Output expected value: 7 removed fifo on position 11
173.00ns INFO cocotb Stimulus value: 8 added in fifo on position 12
175.00ns INFO cocotb Stimulus value: 10 added in fifo on position 13
186.00ns INFO cocotb Output expected value: 8 removed fifo on position 12
193.00ns INFO cocotb Output expected value: 10 removed fifo on position 13
199.00ns INFO cocotb Stimulus value: 7 added in fifo on position 14
219.00ns INFO cocotb Stimulus value: 7 added in fifo on position 15
219.00ns INFO cocotb Output expected value: 7 removed fifo on position 14
232.00ns INFO cocotb Stimulus value: 11 added in fifo on position 16
237.00ns INFO cocotb Output expected value: 7 removed fifo on position 15
250.00ns INFO cocotb Stimulus value: 13 added in fifo on position 17
253.00ns INFO cocotb Output expected value: 11 removed fifo on position 16
263.00ns INFO cocotb Stimulus value: 0 added in fifo on position 18
274.00ns INFO cocotb Output expected value: 13 removed fifo on position 17
294.00ns INFO cocotb Stimulus value: 13 added in fifo on position 19
296.00ns INFO cocotb Output expected value: 0 removed fifo on position 18
304.00ns INFO cocotb Stimulus value: 9 added in fifo on position 20
311.00ns INFO cocotb Output expected value: 13 removed fifo on position 19
313.00ns INFO cocotb Stimulus value: 12 added in fifo on position 21
317.00ns INFO cocotb Output expected value: 9 removed fifo on position 20
322.00ns INFO cocotb Output expected value: 12 removed fifo on position 21
335.00ns INFO cocotb Stimulus value: 13 added in fifo on position 22
339.00ns INFO cocotb Output expected value: 13 removed fifo on position 22
355.00ns INFO cocotb Stimulus value: 9 added in fifo on position 23
369.00ns INFO cocotb Output expected value: 9 removed fifo on position 23
371.00ns INFO cocotb Stimulus value: 2 added in fifo on position 24
376.00ns INFO cocotb Stimulus value: 7 added in fifo on position 25
376.00ns INFO cocotb Output expected value: 2 removed fifo on position 24
387.00ns INFO cocotb Output expected value: 7 removed fifo on position 25
391.00ns INFO cocotb Stimulus value: 2 added in fifo on position 26
407.00ns INFO cocotb Stimulus value: 9 added in fifo on position 27
411.00ns INFO cocotb Output expected value: 2 removed fifo on position 26
413.00ns INFO cocotb Output expected value: 9 removed fifo on position 27
418.00ns INFO cocotb Stimulus value: 3 added in fifo on position 28
427.00ns INFO cocotb Stimulus value: 4 added in fifo on position 29
431.00ns INFO cocotb Output expected value: 3 removed fifo on position 28
433.00ns INFO cocotb Output expected value: 4 removed fifo on position 29
436.00ns INFO cocotb Stimulus value: 15 added in fifo on position 30
449.00ns INFO cocotb Output expected value: 15 removed fifo on position 30
454.00ns INFO cocotb Stimulus value: 13 added in fifo on position 31
464.00ns INFO cocotb Stimulus value: 7 added in fifo on position 32
470.00ns INFO cocotb Output expected value: 13 removed fifo on position 31
482.00ns INFO cocotb Output expected value: 7 removed fifo on position 32
489.00ns INFO cocotb Stimulus value: 1 added in fifo on position 33
494.00ns INFO cocotb Stimulus value: 9 added in fifo on position 34
498.00ns INFO cocotb Output expected value: 1 removed fifo on position 33
513.00ns INFO cocotb Output expected value: 9 removed fifo on position 34
515.00ns INFO cocotb Stimulus value: 8 added in fifo on position 35
518.00ns INFO cocotb Output expected value: 8 removed fifo on position 35
519.00ns INFO cocotb Stimulus value: 8 added in fifo on position 36
521.00ns INFO cocotb.SimpleDUT_hs_test Stimulus complete.
541.00ns INFO cocotb Output expected value: 8 removed fifo on position 36
543.00ns INFO cocotb.SimpleDUT_hs_test Output check passed.
543.00ns INFO cocotb top : <cocotb_coverage.coverage.CoverItem object at 0x7fcc4ce2a8d0>, coverage=32, size=36
543.00ns INFO cocotb top.din_i : ( ASICStoic comment: each of 16 values of DUT 4 bit input din_i is exercised in the test: there no BIN 0-15 = 0) <cocotb_coverage.coverage.CoverPoint object at 0x7fcc4cdfaed0>, coverage=14, size=16
543.00ns INFO cocotb BIN 0 : 2
543.00ns INFO cocotb BIN 1 : 2
543.00ns INFO cocotb BIN 2 : 3
543.00ns INFO cocotb BIN 3 : 1
543.00ns INFO cocotb BIN 4 : 2
543.00ns INFO cocotb BIN 5 : 1
543.00ns INFO cocotb BIN 6 : 1
543.00ns INFO cocotb BIN 7 : 6
543.00ns INFO cocotb BIN 8 : 4
543.00ns INFO cocotb BIN 9 : 6
543.00ns INFO cocotb BIN 10 : 1
543.00ns INFO cocotb BIN 11 : 1
543.00ns INFO cocotb BIN 12 : 2
543.00ns INFO cocotb BIN 13 : 4
543.00ns INFO cocotb BIN 14 : 1
543.00ns INFO cocotb BIN 15 : 1
543.00ns INFO cocotb top.dout_o : ( ASICStoic comment: each of 16 values of DUT 4 bit output dout_o is exercised in the test: there no BIN 0-15 = 0)<cocotb_coverage.coverage.CoverPoint object at 0x7fcc4cdfab10>, coverage=14, size=16
543.00ns INFO cocotb BIN 0 : 2
543.00ns INFO cocotb BIN 1 : 2
543.00ns INFO cocotb BIN 2 : 3
543.00ns INFO cocotb BIN 3 : 1
543.00ns INFO cocotb BIN 4 : 2
543.00ns INFO cocotb BIN 5 : 1
543.00ns INFO cocotb BIN 6 : 1
543.00ns INFO cocotb BIN 7 : 6
543.00ns INFO cocotb BIN 8 : 4
543.00ns INFO cocotb BIN 9 : 6
543.00ns INFO cocotb BIN 10 : 1
543.00ns INFO cocotb BIN 11 : 1
543.00ns INFO cocotb BIN 12 : 2
543.00ns INFO cocotb BIN 13 : 4
543.00ns INFO cocotb BIN 14 : 1
543.00ns INFO cocotb BIN 15 : 1
543.00ns INFO cocotb top.en_din_i : <cocotb_coverage.coverage.CoverPoint object at 0x7fcc4cdfa9d0>, coverage=2, size=2
543.00ns INFO cocotb BIN 0 : 36
543.00ns INFO cocotb BIN 1 : 1
543.00ns INFO cocotb top.rdy_din_o : <cocotb_coverage.coverage.CoverPoint object at 0x7fcc4e96e150>, coverage=2, size=2
543.00ns INFO cocotb BIN 0 : 36
543.00ns INFO cocotb BIN 1 : 36
543.00ns INFO cocotb.regression basic_count passed
543.00ns INFO cocotb.regression ***************************************************************************************
** TEST STATUS SIM TIME (ns) REAL TIME (s) RATIO (ns/s) **
***************************************************************************************
** SimpleDUT_hs_test.basic_count PASS 543.00 0.15 3516.28 **
***************************************************************************************
** TESTS=1 PASS=1 FAIL=0 SKIP=0 543.00 0.25 2138.51 **
***************************************************************************************
VCD info: dumpfile SimpleDUT_hs.vcd opened for output.