Verilog RTL verification

Python, CocoTB infrastructure (and it’s testing) for an UART verification; the infrastructure testing is done without a need for a presence of an actual UART DUT design

A simple Python, CocoTB verification interface for UART verification, again independently tested using again Python/CocoTB with no full UART DUT design necessary

Exploring Python and CocoTB Infrastructure for Testing Without an Actual DUT Design

In this blog post, we will explore a simple Python and CocoTB infrastructure for UART verification. The aim is to test the functionality of the UART interface without the need for an actual UART Design Under Test (DUT) implementation. The Python code utilizes CocoTB, a Python-based framework for verification and testing of digital designs.

In one of the previous blog posts dedicated to CoCoTB and titled "CoCoTB based verification: A small example(https://asicstoic.blogspot.com/2023/06/cocotb-based-verification-small-example.html#comment-form) there is an introduction to CoCoTB (Coroutine-based Co-simulation Test Bench), an open-source Python library used for hardware design verification and testing. The post provides a small example of a CoCoTB testbench/test implemented using the icarus simulator and explains CoCoTB’s key concepts and features.In the last post there is also an explanation of the directory structure and Makefile to facilitate running the simulation . All of this is not going to be repeated in this blog post.

In this blog post files involved are:

A dummy DUT UART module (test_uart.v)

One minor inconvenience with CocoTB is the apparent requirement of having a simulator to run tests. However, in cases where we don't have any DUT/UART RTL Verilog code, it becomes necessary to create a straightforward DUT/UART Verilog module, for example with just a single data signal that we can easily connect to.

Python, CocoTB infrastructure and its testing (test_uart.py).

Here is a source code of the blog post files

test_uart.v (the DUT)

module test_uart
(
inout wire data
);

initial begin
$dumpfile("test_uart.vcd");
$dumpvars;
end

endmodule

test_uart.py

Let's start by examining the Python code that represents the UART verification infrastructure and the infrastructure testing.

import logging
import os
import warnings

import cocotb
from cocotb.regression import TestFactory

from collections import deque
from cocotb.triggers import FallingEdge, Event, Timer
import logging

##############
# UART Source

# this is just a standard python object so
# when you create it you'll specify the data signal that you want to connect to in the design
# and then we'll also specify the UART speed: 9600 bps

# let's go over how this works: you call function "write" with some data,
# then it puts it in the queue and it lets it sync.
# When you create this object what it does is: it calls set immediate value
# to set data to one because we want to UART to start with its data high.
# Then the _run coroutine has been forked so when you
# write data into the queue and it will set sync.
# When it becomes set it will pop the byte out of the queue and it will drive it out
# one bit at a time: starting out with a zero (start bit) and then eight data bits
# and finally one stop bit at one. At that point then it will loop around again if there is another byte
# then it will go ahead and run again, otherwise it will basically sit there and and
# wait until something ends up in the queue.
class UartSource:
def __init__(self, data, baud=9600):
# cocotb just uses standard python loggers which is kind of nice
# we're gonna take the data we're going to add path
# so that gives you a nice prefix for everything: for example
# if you have multiple instances you can tell exactly to what signals they're connected to
self.log = logging.getLogger(f"cocotb.{data._path}")
self.data = data
self.baud = baud
# now for the uart we're going to have to store some of the incoming and outgoing data somewhere
# so one way to do that, the highest performance way to do that is to use a queue
self.queue = deque()
self.sync = Event()

# we want the UART line to start high ( UART start bit is 0 )
# so I'll set immediate value to set the data to high when there is a load of the module
self.data.setimmediatevalue(1)
# this is what actually starts the coroutine _run on UART Source side
cocotb.fork(self._run())

# When you want to send something we'll use write function
# and then we're going to take that data and we're going to put it inside of the queue.
# Actually there will be two functions here, providing both a blocking and a
# non-blocking version and the blocking versions don't necessarily block but it is async.

# Here we can call "write" to put stuff in the buffer
# we can make this block if you want to specify you know maximum buffer size, we can implement that later
async def write(self, data):
self.write_nowait(data)

def write_nowait(self, data):
for b in data:
self.queue.append(b)
# now we need to tell the module that there's actually something in the queue so for that we use an event
self.sync.set()

# now we need the actual implementation of each one of these things written previously
# this is how to write coroutines in cocoTB
async def _run(self):
# this piece is going to be written kind of like a you know standard verilog test bench module
# where you're going to have (if you're making like a not a piece of non-synthesizable code) a
# loop in here that does something and then waits and then does something again ....
while True:
# the first thing we need to do is pull something out of the queue
while len(self.queue) == 0:

# while the queue is empty what we're going to do is: sync clear (we reset that) and then we're going to await it
self.sync.clear()
await self.sync.wait()

# when we actually have something in the queue then we're gonna go ahead and grab the byte
b = self.queue.popleft()

# to await whatever UART bit period is: so we're going to use a timer
# it is specified in nanoseconds
# 1 times 10 to the 9th nanoseconds per second and then you divide it by the baud rate
# and that gets us the number of nanoseconds per bit.
# Put that result to an int because timer gets annoyed when you give it the floating point number sometimes
t = Timer(int(1e9/self.baud), 'ns')

# This is how it works: we set the UART data line to zero for the start bit, then we wait one bit time.
# Further we loop over all the data bits, one at a time. We set that value shift by one and then
# wait. As a result we end up sending all the eight bits in, one at a time over the link.
# Finally we need to provide a stop bit.
# We're sending UART bits lsb first because that's kind of how UART works

# To repeat once again: we're going to take that byte and we're going to shift it out UART style;
# In general this is how uart work: first thing we do is to send a start bit (start bit is low actually)
# and that is why the UART line is initiated to start high
# UART start bit
self.data <= 0

# here we are going to await some time value. So how long are we going to await ?
# We're going to await whatever UART bit period is and for that purpose we're going to use a timer
await t

# Next we want to send the actual data bits
# another extension that you can make to this is the ability
# to change the number of bits per byte for example.
# Right now I'm not going to bother with that just for simplicity
# data bits
for k in range(8):
bit = b & 1
print(f"Write bit {k}: {bit}")

self.data <= bit
b >>= 1
await t

# UART stop bit
self.data <= 1

# Then I'm going to await some time value. So how long are we going to await ?
# We're going to await whatever UART bit period is right so we're going to use a timer
await t
########################################
# so far what we've defined here is basically the interface to the test bench
# at least a sending data to UART ( UART Source )

# Further it is described what the second part of test bench interface looks like, this time including receiving data from UART
# (UART Sink)

##############
# UART Sink

# As before this is also just a standard python object so
# when you create it you'll specify the data signal that you want to connect to in the design
# and then we'll also specify the UART speed: 9600 bps
class UartSink:
def __init__(self, data, baud=9600):
# Same conclusion as before: cocotb just uses standard python uh loggers which is kind of nice.
# Again we're gonna take the data and we're going to add path
# so that gives us a nice prefix for everything: for example
# in a case when there are multiple instances we can tell exactly to what signals they're connected to
self.log = logging.getLogger(f"cocotb.{data._path}")
self.data = data
self.baud = baud
# Again as before: for the uart we're going to have to store some of the incoming and outgoing data
# somewhere so one way to do that all right the highest performance way to do that is to use a queue
self.queue = deque()
self.sync = Event()

# this is what actually starts the coroutine _run on UART Sink side
cocotb.fork(self._run())

# instead of write and write_nowait we're going to have read

# The idea here is you can attempt to read a certain
# number of bytes out of the internal queue. We can't simply call read no weight in
# this case eventually we will do that we will return
# self.read no weight with same count parameter but before that
# we need to make sure that the queue is not empty.
async def read(self, count=-1):
# you can read which could block until data is available
while len(self.queue) == 0:
# await at the synchronization signal so I can do self.sink at clear
self.sync.clear()
await self.sync.wait()
return self.read_nowait(count)

# read_nowait which will attempt to read whatever is currently in the buffer
def read_nowait(self, count=-1):
if count < 0:
count = len(self.queue)
data = bytearray()
for _ in range(count):
data.append(self.queue.popleft())
return bytes(data)

# now we need the actual implementation of each one of these things written previously
# and in this case we're going to be trying to receive the the bits over the UART wire
async def _run(self):
while True:
# so what are we doing here, what we need to do is to wait for a start bit
await FallingEdge(self.data)

# This definition of t is exactly the same as for UART Source side
t = Timer(int(1e9/self.baud), 'ns')

# Once we have a start bit now we need to wait half of the bit time
# then we end up right in the middle of the bit
# start bit
await Timer(int(1e9/self.baud/2), 'ns')

# From this point on we can receive the 8 data bits
# data bits
# in b , we are going to "build" our byte. Here we are storing in "b" bit by bit what we are receiving from the UART.
b = 0
for k in range(8):

# At this point we're going to do await t: so we wait one bit time.
# When we started at the beginning, when we detected the edge of
# the start bit, we waited half a bit time so that gets us in the middle of the start bit.
# Next we wait a whole bit time so we arrived in the middle of the data bit
await t
bit = self.data.value.integer

# print bit value of k
print(f"Read bit {k}: {bit}")

# received bit is shifted by k and put in b
b |= bit << k

# put b in the queue
self.queue.append(b)
self.sync.set()

# In order to test UART Source and Sink we need for example
# a very basic test bench (TB)
# One thing that's slightly annoying about CocoTB:
# It seems to be impossible to run tests in CocoTB without a simulator.
# Since so far we didn't actually have any DUT/UART verilog code
# there is a need now to create a very simple DUT/UART verilog module
# with a single data signal in it that we can connect to
class TB:
def __init__(self, dut):
self.dut = dut

self.log = logging.getLogger("cocotb.tb")
self.log.setLevel(logging.DEBUG)

# Here are defined handles for our UART Source and Sink
self.source = UartSource(dut.data, baud=9600)
self.sink = UartSink(dut.data, baud=9600)

async def run_test(dut):
tb = TB(dut)

# Here make sure we get the same data on the other side
# so here is some test data defined , to send over UART
test_data = b'\xaa\xbb\xcc\xdd'
await tb.source.write(test_data)

# the next thing we need to do is actually to read everything out of the uart
# and make sure we get the correct result

# The problem with the read command is that it only waits for at least one data byte.
# Since it doesn't wait until you have everything we have to actually a while loop to wait until all test_data arrive in rx_data
rx_data = bytearray()

while len(rx_data) < len(test_data):
rx_data.extend(await tb.sink.read())

# when all of this is done then we will print all arrived UART data
personal_message = "User received data:"
print(personal_message)
print(rx_data)

# And finally the last thing to do is: make sure we get the correct result
assert rx_data == test_data, f"Output mismatch: Expected {rx_data}, got {test_data}"

if cocotb.SIM_NAME:
factory = TestFactory(run_test)
factory.generate_tests()

# cocotb-test
tests_dir = os.path.dirname(__file__)

def test_uart(request):
dut = "test_uart"
module = os.path.splitext(os.path.basename(__file__))[0]
toplevel = dut

veriog_sources = [
os.path.join(tests_dir, f"{dut}.v"),
]

parameters = {}

extra_env = {f'PARAM_{k}': str(v) for k, v in parameters.items()}

sim_build = os.path.join(tests_dir, "sim_build",
request.node.name.replace('[', '-'), replace(']', ''))

cocotb_test.simulator.run(
python_search=[tests_dir],
verilog_sources=verilog_sources,
toplevel=toplevel,
module=module,
parameters=parameters,
sim_build=sim_build,
extra_env=extra_env,
)

A waveform detail of the verification run

Simulation log file

rm -rf sim_build

make sim MODULE=test_uart TOPLEVEL=test_uart

make[1]: Entering directory '<user path>/cocotbext_uart'

rm -f results.xml

make -f Makefile results.xml

make[2]: Entering directory '<user path>/cocotbext_uart'

mkdir -p sim_build

/usr/bin/iverilog -o sim_build/sim.vvp -D COCOTB_SIM=1 -s test_uart -f sim_build/cmds.f -g2012 <user path>cocotbext_uart/test_uart.v

rm -f results.xml

MODULE=test_uart TESTCASE= TOPLEVEL=test_uart TOPLEVEL_LANG=verilog \

/usr/bin/vvp -M /home/tilic/.pyenv/versions/3.11.3/lib/python3.11/site-packages/cocotb/libs -m libcocotbvpi_icarus sim_build/sim.vvp

-.--ns INFO gpi ..mbed/gpi_embed.cpp:76 in set_program_name_in_venv Did not detect Python virtual environment. Using system-wide Python interpreter

-.--ns INFO gpi ../gpi/GpiCommon.cpp:101 in gpi_print_registered_impl VPI registered

0.00ns INFO cocotb Running on Icarus Verilog version 11.0 (stable)

0.00ns INFO cocotb Running tests with cocotb v1.8.0 from /home/tilic/.pyenv/versions/3.11.3/lib/python3.11/site-packages/cocotb

0.00ns INFO cocotb Seeding Python random module with 1688562410

0.00ns INFO cocotb.regression pytest not found, install it to enable better AssertionError messages

0.00ns INFO cocotb.regression Found test test_uart.run_test_001

0.00ns INFO cocotb.regression running run_test_001 (1/1)

Automatically generated test

<user path>/cocotbext_uart/test_uart.py:51: DeprecationWarning: cocotb.fork has been deprecated in favor of cocotb.start_soon and cocotb.start.

In most cases you can simply substitute cocotb.fork with cocotb.start_soon.

For more information about when you would want to use cocotb.start see the docs,

https://docs.cocotb.org/en/latest/coroutines.html#concurrent-execution

cocotb.fork(self._run())

<user path>/cocotbext_uart/test_uart.py:156: DeprecationWarning: cocotb.fork has been deprecated in favor of cocotb.start_soon and cocotb.start.

In most cases you can simply substitute cocotb.fork with cocotb.start_soon.

For more information about when you would want to use cocotb.start see the docs,

https://docs.cocotb.org/en/latest/coroutines.html#concurrent-execution

cocotb.fork(self._run())

<user path>/cocotbext_uart/test_uart.py:103: DeprecationWarning: Setting values on handles using the ``handle <= value`` syntax is deprecated. Instead use the ``handle.value = value`` syntax

self.data <= 0

<user path>/cocotbext_uart/test_uart.py:118: DeprecationWarning: Setting values on handles using the ``handle <= value`` syntax is deprecated. Instead use the ``handle.value = value`` syntax

self.data <= bit

<user path>/cocotbext_uart/test_uart.py:123: DeprecationWarning: Setting values on handles using the ``handle <= value`` syntax is deprecated. Instead use the ``handle.value = value`` syntax

self.data <= 1

Write bit 0: 0

Read bit 0: 0

Write bit 1: 1

Read bit 1: 1

Write bit 2: 0

Read bit 2: 0

Write bit 3: 1

Read bit 3: 1

Write bit 4: 0

Read bit 4: 0

Write bit 5: 1

Read bit 5: 1

Write bit 6: 0

Read bit 6: 0

Write bit 7: 1

Read bit 7: 1

Write bit 0: 1

Read bit 0: 1

Write bit 1: 1

Read bit 1: 1

Write bit 2: 0

Read bit 2: 0

Write bit 3: 1

Read bit 3: 1

Write bit 4: 1

Read bit 4: 1

Write bit 5: 1

Read bit 5: 1

Write bit 6: 0

Read bit 6: 0

Write bit 7: 1

Read bit 7: 1

Write bit 0: 0

Read bit 0: 0

Write bit 1: 0

Read bit 1: 0

Write bit 2: 1

Read bit 2: 1

Write bit 3: 1

Read bit 3: 1

Write bit 4: 0

Read bit 4: 0

Write bit 5: 0

Read bit 5: 0

Write bit 6: 1

Read bit 6: 1

Write bit 7: 1

Read bit 7: 1

Write bit 0: 1

Read bit 0: 1

Write bit 1: 0

Read bit 1: 0

Write bit 2: 1

Read bit 2: 1

Write bit 3: 1

Read bit 3: 1

Write bit 4: 1

Read bit 4: 1

Write bit 5: 0

Read bit 5: 0

Write bit 6: 1

Read bit 6: 1

Write bit 7: 1

Read bit 7: 1

User received data:

bytearray(b'\xaa\xbb\xcc\xdd')

4010391.00ns INFO cocotb.regression run_test_001 passed

4010391.00ns INFO cocotb.regression **************************************************************************************

** TEST STATUS SIM TIME (ns) REAL TIME (s) RATIO (ns/s) **

**************************************************************************************

** test_uart.run_test_001 PASS 4010391.00 0.01 453721010.36 **

**************************************************************************************

** TESTS=1 PASS=1 FAIL=0 SKIP=0 4010391.00 0.10 40232772.17 **

**************************************************************************************

ASIC Stoic

Total Pageviews

Labels

Wednesday, July 5, 2023

Exploring Python and CocoTB Infrastructure for Testing Without an Actual DUT Design

Here is a source code of the blog post files

test_uart.v (the DUT)

A waveform detail of the verification run

Simulation log file