NPU Model Testing Tool: vpm_run

vpm_run is a VIPLite-driven tool for rapid validation of NPU model files on the Allwinner T527 board side.

• vpm_run serves as a common model running environment to run any NBG model file that is correctly deployed.
• vpm_run is based on the VIPLite network layer API.
• vpm_run has the default post-processing program TOP5.

---

Download the AI-SDK Sample Repository

X86 PC / Device

git clone https://github.com/ZIFENG278/ai-sdk.git

The Board Side Uses vpm_run

Enter the vpm_run Sample Code File Directory

Device

cd ai-sdk/examples/vpm_run

Compilation Example

Device

make AI_SDK_PLATFORM=t527
make install AI_SDK_PLATFORM=t527 INSTALL_PREFIX=./

Parameter analysis:

• AI_SDK_PLATFORM — specify SoC, options: a733, t527
• INSTALL_PREFIX — specify the installation path

How to Use

vpm_run -s sample.txt -l loop_run_count -d device_index
-s sample.txt:        to include one or more network binary graph (NBG) data file resources.
                      See sample.txt for details.
-l loop_run_count:    the number of loop run network.
-d device_index:      specify this NBG runs device.
-t time_out:          specify milliseconds time out of network.
-b bypass_level:      set value 1 to bypass saving output txt/binary file and showing top5.
--show_top5 level:    set value 1 to show top5 when bypass_level is 0.
--save_txt level:     set value 1 to save txt output when bypass_level is 0.
-c core_index:        specify this start core index of device.
--layer_profile_dump: set value 1 to enable NPD function.
--preload:            set value 1 to enable preload coeff into vipsram.
--op_segment:         set which operations will be run. example: --op_segment 10,20 means run 10 ~ 20
--layer_dump:         layer dump. eg: --layer_dump -1 dump all layer, --layer_dump 19, --layer_dump 18,20
-h:                   help
example: ./vpm_run -s sample.txt -l 10 -d 1  specify the NBG runs 10 times on device 1.

Running Example

Import Environment Variables

Device

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:your-path-to-sdk/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v1.13  # NPU_SW_VERSION
## examples
## export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/orangepi/work/npu/ai-sdk/viplite-tina/lib/aarch64-none-linux-gnu/v1.13

Tips: For the NPU version comparison table to determine NPU_SW_VERSION, T527 uses v1.13. Refer to the NPU version comparison table for more information.

Make a Profile

This example profile executes two NBG models: the first is a multi-input model that saves the output file and compares it to golden, and the second is a single-input model that does not save the output file.

[network]
./network_binary.nb
[input]
# Multi-input model: configure multiple input files; single-input model: configure 1 input file.
./input_0.dat
./input_1.dat
[golden]
# Optional
./int8_output_003.dat
[output]
# Optional
./output_0.dat

[network]
./network_binary_single.nb
[input]
./input1.tensor

The sample.txt file is shown above. The main tags are described as follows:

• [network] — the path to the NBG file.
• [input] — the path to the input data. Input data can be in two formats: the input.tensor file generated by the Acuity Toolkit inference phase, or the input.dat file generated during the IDE simulation phase.
• [golden] — path to optional golden data. If present, the golden data is binary-compared against the vpm_run output tensor: consistent results print pass, inconsistent results cause vpm_run to exit.
• [output] — optional. The save path of the output data.

Go to the Sample Installation Directory

Device

cd $INSTALL_PREFIX/etc/npu/vpm_run
./vpm_run -s sample.txt -l 1

(.venv) rock@radxa-cubie-a7a:~/ai-sdk/examples/vpm_run/etc/npu/vpm_run$ ./vpm_run -s sample.txt -l 1
loop_count=1, device_index=0, core_index=-1, file_name=sample.txt, time_out=0x0, bypass=1
enable_npd=0, preload=0
show_top50, save_txt=0
init vip lite, driver version=0x00020003...
VIPLite driver software version 2.0.3.2-AW-2024-08-30
vip lite init OK.

cid=0x1000003b, device_count=1
  device[0] core_count=1
config file read network count=1
init test resources, task_count: 1 ...
create/prepare networks ...
task i=0, binary name: ./network_binary.nb
nbg name=./network_binary.nb
create network 0: 12354 us.
input 0 dim 3 224 224 1, data_format=5, quant_format=1, name=input/output[0], dfp=13
output 0 dim 1000 1 0 0, data_format=1, name=uid_2_sub_uid_1_out_0, none-quant
memory pool size=1073536byte
network core count=1
prepare network 0: 1409 us.
golden file count=0
input 0 name: ./input_0.dat
read input and golden 0: 541 us.
task: 0, loop count: 1
start to run network=./network_binary.nb
run time for this network 0: 3160 us.
run network done...
profile inference time=2945us, cycle=2916338
destroy test resource task_count=1
vpm run ret=0