This is an old revision of the document!

Getting Started with Vitis AI (Under Construction)

Under Construction

Sources used in this guide are not currently public.

For now it only covers legacy DNNDK examples as the full Vitis AI library requires deploying the DPU as a Vitis acceleration kernel and is only compatible with Zynq UltraScale

Introduction

Vitis AI is Xilinx's development stack for implementing accelerated AI inference on their hardware platforms such as the Zynq 7000 and Zynq UltraScale. It consists of the DPU IP core, a software library for developing applications, software tools for training, quantization and deployment of AI models and a large set of prebuilt AI models and example projects.

This guide walks through the process of installing the development stack, building and deploying an accelerated application on a Digilent board.

Prerequisites

  • Ubuntu 18.04 LTS or another Linux distribution supported by Petalinux 2020.1
  • Petalinux 2020.1
  • Docker (Installation steps are located here: https://docs.docker.com/engine/install/)
  • 50+ GB of free hard drive space.

Guide

Downloading the support project and building the Petalinux SDK

In order to build the DNNDK library and the applications, the Petalinux SDK is required to be built and installed.

Clone the umbrella git repository for your chosen board and checkout the branch for the AI support project: 

git clone https://github.com/Digilent/Zybo-Z7.git
cd Zybo-Z7
git checkout 20/AI/master

NOTE: The given commands are for the Zybo Z7-20 AI project without Pcam 5C support.

Source the Petalinux Tools and build the SDK 

source <Petalinux_Install>/settings.sh
cd os
petalinux-build --sdk

Navigate to the images/linux folder and install the SDK 

cd images/linux
./sdk.sh -d <install_path>

Downloading Vitis-AI and building the DNNDK library

Clone the Vitis-AI repository and checkout version 1.2.1 

git clone https://github.com/Xilinx/Vitis-AI.git
cd Vitis-AI
git checkout v1.2.1

In order to configure the library to use the Linux driver instead of the XRT runtime, changes are required to be made to the Makefile.cfg and build the library

Navigate to the DNNDK source folder and open Makefile.cfg in a text editor. 

cd mpsoc/vitis_ai_dnndk_src/
gedit Makefile.cfg

Comment out line 11 by placing # at the begining of the line and uncomment line 12 by removing it, the file should look like the image and save the file.

Now enable the Petalinux SDK built earlier and build the library 

source <sdk path>/environment-setup-cortexa9t2hf-neon-xilinx-linux-gnueabi
OR
source <sdk path>/environment-setup-aarch64-xilinx-linux
make
make python # build the python support files
make package

You should have a new file called vitis-ai_v1.2_dnndk.tar.gz

Preparing the AI model for building the application

Download Xilinx's AI Model Zoo 

cd <Vitis-AI repo location>/AI-Model-Zoo/
sh get_model.sh

You should have a new folder called models

Build the Docker container for Vitis AI 

cd <Vitis-AI repo location>/docker
./docker_build_cpu.sh
-OR- 
./docker_build_gpu.sh #if you have NVIDIA CUDA support

Before entering the Docker container there are 2 files needed to be placed inside the Vitis-AI repository as there will be no easy access to files outside the container

One of them is a .hwh, usually called system.hwh, which contains information about the DPU and can be found in :

  • In the AI board project repository
  • In the Vivado project : <path_to_your_project>/<name_of_project>.srcs/sources_1/bd/<name_of_bd>/hw_handoff/<name_of_bd>.hwh

If the DPU IP settings have been modified then it is required to get the file from the Vivado project as it will invalidate the one in the repository.

The other one is called arch.json and will be created with the following content 

{
    "target"   : "DPUCZDX8G",
    "dcf"      : "./dpu-08-07-2020-10-30.dcf",
    "cpu_arch" : "arm32"
}

Replace arm32 with arm64 if you're using Zynq Ultrascale

Activate the docker container using the command: 

<Vitis-AI repository location>/docker_run.sh xilinx/vitis-ai-cpu:latest 
OR
<Vitis-AI repository location>/docker_run.sh xilinx/vitis-ai-gpu:latest

Accept the terms and conditions and you should be presented with a bash prompt from inside the container

Navigate to the model folder you need, in this case I need the model resnet50 for the demo with the same name, and copy the 2 files mentioned earlier: 

cd AI-Model-Zoo/models/all_models_1.2/caffe/cf_resnet50_imagenet_224_224_7.7G_1.2
cp /workspace/<your hwh file> ./
cp /workspace/arch.json ./

Now we need to generate the DPU metadata and compile the model for the application. If the model is made in Tensorflow(the model folder starts with tf_): 

conda activate vitis-ai-tensorflow
dlet -f <your hwh file>
#Open the arch.json file and make sure that the dcf parameter matches the newly created .dcf file
vai_c_tensorflow -f quantized/deploy_model.pb -a ./arch.json -n <name_of_the_model> -o output

If the model is made in Caffe(the model folder starts with cf_): 

conda activate vitis-ai-caffe
dlet -f <your hwh file>
#Open the arch.json file and make sure that the dcf parameter matches the newly created .dcf file
vai_c_caffe -p quantized/Edge/deploy.prototxt -c quantized/Edge/deploy.caffemodel -a ./arch.json -n <name_of_the_model> -o output

If it successfully compiled there should be a new folder called output and the .elf file is the compiled model. Now exit from Vitis-AI environment with 

exit

Building an DNNDK example

Now that the AI model is prepared to be added into an application, we need to install the DNNDK library inside the Petalinux SDK sysroot. Go to the DNNDK library sources, extract the created archive and run the installation script: 

cd <Vitis-AI repository path>/mpsoc/vitis_ai_dnndk_src/
tar -xvzf vitis-ai_v1.2_dnndk.tar.gz
cd vitis-ai_v1.2_dnndk
./install.sh <Petalinux SDK path>/sysroots/cortexa9t2hf-neon-xilinx-linux-gnueabi
-OR-
./install.sh <Petalinux SDK path>/sysroots/aarch64-xilinx-linux

To build the application there are two ways, using the command line or the Vitis development environment

Reactivate the Petalinux SDK in case you've closed the terminal window and navigate to the DNNDK sample you want to compile 

source <Path to SDK>/environment-setup-cortexa9t2hf-neon-xilinx-linux-gnueabi
OR
source <Path to SDK>/environment-setup-aarch64-xilinx-linux
cd <Vitis-AI repository path>/mpsoc/vitis_ai_dnndk_samples/<your chosen example>
C/C++ application

To compile the model we have to copy the compiled model and tweak the Makefile if you use a Zynq 7000 based board 

mkdir model
cp <Path to your model>/output/<name_of_model_elf> ./model/
gedit Makefile

If you need to modify the Makefile, change this line 

CFLAGS +=  -mcpu=cortex-a53

To: 

CFLAGS +=  -mcpu=cortex-a9

And save the changes

Now build the application: 

make

You should have a new file called as your application

Python application
To run Python based application you only need to copy the model and compile it as a shared library: 
cp <Path to your model>/output/<name_of_model_elf> ./
gcc -fPIC -shared your_model.elf -o libdpumodel<your_model>.so

Deploying the DNNDK example

Uploading the files to the board

1. Plug in the Ethernet cable from a router into the board, and start the board.

2. Start the serial console using a terminal program like TeraTerm or Putty

3. Get the ip address using “ip addr” or ifconfig

4. Upload the vitis-ai_v1.2_dnndk.tar.gz to the board: scp <path to Vitis-AI>/mpsoc/vitis_ai_dnndk_src/vitis-ai_v1.2_dnndk.tar.gz root@<board_ip>:~/

5. Upload your application executable or the python script and the .so file using scp <path to the file> root@<board_ip>:~/

6. Download the sample data provided by Xilinx vitis-ai_v1.2_dnndk_sample_img.tar.gz and upload the archive to the board

7. Connect to the board using ssh with X11 forwarding: ssh -X root@<board_ip>

Installing DNNDK and running the application

The following steps are done on the board in an ssh session with X11 forwarding.

1. Extract the DNNDK library and the sample data using tar -xvzf <name_of_archive>

2. Go to the DNNDK library folder and run the install script: 

cd vitis-ai_v1.2_dnndk
./install.sh

3. Create a new folder in the sample data and copy your application there 

mkdir ~/vitis-ai_v1.2_dnndk/test
cp <application files> ~/vitis-ai_v1.2_dnndk/test

4. Load the dpu driver using 

modprobe dpu

Then you run the application. The resnet50 demo should look like the image

References