Optimize AI for Speed and Robustness in the Lab

Optimization of the AI

For better AI operation, we need to have the best optimal hardware and software settings. These settings will be a determining factor to have the best possible data collection as input to train the AI.

Jetracer Optimization

Waveshare has a couple of sets of examples using JupyterLab with Python libraries to operate AI operations with the JetRacer. To experiment with these examples, clone this repository into your JetRacer: [1].

Gamepad(Teleoperation)

Here is the resource to use teleoperator on the jetracer [2]

• Observations:

The challenge here was to control the Jetracer with a gamepad through Matlab. This method was not possible because

1. The gamepad's adapter must be plugged into the Jetson Nano for real-time control.
2. Gamepad uses a Python library before it functions with the JetRacer properly.
3. Matlab cannot be installed on JetRacer [3]. Since it is an ARM processor (aarch64)[4].

Hence, it is difficult to control it directly from MATLAB.

• Alternative Approach

Instead of recreating the Python library used on JupyterLab with MATLAB from scratch, we create a Python library that will run all necessary libraries needed to operate the JetRacer, then run it with MATLAB in the background. This topic will be covered in the next chapter.

Camera

On the JetRacer, we have the IMX219-160 Camera [5], able to record at 60 fps at a resolution of 3280 x 2464.

• Observations:

We want to record and store the videos on the main PC and process them with MATLAB.

1. Video quality is slower than expected
2. The recording of the jetracer is sent to the main pc through wifi which may impact the quality
3. Multiple functions are processed at once, e.g., steering, recording, and storing. Which reduces the camera output
4. Different resolutions and fps were tested and the optimal video resolution with the above mentioned limitations was 25 fps at 256 * 256
5. 224*224 was used in the JupyterLab example [6].

• Alternative Approach:

Check if there are any limitations applied to the camera sensor [7] or record directly on the Jetracer, then manually transfer the resulting data with corresponding time stamps to the main PC.

Steering and Speed

Steering is a valuable input to train an AI, especially in autonomous driving. In this project, the default steering values were used, and the speed was limited to an appropriate value as well, as seen below.

• Observations:

1. The steering values range from

Steering: -1.0 (far left), Steering: -0.003936767578125 (straight) and Steering: 0.999969482421875 (far right).

1. Steering is a bit complex because it is difficult to be precise with the left analog stick (taking consistent turns on the track is very complicated)

• Alternative Approach

1. Manually increment steering with discrete values. NB: This approach was used.
2. Use a more precise gamepad with PWM.

Video format

Recording and saving the video in the best possible quality is very important because, it can be cropped, rectified and converted to other video types, e.g., grayscale, bird's-eye view, black and white (BW), etc.

• Observations:

1. MATLAB provides video writers like Uncompressed AVI, Motion JPEG 2000 and MPEG-4.
2. Uncompressed AVI is best suited since it provides no compression that may result in losses as compare to the other 2.