Optimize AI for Speed and Robustness in the Lab

Optimization of the AI

For better AI operation, we need to have the best optimal hardware and software settings. These settings will be a determining factor to have the best possible data collection as input to train the AI.

Jetracer Optimization

Waveshare has a couple of sets of examples using JupyterLab with Python libraries to operate AI operations with the JetRacer. To experiment with these examples, clone this repository into your JetRacer: [1].

Gamepad(Teleoperation)

Here is the resource to use teleoperator on the jetracer [2]

• Observations:

The challenge here was to control the Jetracer with a gamepad through Matlab. This method was not possible because

1. The gamepad's adapter must be plugged into the Jetson Nano for real-time control.
2. Gamepad uses a Python library before it functions with the JetRacer properly.
3. Matlab cannot be installed on JetRacer [3]. Since it is an ARM processor (aarch64)[4].

Hence, it is difficult to control it directly from MATLAB.

• Alternative Approach

Instead of recreating the Python library used on JupyterLab with MATLAB from scratch, we create a Python library that will run all necessary libraries needed to operate the JetRacer, then run it with MATLAB in the background. This topic will be covered in the next chapter.

Camera

On the JetRacer, we have the IMX219-160 Camera [5], able to record at 60 fps at a resolution of 3280 x 2464.

• Observations:

We want to record and store the videos on the main PC and process them with MATLAB.

1. Video quality is slower than expected
2. The recording of the jetracer is send to main pc throught wifi which may impact the quality
3. Multiple functions are processed at once, e.g., steering, recording, and storing. Which reduces the camera output
4. Different resolutions and fps was tested and the optimal video resolution with the above mentioned limitations was 25 fps at 256 * 256

• Alternative Approach:

Check if there are any limitations applied to the camera sensor [6] or record directly on the Jetracer, then manually transfer the resulting data with corresponding time stamps to the main PC.

Steering and Speed

Steering is a valuable input to train an AI especially in autonomous driving. In this project, The default steering values were used and the speed was limited to an appropriate values aswell as seen below.

• Observations:

1. The steering values ranges from;

Steering: -1.0 (far left), Steering: -0.003936767578125 (straight) and Steering: 0.999969482421875 (far right).

1. Steering is a bit complex because it is difficult to be precise with the left analog stick (taking consistent turns on the track is very complicated)

• Alternative Approach

1. Using steering by implementation with discrete values. NB: This approach was used.
2. Use a more precise gamepad with PWM.

Video format

• Observations:

• Alternative Approach: