Controlled Autonomous Driving for a JetRacer

Vid.1 JetRacer ROS AI Driving at 1.44 m/s

Abb. 1: JetRacer ROS AI Roboter von Waveshare

Autor:	Yashodhan Vishvesh Deshpande
Art:	bachelor thesis
Starttermin:	20.01.2026
Abgabetermin:	19.05.2026
Betreuer:	Prof. Dr.-Ing. Schneider

Task List

1. Research of similar projects
2. Evaluation of the solutions using a morphological box (Zwicky box)
3. Choose a software toolchain
4. Design and print a platform for the TopCon-prism.
5. Record the ground truth data
6. Track the JetRacer on a digital map.
7. Control the JetRacer in the right lane without obstacles by camera data.
8. Evaluate the results based on technical features
9. Discussion of the results
10. Testing of the system requirements - proof of functionality
11. Scientific documentation as a report
12. Scientific advertising as a wiki article with a result video.

Thematic Introduction

The JetRacer:_ROS_AI_Kit can be programmed in different ways. The mechatronic student learn to develop C-code, m-code oder model based development with simulink. The task is to develop a "easy to use" framework to get quick results. After choosing the toolchain based on a morphological box an attachment for the reference prism (Referenzmessung_mit_der_Topcon_Robotic_Total_Station) has to be build. By this the ground-truth-data (vehicle pose) can be recorded for further analyzing. The main task is to develop a program that acquires an image frame, transforms it into bird-eye-view to measure the lane's curvature. Based on the lane curvature and actual driving status the car should steer into the right lane. Optional a gyroscope can measure the yaw-rate of the JetRacer.

The goal is to drive in the right lane as fast as possible.

Optional an obstacle detection an avoidance by LiDAR can be added. Future students can compare an AI lanekeeping to this classic image processing approach.

The Bachelor Thesis was carried out in the Autonomous Systems Laboratory at Hochschule Hamm-Lippstadt. The Bachelor Thesis was carried out under the supervision of Prof. Dr. Ulrich Schneider and Prof. Marc Ebmeyer. The Laboratory is designed for Mechatronic and Electronic Engineering Students to realize there Academic Projects.

Fig.2 Working Timelines for Thesis

Bachelor Thesis Plan

The image shown in Fig.2 is the thesis timeline that is illustrated using a Gantt Chart.

Theoretical Background

• Hardware Requirements

The Jetracer ROS Ai kit is used for this thesis as a primary autonomous vehicle. All its system that are integrated in the Jetracer were used for example the LIDAR, Gyroscope, camera, motors, Jetson nano board, Raspberry Pi RP2040 board and the batteries. The autonomous system Laboratory PC was used for the image processing. The operating system for the Jetracer was installed on a external 64 GB micro SD card. To setup the OS a USB supported mouse and keyboard along with HDMI supported monitor was used. The autonomous system laboratory Wifi was used for communication between the Jetracer and the image processing PC. The Topcon Total robotic station PS-103A was used to measure the ground truth of the vehicle by prism tracking. The 3D printing was performed using laboratory facilities of the university. A checkerboard black and white square pattern of 30.83mm length was used to calibrate the camera. Lastly a infrared sensor based speed measuring device was used to measure the real time speed of the vehicle. The hardware requirements are summerised in Tab. 1.

Tab. 1 Hardware Requirements

Component Type	Component Name
Autonomous Vehicle	Jetracer ROS AI Kit
Image processing Unit	Autonomous System Laboratory Windows 10 PC
OS platform	64 GB mircro SD card
Communication setup	Autonomous System Laboratory Wifi
Jetracer OS setup hardware	USB supported keyboard, mouse and HDMI supported monitor
Ground truth data tracking	TopCon Robotic Total Station PS-103A Kit
3D printing	HSHL University 3D printer
Camera calibration setup	Autonomous system laboratory camera calibration checkerboard
Vehicle speed measurement	Infrared based vehicle speed measurement device

• Software Requirements

windows 10 OS PC is required where Matlab 2019b is installed. This is where the image processing occurs. The Matlab should have the necessary extension packages like Matlab Coder and GPU Coder to generate CUDA codes that can be later exported to the Jetracer. A Solidowork 2019 version is also required for CAD designing and 3D printing of the prism holder from the Topcon system. Putty software was used for initial serial communication with the Jetracer. The Jetpack OS was installed on the 64 GB micro SD card that was used by the Jetracer Jetson Nano Board. The installation guide can be found on JetPack_OS_Online_Guide .Knowledge of Matlab m code, Python and bash code is needed as these are the programming languages used in this project. Python and bash programming languages are used for the Jetracer side server programming while the Matlab m code is used Windows PC side programming. The Tab. 2 summerizes the software requirements for this thesis.

Tab. 2 Software Requirements

Software Applications
Matlab 2019b
Solidworks 2019
Jetpack OS
Windows 10 OS
Putty
NVIDIA CUDA Toolkit
MATLAB Coder Support Package for NVIDIA Jetson
GPU Coder

• Lateral Offset Error

Fig.3 Error Estimation

The lateral offset error is the horizontal distance between vehicle position and the target path trajectory. This error is calculated at a specific look ahead distance in the birds eye view image to avoid jitter in the target path polynomial construction. This error can be calculated by the difference between the target path polynomial trajectory and the birds eye view image center at the look ahead distance. The error calculation equation is $e_L=(x_T-B_c).s_x$ where $e_L$ is the lateral error in m. $x_t$ is the target path x-coordinate in the bird’s-eye-view image [pixels]. $B_c$ is the horizontal center coordinate of the bird’s-eye-view image [pixels]. $s_x$ is the Pixel-to-meter scaling factor in x-direction [m/pixel].

• Localized Heading Error

The localized heading error is used to compensate between the angular orientation of the vehicle and target path trajectory direction. This error is estimated by calculating the angular difference between a chord on the target path polynomial and a unit vector that is parallel to the camera optical axis. $u(t)=K_{P}e(t)+K_I\int e(t)dt+K_D\frac{de(t)}{dt}$

Repository

• Please link your source code here.
• Source Code should match the coding guidelines (MATLAB^® | C)
• Add a header and help to your files to make them understandable for the team (MATLAB^® | C).
• Describe what source code does what.
• What are the preconditions in hardware and software (Autonomous Driving Toolbox, Parallel Computing Toolbox, etc.)?
• Wird eine spezielle Hardware/GPU benötigt?

Source Code Usage Guideline

File Name	Usage	Software Requirements	Hardware Requirements
jetson_lane_detection_video_simulator.m	Lane detection simulation using a recorded video of track	Matlab 2019b, Computer Vision Toolbox, Image Processing Toolbox, Parallel Computing Toolbox (GPU acceleration), appcameracalibration.mat file, track_recording.avi file	Windows 10 PC
appcameracalibration.mat	camera calibration file for Jetracer camera IMX219 - 160 with image resolutions of 360px × 480px	Matlab 2019b	Windows 10 PC
Beispiel	Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel	Beispiel

List of open questions

• The video has images sized 360px x 480px. This does not fit he calibration parameters cameraParams.Intrinsics.ImageSize (480 x 640). This has to match.

Links

• JetRacer ROS Kit
• Waveshare-Wiki: JetRacer ROS AI Kit

---

→ zurück zum Hauptartikel: Studentische Arbeiten