Automated lane following of a Waveshare JetRacer with artificial intelligence
Autor: Tasawar Siddiquy
Type: Bachelor Thesis
Duration: 14.09.2022 - 09.01.2023
Supervisor: Prof. Schneider
Tasks
- Mechanical and electrical construction of a JetRacer
- Literature research on the available programming environments
- Comparison and selection of a suitable programming environment
- Familiarization with the programming environment
- Training the deep learning network for track guidance
- Module and system tests with the JetRacer on the track
- Presentation of results and outlook
- Documentation of the handling of the JetRacer in the HSHL Wiki
- Provision of demo programs for easy access
- Documentation according to scientific standards
Requirements for the scientific work
- Scientific approach (project plan, etc.), useful article: Gantt Diagramm erstellen
- Weekly progress reports (informative)
- Project presentation in the wiki
- Studentische Arbeiten bei Prof. Schneider
- Anforderungen an eine wissenschaftlich Arbeit
Getting Started
Nutzen Sie diese Artikel, um sich in das Thema einzuarbeiten:
- Wiki-Artikel_schreiben, die Vorlage finden Sie hier: Artikelvorlage
- Regeln zum Umgang mit SVN
- HSHL-Wiki: JetRacer
Useful Articles
- Anleitung zum einfachen Einstieg in ROS2
- Doku des Projektstands WS2021/22
- Sicherer Betrieb eines AMR
- Navigation eines AMR mit ROS2
Project Plan
Requirements
Hardware Requirements
Software Requirements
Python was chosen for the implementation because it is very user-friendly and makes it possible to connect the Windows PC and JetRacer directly through wifi. Jupyter notebook was used to create the code. All of the necessary JetRacer routines are already present in a library file that has already been created for the device.
Functional System Design / Technical System Design
Component Specification
Specifications of the Jet-Racer [1]
Programming
For the implementation phase, a Jupyter notebook was utilized, and the code was developed in Python since it is extremely user-friendly and permitted direct wifi connectivity between the Windows PC and the Jetracer. A library file for the jetracer already exists. It has all of the basic functionalities for the jetracer and has been designed.
First, using the Etcher tool, an SD card was produced with a prebuilt image based on the jetpack 4.5 file. This file was obtained from the waveshare website. Installing the jetpack 4.5 file followed. The NVIDIA JetPack software development kit is the most comprehensive instrument for creating apps with artificial intelligence.The JetPack software development kit is compatible with all Jetson modules and developer kits. The most recent Linux Driver Package for the Linux operating system is included in the JetPack software development kit, together with CUDA-X accelerated libraries and APIs for Deep Learning, Computer Vision, Accelerated Computing, and Multimedia. The software development kit further contains the Linux kernel.
The micro SD card was then inserted into the Jetson nano, and the device is now set to connect to a wifi network using the Jupyter notebook. Having gotten the IP address via the USB connection, the system is now accessible from the personal computer. After successfully login in, the Jupyter notebooks' code will be available and can be modified. Following the installation of Python packages, the power mode should be specified. This may be seen on their website in full detail. First, it was evaluated using a controller that enables the user to manually manipulate the jetracer. The jetracer comes with this controller. Inside is a file for a simple motion and teleportation notebook, which the controller may use to operate the jetracer. It is advised that the controller be connected to the computer through the jupyter notebook, which then transfers the signal to the jetracer. The controller may be calibrated by utilizing the ipywidgets library. Using the Jupyter notebook, every aspect of the car, including the throttle gain, steering offset, and axes, may be modified[1].
The process of collecting data and feeding it into a neural network model is one of the most crucial components of developing one. For our needs, the data must portray the track from many perspectives. The jetracer's camera has been employed for this purpose. The camera widget is configured to show a preview on one screen and precisely define the direction vector, depending on which the autonomous vehicle will go, on the second screen. Use the widget's sliders to modify these parameters[1].
Component Testing
The output float value has been tested. It is giving a positive or negative value, which means the steering values are correct. And these positive and negative values enable the JetRacer to understand which way to turn the steering.
Result
After successfully completing all of the processes, the car was able to drive without assistance. It has been put through a significant number of test laps, and each time it has demonstrated its ability to stay on the track and negotiate its lanes properly.
In the following gif, one lap has been recorded, which takes 25 seconds.
Summary
There are several different methods that can be used to create lane-following. Additionally, you could use Matlab, Simulink, or ROS to achieve this. However, this thesis has accomplished it by utilizing neural networks and a number of library applications. It is also conceivable to state that the result will vary based on the specific model that was applied to the data because different models are created to do a range of functions.
Lessons Learned
There are types that emphasize precision while others concentrate on speed. However, these factors must be considered in the car industry. Additionally, how it is done depends on the people who are interested in the different vocations. Artificial intelligence is going to rule the world in the future. This project on the subject of my thesis has resulted in a good level of deep learning proficiency. The number of applications the Convolutional Neural Network could have surprised me when I initially learned about it.
Project Documents
YouTube Video
You can find the video of this project on Youtube under the link:
Web Links
References
- ”jetracer ai kit b, ai racing robot powered by jetson nano”, comes with waveshare jetson nano dev kit, https://www.waveshare.com/wiki/mainpage, 2023.25
- Deep learning with matlab, nvidia jetson, and ros video,https://de.mathworks.com/videos/matlab-and-simulink-robotics-arena-deep-learning-with-nvidia-jetson-and-ros–1542015526909.html, 2023.
- Kacper Podbucki and Tomasz Marciniak. Aspects of autonomous drive control using nvidia jetson nano microcomputer. In 2022 17th Conference on Computer Science and Intelligence Systems (FedCSIS), pages 117–120, 2022.
→ zurück zum Hauptartikel: Studentische Arbeiten