JetRacer: Optimierung der Streckenführung: Unterschied zwischen den Versionen
Keine Bearbeitungszusammenfassung |
|||
| (14 dazwischenliegende Versionen von 2 Benutzern werden nicht angezeigt) | |||
| Zeile 2: | Zeile 2: | ||
[[Kategorie:JetRacer]] | [[Kategorie:JetRacer]] | ||
[[Datei:JetRacer Waveshare.jpg|thumb|rigth|500px|Abb. 1: JetRacer AI Pro von Waveshare]] | [[Datei:JetRacer Waveshare.jpg|thumb|rigth|500px|Abb. 1: JetRacer AI Pro von Waveshare]] | ||
'''Autor:''' Arfat Kamal<br/> | '''Autor:''' [[Benutzer:Arfat Kamal|Arfat Kamal]]<br/> | ||
'''Art:''' | '''Art:''' Projektarbeit<br> | ||
''' | '''Start:''' 20.03.2023<br> | ||
'''Geplante Abgabe:'''<br> | |||
'''Betreuer:''' [[Benutzer:Ulrich_Schneider| Prof. Schneider]]<br/> | '''Betreuer:''' [[Benutzer:Ulrich_Schneider| Prof. Schneider]]<br/> | ||
| Zeile 18: | Zeile 19: | ||
== Anforderungen an die wissenschaftliche Arbeit == | == Anforderungen an die wissenschaftliche Arbeit == | ||
* Wissenschaftliche Vorgehensweise (Projektplan, etc.), nützlicher Artikel: [[Gantt-Diagramm| Gantt Diagramm erstellen]] | * Wissenschaftliche Vorgehensweise (Projektplan, etc.), nützlicher Artikel: [[Gantt-Diagramm| Gantt Diagramm erstellen]] | ||
* | * Wöchentliches Meeting (im Labor): donnerstags 15:15-15:45 Uhr | ||
* Projektvorstellung im Wiki | * Projektvorstellung im Wiki | ||
* Tägliche Sicherung der Arbeitsergebnisse in SVN | * Tägliche Sicherung der Arbeitsergebnisse in SVN | ||
| Zeile 35: | Zeile 36: | ||
==== Familiarization with the existing Framework ==== | ==== Familiarization with the existing Framework ==== | ||
===== Connecting the Jetracer ===== | |||
To be familiarized with the existing framework, [[JetRacer: Teach-In Tutorial]] was followed. Inside the Teach-In Tutorial, another link from [https://www.waveshare.com/wiki/JetRacer%20Pro%20AI%20Kit Waveshare] can be found, which provides a detailed guide to run and train the Jetracer. In addition to these, I am writing some personal remarks that I think would be helpful. | To be familiarized with the existing framework, [[JetRacer: Teach-In Tutorial]] was followed. Inside the Teach-In Tutorial, another link from [https://www.waveshare.com/wiki/JetRacer%20Pro%20AI%20Kit Waveshare] can be found, which provides a detailed guide to run and train the Jetracer. In addition to these, I am writing some personal remarks that I think would be helpful. | ||
| Zeile 42: | Zeile 45: | ||
After we follow all the steps and successfully connect the Jetracer to PC, we get a new IP for the Jetracer. Once we get this, we can just dierectly connect to the Jetracer using that. But sometimes the IP changes randomly. So far I have seen '20', '21' and '25' for the last digit. So if there is trouble with the connetion, it is advisable to check the IP which is shown on the small OLED display of Jetracer like the figure below. | After we follow all the steps and successfully connect the Jetracer to PC, we get a new IP for the Jetracer. Once we get this, we can just dierectly connect to the Jetracer using that. But sometimes the IP changes randomly. So far I have seen '20', '21' and '25' for the last digit. So if there is trouble with the connetion, it is advisable to check the IP which is shown on the small OLED display of Jetracer like the figure below. | ||
[[Datei:IP.jpg|thumb|links|mini|200px|Fig. 1: IP on OLED of the Jetracer]] | [[Datei:IP.jpg|thumb|links|mini|200px|Fig. 1: IP on OLED of the Jetracer]] | ||
<br clear=all> | |||
===== Training the Jetracer ===== | |||
The jetracer is trained using the 'interactive_regression.ipynb' file, which is available inside /jetracer/notebooks. We just need to open the file inside JupyterLab and run all cells. The last program will display the following interface like Figure 2. | |||
[[Datei:Camera Interface.png|links|300px|mini|Fig. 2: Camera Interface]] | |||
Inside the interface, the left most window shows the live preview of the camera and this is where we show the car the ideal path by clicking on a point on the track where we would like it to go. Once we click, a photo is taken and saved inside the dataset. The window in the middle shows the photo and the green dot represents the ideal point that we chose. The right most window also shows a live preview, with the blue dot which shows the ideal point on the track where we would like it to follow. In figure 2, the blue dot is not at the ideal path since it had not been trained yet. | |||
<br clear=all> | |||
==== Project Plan ==== | |||
[[Datei:Project Plan nov.png|links|300px|mini|Fig. 3: Project Plan] | |||
<br clear=all> | |||
==== torch2trt ==== | |||
torch2trt is a PyTorch to TensorRT converter which utilizes the TensorRT Python API. We use this converter because it is easy to use, it can convert modules with a single function call 'torc2trt'. torch2trt is being used in our project since it allows us to take our PyTorch model and convert it into a format that can be efficiently run on NVIDIA GPUs using TensorRT. We are using NVIDIA Jetson Nano in our autonomous car, that's why torch2trt is a good fit for optimization. | |||
== Nützliche Artikel == | |||
* [https://de.mathworks.com/videos/matlab-and-simulink-robotics-arena-deep-learning-with-nvidia-jetson-and-ros--1542015526909.html Deep Learning with MATLAB, NVIDIA Jetson, and ROS] | |||
* [https://github.com/NVIDIA-AI-IOT/jetracer NVidia: JetRacer] | |||
* [https://www.formulaedge.org/what-is-jetracer] | |||
* [https://www.waveshare.com/wiki/JetRacer_AI_Kit Waveshare Wiki] | |||
Aktuelle Version vom 20. Februar 2024, 17:04 Uhr
Autor: Arfat Kamal
Art: Projektarbeit
Start: 20.03.2023
Geplante Abgabe:
Betreuer: Prof. Schneider
Aufgabenstellung
- Einarbeitung in das bestehende Framwework
- Optimierung der KI für den Rundkurs im Labor Autonome Systeme (Geschwindigkeit, Robustheit).
- Nutzung von MATLAB zum Anlernen des Jetson Nano.
- Nutzung von ROS2 zum Anlernen des Jetson Nano.
- Bewertung der Vor- und Nachteile der Programmierungebungen.
- Dokumentation nach wissenschaftlichem Stand im HSHL-Wiki
Anforderungen an die wissenschaftliche Arbeit
- Wissenschaftliche Vorgehensweise (Projektplan, etc.), nützlicher Artikel: Gantt Diagramm erstellen
- Wöchentliches Meeting (im Labor): donnerstags 15:15-15:45 Uhr
- Projektvorstellung im Wiki
- Tägliche Sicherung der Arbeitsergebnisse in SVN
- Tracking der geleisteten Arbeitsstunden
- Studentische Arbeiten bei Prof. Schneider
- Anforderungen an eine wissenschaftlich Arbeit
SVN-Repositorium
Getting started
Lesen Sie zum Einstieg diesen Artikel
Siddiquy, T.: Automated lane following of a Waveshare JetRacer with artificial intelligence. Bachelorarbeit
Documentation
Familiarization with the existing Framework
Connecting the Jetracer
To be familiarized with the existing framework, JetRacer: Teach-In Tutorial was followed. Inside the Teach-In Tutorial, another link from Waveshare can be found, which provides a detailed guide to run and train the Jetracer. In addition to these, I am writing some personal remarks that I think would be helpful.
When connecting to the Jetracer from PC for the first time, the default IP is used, which is 192.168.55.1:8888. It is essential that the Jetracer is connected to the PC via USB during this procedure. Otherwise it will not be connected.
After we follow all the steps and successfully connect the Jetracer to PC, we get a new IP for the Jetracer. Once we get this, we can just dierectly connect to the Jetracer using that. But sometimes the IP changes randomly. So far I have seen '20', '21' and '25' for the last digit. So if there is trouble with the connetion, it is advisable to check the IP which is shown on the small OLED display of Jetracer like the figure below.
Training the Jetracer
The jetracer is trained using the 'interactive_regression.ipynb' file, which is available inside /jetracer/notebooks. We just need to open the file inside JupyterLab and run all cells. The last program will display the following interface like Figure 2.
Inside the interface, the left most window shows the live preview of the camera and this is where we show the car the ideal path by clicking on a point on the track where we would like it to go. Once we click, a photo is taken and saved inside the dataset. The window in the middle shows the photo and the green dot represents the ideal point that we chose. The right most window also shows a live preview, with the blue dot which shows the ideal point on the track where we would like it to follow. In figure 2, the blue dot is not at the ideal path since it had not been trained yet.
Project Plan
[[Datei:Project Plan nov.png|links|300px|mini|Fig. 3: Project Plan]
torch2trt
torch2trt is a PyTorch to TensorRT converter which utilizes the TensorRT Python API. We use this converter because it is easy to use, it can convert modules with a single function call 'torc2trt'. torch2trt is being used in our project since it allows us to take our PyTorch model and convert it into a format that can be efficiently run on NVIDIA GPUs using TensorRT. We are using NVIDIA Jetson Nano in our autonomous car, that's why torch2trt is a good fit for optimization.