Diskussion:JetRacer: Spurführung mit künstlicher Intelligenz

Aus HSHL Mechatronik
Zur Navigation springen Zur Suche springen
# Termin Planung für die Woche Fortschritt
1 12.06.2024- 19.06.2024
  • Safety briefing from Mr Marc Ebmeyer's laboratories
  • Familiarization with the existing frame work
  • Familiarize with the existing system using the wiki article. Gantt chart
  • Handing over the keys to Marc Ebmeyer's laboratories
  • Make Research on the topic
  • Draft a timeline for the project.
2 20.06.2024-26.06.2024
  • Build the Jetracer
  • Setup and Flash Nvidia OS
  • Connect JetRacer to local WLAN
  • Setup Headless control of jetracer
  • Errors and Crashes on the OS (details on Errors and solutions)
  • Find the optimal OS for the jet racer
  • Optimization of the AI for the circuit, the controller (e.g. PD controller) in the laboratory Autonomous systems (speed, robustness).
3 26.06.2024 - 05.07.2024
  • Fixing, debuging errors and testing different Flashversions of the JetRacer
  • Teach the JetRacer automated driving
  • Steering JetRacer with Gamepad
  • Teaching Jetracer interactive_regression
  • Make Research on Matlab, Deep learning and ROS
  • Document Errors
4 05.07.2024 - 26.07.2024
  • Link Jetracer to MATLAB
  • Install and check prerequisite for the jetracer like: CUDA, Cudnn, TensorRT
  • Reinstall CUDA 10.2, CuDNN and SDL1.2 Paths
  • Attempt to programme Jetracer using MATLAB:
  1. Image recognition
  2. Line following
  • Document Errors + Solutions (E.g. Incompatible GPU, Missing Environment Variables paths like: CUDA_PATH and NVIDIA_CUDNN)
  • Speak with other student working on Autonome Spurführung mit einem JetRacer ROS AI Robot
  • Update the Professor by Email for Feedback
6 29.07.2024 -
  • Programme Jetracer using ROS2
  • Evaluation of the advantages and disadvantages of the programming environments.
  • Successfully install and run ROS on jetracer
  • Recreate Alexnet Example on ROS image classification with GPU coder
7 12.08.2024
8 19.08.2024
  • Use the Game controller to steer

Dokumentieren Sie bitte hier Ihren Fortschritt.

9 26.08.2024
  • Programme and teach Jetracer using Matlab and Simulink
  • Understand the Libraries used in JupytrLab
  • Working on previous Task
10 04.09.2024
  • Discussion of subject for the bachelorthesis.
  • Drive the JetRacer in the right lane counterclockwise with the gamepad controller. Limit the speed via Software to a maximum (e. g. 1 m/s).
  • Take a video while driving a lap with MATLAB® using a MATLAB®-script.
  • Load the pretrained NN.
  • Train the pretrained NN with MATLAB® with a MATLAB®-App (GUI) by clicking the desired path in the images.
  • Use classic lane tracking algorithms to teach the NN automatically.
  • Write a PD-contoller that uses the NN to drive in the right lane. Program this in MATLAB® and let it run on the JetRacer-GPU using GPU Coder.
  • Goal: the car should drive autonomously several laps in the right lane as fast as possible.
  • Student choosed a name fot the bachelorthesis.
  • Provide debug option to adjust speed(Throttle) of jetracer to a maximum (e. g. 1 m/s) only on matlab.
11 09.09.2024 - 13.09.2024
  • Student did not attend the meeting.
  • Please cancle the meeting if you are not coming.
  • Drive the JetRacer in the right lane counterclockwise with the gamepad controller. Limit the speed via Software to a maximum (e. g. 1 m/s) wit Professor's assistance.
  • Take a video while driving a lap with MATLAB® using a MATLAB®-script.
  • Load the pretrained NN.
  • Successfully Drive the JetRacer in the right lane counterclockwise with the gamepad controller on matlab. Hard code the speed Limit via Software to a maximum (e. g. 1 m/s).
  • Successfully took a video while driving a lap with MATLAB® using a MATLAB®-script.
  • Load the pretrained NN.
  • Attempt to train the pretrained NN with MATLAB® with a MATLAB®-App (GUI) by clicking the desired path in the images.
  • Documented all my work on wiki platform
12 16.09.2024 - 20.09.2024
  • Use classic lane tracking algorithms to teach the NN automatically.
  • Write a PD-contoller that uses the NN to drive in the right lane. Program this in MATLAB® and let it run on the JetRacer-GPU using GPU Coder.
  • Goal: the car should drive autonomously several laps in the right lane as fast as possible.

Send professor recorded video online

13 21.09.2024

Student did not appear to the weekly appointment. Further research on how to train jetracer

Feedback from professor to use hshl gpt for advance research on how to train a NN

14 30.09.2024

Asking hshl gpt on how to train a NN with the video input from jetracer

Training jetracer based on the researches Uning alternative method(using gamepad to train the jetracer frame by frame with linear image processing method)

15 07.1.2024

Show professor my work progress on NN traning and approche to teach jetracer with controller instead of mouse(Clicking each desirable path on the frame)

  • Testing the jetracer steering and develop PID control
16 14.10.2024
  • Testing jetracer to complete a complete lap on the right lane
  • Recieved feedback from professor to train jetracer to get both input from controller and camera
  • Training the jetracer to get both input from controller and camera
  • Documenting all work progress
17 21.10.2024
  • Programming jetracer to get input from both video and controller
  • Testing and debugging
  • Documentation feedback
  • Meeting with professor
  • Documenting all work progress
18 28.10.2024
  • Training NN with new inputs from the jetracer (controller and camera)
  • Documenting all work progress
19 21.10.2024
  • Programming jetracer to get input from both video and controller(steering angle)
  • Testing and debugging
  • Documentation feedback
  • Meeting with professor
  • Documenting all work progress
120 04.11.2024
  • Testing NN with both inputs from steering angle and camera
  • Documenting all work progress

Errors and Solutions

# Dates Error Descriptions Solutions
1 12.06.2024- 05.07.2024
  • Build the Jetracer
  • Setup Integrated Nvidia OS
  • Connect JetRacer to local WLAN
  • Setup Headless control of jetracer
  • Teach the JetRacer automated driving
  • Steering JetRacer with Gamepad
  • Teaching Jetracer interactive_regression
  • Testing different Flashversions of the JetRacer

Dokumentieren Sie bitte hier Ihren Fortschritt.

* jetbot-043_nano-4gb-jp45.zip


  1. Warning!! this is for Jetbot, not Jetracer.
  2. Jetracer cannot detect SD-card
  3. Connot lunch Nvidia OS because HMI Screen always dark and Jetson Nano turns on but doesn’t boot

Dokumentieren Sie bitte hier Ihren Fortschritt.

* jetson-nano-jp461-sd-card-image.zip(Jetson-nano developer kit)
  1. No build in Jetracer packages like: basic_motion, teleoperation, road_following etc
  2. Incompatible with Jetracer and gives error : jetson nano Adafruit-Blinka requires Python '>=3.7.0' but the running Python is 3.6.9
  3. Incompatible with Jetracer and gives error :No matching distribution found for Adafruit-Blinka>=7.0.0 (from adafruit-circuitpython-servokit)
  4. Cannot run "sudo nvpmodel -m1" and shows nvpmodel not found ||


* jetcard_nano-4gb-jp451.zip (Jetpack4.5.1)
  1. Working but no build in Jetracer packages like: teleoperation, road_following etc
  2. Solution is to use github repository: https://github.com/NVIDIA-AI-IOT/jetracer to clone the teleoperation, road_following etc
 Dokumentieren Sie bitte hier Ihren Fortschritt.


* jetrace.zip (Jetpack4.5)
  1. Incompatible with Jetracer and gives error :ValueError: No I2C device at address: 0x60 (error comes from step 5 of https://www.waveshare.com/wiki/JetRacer_AI_Kit. do not immplement it for pro versions )
  2. only interactive_regression package works with no error
  3. Solution is to use github repository: https://github.com/NVIDIA-AI-IOT/jetracer to clone the basic_movement, teleoperation, road_following etc
  4. Can't connect to SSH header
 Dokumentieren Sie bitte hier Ihren Fortschritt.
2 08.07.2024 - 02.09.2024
  • Research for resources and links on how to programme Jetracer using MATLAB/ROS2
  • programme Jetracer using MATLAB
  • Use of ROS2 to teach the Jetson Nano.

Dokumentieren Sie bitte hier Ihren Fortschritt.

Cuda 10.2 set path
dpkg: error processing package nvidia-l4t-bootloader

https://forums.developer.nvidia.com/t/solution-dpkg-error-processing-package-nvidia-l4t-bootloader-configure/208627

```Code: sudo mv /var/lib/dpkg/info/ /var/lib/dpkg/backup/ sudo mkdir /var/lib/dpkg/info/

```

Next update repos and force install .

```Code:

sudo apt-get update sudo apt-get install -f

```

Move the new structure dpkg/info to old info

```Code:

sudo mv /var/lib/dpkg/info/* /var/lib/dpkg/backup/

```

Remove the new dpkg structure folder and back the old

```Code:

sudo rm -rf /var/lib/dpkg/info sudo mv /var/lib/dpkg/backup/ /var/lib/dpkg/info/ ```

resources: https://elinux.org/Jetson/Installing_CUDA

Nvcc not Found

- **Edit `.bashrc`:** Since `vim` and `nano` are unavailable, use `vi` if it's installed: 

   ``Code:
   vi ~/.bashrc
   
   ```

- **Clean Up Environment Variables:** Update the `.bashrc` file to remove duplicates and ensure only one set of `PATH` and `LD_LIBRARY_PATH` entries: 

   ```Code:
   # Add CUDA bin & library paths
   export PATH=/usr/local/cuda-10.2/bin:$PATH
   export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH
   
   ```

- **Save and Exit:** 

   - In `vi`, press `i` to enter insert mode.
   - Make your changes.
   - Press `Esc`, then type `:wq` and press `Enter` to save and exit.

- **Source `.bashrc`:** Code: source ~/.bashrc

Install Cudnn

Dokumentieren Sie bitte hier Ihren Fortschritt.

SDL 1.2 installation

1. **`nvcc`**: Ensure CUDA is correctly installed and its path is added to your `.bashrc` file. 2. **SDL 1.2**: Install the development libraries for SDL 1.2 on your system. You can typically do this with: 

   ```code:
   sudo apt-get install libsdl1.2-dev
   
   ```
No publick key(update error)

The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC Reading package lists... Done W: GPG error: http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC E: The repository 'http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease' is not signed. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details.

The error message indicates that the GPG key for the NVIDIA CUDA repository is missing. To resolve this, you can manually add the key:

1. **Add the missing key**: 

   ```Code:
  
   sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
   
   ```

2. **Update the package list again**: 

   ```Code:
   sudo apt-get update
   
   ```

This should fix the GPG error and allow you to update the repositories.

Structure of the thesis

  1. Introduction
    1. Task overview
    2. Gantt-Chart
    3. Organisation Plan/Outline

Where do I find the solutuins to the tasks.

  1. Theoretical Background
  2. Solution Way/Alternatives
  3. Solution/Result
  4. Conclusion
    1. Discussion of the results
    2. Outlook

Literature/source Attachments/SVN Links

Abgerufen von „https://wiki.hshl.de/wiki/index.php?title=Diskussion:JetRacer:_Spurführung_mit_künstlicher_Intelligenz&oldid=126132“