Lane Keeping with AI
| Autor: | Moye Nyuysoni Glein Perry |
| Art: | Project Work |
| Starttermin: | 14.11.2024 |
| Abgabetermin: | 31.03.2025 |
| Betreuer: | Prof. Dr.-Ing. Schneider |
Introduction
This project focuses on developing an autonomous lane-keeping system for a 1:20 scale model car equipped with a camera. The system’s goal is to train an artificial intelligence (AI) Model to drive autonomously on the track using image processing and signal processing techniques. In this work, steering angle data and video recordings are employed to train an artificial intelligence (AI) model. Two main approaches are explored:
- Training a Convolutional Neural Network (CNN) from scratch
- Using Transfer Learning (with ResNet-50 and integrated PID/MPC control)
These methods are compared using a morphological (Zwicky) box framework that considers various design parameters. Signal processing plays a key role in preprocessing the visual data and augmenting the training dataset.
Lane-keeping refers to the ability of a vehicle to detect and maintain its position within a designated lane on the road. It is a critical function for autonomous driving, helping to enhance road safety and reduce driver workload. Our target location is the mechatronics lab at HSHl. below are videos recorded using videos from a JetRacer
Visualized Data of Sample01
Visualized Data of Sample02
System Requirements
The lane-keeping system must fulfill the following requirements:
Autonomous Operation: Detect lane markings and maintain the vehicle in the right lane.
Real-Time Processing: Process video data in real time to generate accurate steering commands.
Data Synchronization: Synchronize steering angle measurements with video frames.
Robustness: Handle varying lighting conditions, noise, and occlusions using signal processing techniques.
Scalability & Adaptability: Provide flexibility to incorporate different AI training strategies (from-scratch CNN vs. transfer learning).
Visualization & Analysis: Use MATLAB® scripts (e.g., zeigeMessdaten.m) to visualize and verify data and performance.
Documentation & Reproducibility: Maintain thorough scientific documentation including code, parameters, and evaluation criteria.
Research and Methodological Approach
Solution Alternatives and Morphological Box
The morphological box is used to systematically explore and compare various solutions based on design parameters and their possible values. The following table summarizes the alternatives:
Signal Processing in Lane Detection
Signal processing is central to the project:
Preprocessing: Frames are converted to grayscale, contrast-adjusted, resized, and noise-filtered (using Gaussian filters and morphological operations) to enhance lane features.
Data Augmentation: Multiple versions of each frame (e.g., flipped, brightness-adjusted) are created to increase dataset diversity and improve model generalization.
Feature Extraction: Edge detection and Hough transforms (with morphological closing) are applied to detect lane markings robustly, especially in challenging scenarios such as dotted or faded lines.
These techniques enhance model training, leading to more reliable lane detection.
Implementation in MATLAB®
Two main implementation strategies have been developed and compared.
Data Acquisition and Preprocessing
Data Sources: Videos and corresponding steering angle data (from Messdaten_03.mat to Messdaten_15.mat) are used.
Preprocessing Pipeline:
Convert each frame to grayscale.
Adjust contrast and resize frames (e.g., to 64×64 for the CNN from scratch or 224×224 for ResNet-50).
Normalize steering angle data to a fixed range (e.g., [-0.2, 0.2]) and discretize for classification tasks.
CNN from Scratch
Overview This approach builds a CNN from scratch with the following key steps:
Directory Setup & GPU Initialization: The code resets the GPU, sets up directories for training images, validation images, and saving trained networks.
Data Loading & Preprocessing: Video frames are preprocessed (grayscale conversion, contrast adjustment, resizing) and synchronized with steering angle data.
Discretization of Steering Angles: Steering angles are normalized and discretized into bins (21 bins over the range) to form classification labels.
Data Augmentation: Due to low training data, we apply data augmentation (small rotations, translations) to boost the training dataset.
Neural Network Architecture: The custom CNN architecture consists of three convolutional layers with batch normalization, ReLU activation, max pooling, followed by fully connected layers and a softmax output. Training & Simulation: The network is trained using MATLAB’s GPU acceleration and validated on a separate dataset. Finally, the trained network is used to simulate lane keeping, and the output video is generated with overlaid steering angle comparisons.
Key Code Segments:
% Define On-The-Fly Data Augmentation augmenter = imageDataAugmenter( ... 'RandRotation', [-2,2], ... 'RandXTranslation', [-3,3], ... 'RandYTranslation', [-3,3]); augimdsTrain = augmentedImageDatastore([64 64 1], imdsTrain, 'DataAugmentation', augmenter);
.
Transfer Learning with ResNet-50 and PID Control
Overview This method leverages transfer learning using the pretrained ResNet-50 model:
Data Preprocessing: Similar preprocessing (resizing to 224×224) is applied with additional lane detection computations. An improved lane deviation function employs Gaussian filtering, Canny edge detection, morphological closing, and Hough transforms to compute a steering correction.
Data Augmentation: The dataset is augmented by flipping images and adjusting brightness, which is especially important for increasing robustness in the learning process.
Network Adaptation: The final classification layers of ResNet-50 are replaced with new regression layers that output a continuous steering angle.
Control Integration: The predicted steering angle is further refined using a PID controller (or an MPC variant), providing smoother control signals for lane keeping.
Training and Simulation: The network is trained combined with our dataset, and the performance is validated by simulating the lane-keeping task with temporal smoothing of the steering command.
Key Code Segments:
% Remove final layers and add regression layers for steering angle prediction
lgraph = removeLayers(lgraph,
{'fc1000','fc1000_softmax','ClassificationLayer_fc1000'});
newFc = fullyConnectedLayer(1, 'Name', 'fcSteering');
regressionLayer = regressionLayer('Name', 'regOut');
lgraph = addLayers(lgraph, newFc);
lgraph = addLayers(lgraph, regressionLayer);
lgraph = connectLayers(lgraph, 'avg_pool', 'fcSteering');
lgraph = connectLayers(lgraph, 'fcSteering', 'regOut');
This snippet illustrates the network modification required for regression, transforming the classification network into one that predicts a continuous value.
Additionally, a PID controller is implemented to smooth the output of the network, ensuring that the control signals are both responsive and stable.
Task
- Set up requirements for a lane keeping system.
- Research on solutions for the task (morphological box with technical creteria).
- Implementation in MATLAB®. Teach a AI to drive in the right lane.
- Use the Video data here.
- Use the recorded steering angle here.
- The data can be visualized with zeigeMessdaten.m (see fig. 1).
- Verify the results with Rundkurs01.avi and Rundkurs02.avi.
- Evaluation of the solutions using a morphological box (Zwicky box) based on technical creteria
- Discussion of the results
- Testing of the system requirements - proof of functionality
- Scientific documentation of every step as a wiki article with an animated gif
→ zurück zum Hauptartikel: Signalverarbeitung mit MATLAB und Künstlicher_Intelligenz