AlphaBot: Lane tracking with camera

Abb. 1: AlphaBot: Lane tracking with camera

Autor:	Syed Muhammad Abis Rizvi
Art:	Praxissemester
Studiengang:	ELE
Starttermin:	02.03.2026
Abgabetermin:	21.06.2026
Betreuer:	Prof. Dr.-Ing. Schneider
Sprache:	DE EN

Introduction

This is an Alphabot by waveshare which supports advanced navigation and sensing features. In this project our task is to drive the Alpha bot with in the right lane along the course by using a pixy2 camera and modify its speed limit. We also added a GY-85 gyroscope IMU sensor to record the Yaw rate so that we can later calculate the yaw angle from the yaw rate data, for getting better output results we replaced the Aplhabot's ardunio uno board with arduino r4 wifi, so that we can have get accurate results via web sever which we can monitor on matlab.These components works together so that Alpha bot can drive independently in real time.

Requirements

Table 1: Requirements for the Unit

Req.	Description	Priority
1	An AMR must drive autonomously in the right-hand lane.	1	✓
2	The Topcon Robotic Total Station is used as the reference measurement system.	1	✓
3	The AMR must evaluate the road data via camera (Pixy 2.1) to follow the lane.	2	✓
4	The reference values ​​must be recorded with MATLAB (x, y, ${\mathrm{\Psi }}_T$).	1	✓
5	Measurement errors must be appropriately filtered.	1	✓
6	The two-dimensional digital map showing the robot's pose during movement must be provided as a MATLAB® file (.mat).	1	✓
7	The solution path and solution must be documented in this wiki article.	1	✓
8	An AlphaBot must be used as the AMR.	1	✓
9	MATLAB®/Simulink must be used as the control software.	1	✓	Arduino was used instead of MATLAB/Simulink because Simulink generated errors. This alternative solution was approved by Prof. Schneider and MATLAB was used for getting Gryo sensor outupt results such as YawRate and Yaw Angle.
10	From the measured yaw rate ${\dot{\mathrm{\Psi }}}_G$ the yaw angle ${\mathrm{\Psi }}_G$ must be determined and compared to the reference ${\mathrm{\Psi }}_T$ from Req. 4.	1	✓
11	The AlphaBot's speed must be optimized to it's maximum.	1	✓

Planning

I planned the Aphabot in such a way that first I used Pixy2 camera for lane detection on Arduino, once it was successfully achieved I mounted Gyro-85 modul on the Alphabot to record the YawRate and YawAngle and again programmed this on Arduino. And everthing was working and I was geting all outputs on Arduino serial monitior. After that I added a bluetooth modul HC-05 to get my output wirelessly on Matlab but unfortunately it was lagging so I replaced the Alphabot's arduino uno with arduino r4 wifi for getting more accurate output via webserver which later mointored on matlab.I am getting seperate output graphs for all the individual outputs and I have also added a counter for output results.

Control Loop

AlphaBot Control Loop

The Pixy2 camera initiates the AlphaBot line-following control loop by using its vector-based line tracking algorithm to identify the track line. The observed path is represented by a directed vector created by the camera that has a head (end point) and a tail (start point). The line center position, or targetX, is determined by processing this data.The tracking error (e) is calculated by comparing the measured position with the intended image center (frameCenter = 39 px). The steering controller receives this inaccuracy and uses it to calculate the necessary steering correction to maintain the robot's alignment with the line. Additionally, the robot's yaw rate is measured by the GY-85 gyroscope, which provides a stabilizing term that enhances tracking performance during curves and quick moves.The speed control logic and the resulting steering command are then applied to the AlphaBot motors. The robot's position in relation to the track varies as it moves, and the Pixy2 camera continuously produces updated line measurements. A closed-loop feedback system that allows precise and reliable line following is created by feeding these measurements back to the controller.

1. Configure the pixy 2.1, so it detects the right line.
2. If the vector direction switches, correct/invert it.
3. Calculate the lateral position error $ey=y_{line}-yset$ of the AlphaBot from the position of the line-vector.
4. Calculate the curvature ${\mathrm{\Psi }}_{line}=\arctan (\frac{dy}{dx})$ of the AlphaBot from the direction of the line-vector.
5. Take the perspective transformation $e_{\mathrm{\Psi }}={\mathrm{\Psi }}_{line}-{\mathrm{\Psi }}_{Set}$ into account.
6. Measure and low pass filter the yaw rate \dot{\Psi}.
7. Lateral controller: $u=K_y\cdot e_y+K_{\mathrm{\Psi }}\cdot e_{\mathrm{\Psi }}+K_{\dot{\mathrm{\Psi }}}\cdot \dot{\mathrm{\Psi }}$
8. look ahead: 0,5 m into the future
9. longitudinal controller: speed up if the road is straight; lower speed in curves

AlphaBot Project Gantt Chart

Gantt Chart

Gantt Chart

The Gantt chart shows the schedule of the AlphaBot project. It includes the main project phases such as planning, implementation, testing, and evaluation. The chart helps track the progress of tasks and milestones throughout the project.

Working principle

The AlphaBot uses the Pixy" camera to detect the lane which helps Alphabot to poistion itself on the track and based on this infromation the motor runs and the Alphabot runs independently on the track and stop immediately when it does not detect any line.The horizontal x-coordinate value ranges between 0 and frameWidth (79) and the vertical y-coordinate value ranges between 0 and frameHeight (52).As they are created on a 79x52 occupancy grid and greater resolutions are not possible due to memory limitations, the line/vector coordinates have a lower resolution so that is why (39) is the estimated frame centerof Pixy camera.

GY-85 IMU sensor measures the yaw rate, which is used to detemine the AlphaBot's (yaw angle) according to its movements.

For more stable ouput I used arduino r4 wifi which sends output via websever which we can also monitor on matlab.

All componennts work together in real time to make sure the AlphaBot runs independently in real time.

Lane Tracking Implementation Results

Lane tracking with Pixy2 camera

The video shows the result of the lane-tracking implementation, demonstrating the Alphabot successfully following the track.

Lane tracking Imlementation Results With New Controller

Lane tracking with Pixy2 camera updated controller

The video shows the result of the lane-tracking implementation, demonstrating the Alphabot successfully following the track with the updated controller.

Lane Following Closeup

Lane following.mp4

The video shows Alphabot running on the track in the middle lane.

YawRate and YawAngle Results With Counter On MALTAB

YawRate and YawAngle results with Counter on MALTAB

The grpah shows how gyroscope data is processed in MATLAB.The yaw angle is obtained by filtering the raw yaw rate and then integrating it over time. A counter was added to moniter how many samples had been processed during the data-processing routine.This output is only used to demonstrate the processing steps and is not based on an actual track measurement.

Prism Mounted On The Alphabot

Prism mounted on Alphabot

A prism was mounted on the AlphaBot so that the Topcon can accurately track and record the Alphabot's position during journey.

TOPCON Implementation Results

TopCon Slowed

Prism tracking with TopCon

The videos above provide a clearer view of how the TopCon is tracking prism mounted on the top of the Alphabot throughtout the journey.

TOPCON Implementation MATLAB Results

TopCon Results

The above graph shows the Alphabot's trajectory (red Line) on the reference track map (black lines).This was recorded on MATLAB by the help of TopCon and Prisma mounted on the top of Alphabot. You can see that Alphabot(red path) closely follows the planned route of entire track while small variations can be seen particularly at curves and bends but in overall journey Alphabot maitained stable navigation and successfully finished the track lap.

TOPCON Vs Gyro GY-85 Sensor Comparision

Yaw Rate Comparison

Yaw Angle Comparison

The first graph shows the comparison of the yaw rate measured by the Gyroscope (blue) and the TopCon (red).Both signals stay nearly zero when the Alphabot was moving in straight line and they rise when the Alphabot makes turns. The Topcon data displays multiple sharp spikes between 15 and 18 seconds, suggesting measurement noise or temporary estimation errors. By contrast, the gyroscope offers a smoother and more stable yaw-rate measurement during the experiment.

The second graph shows the comparison of the yaw angle measured by the Gyroscope (blue) and the TopCon (red).Both measurements follow a similar trend, showing that the Alphabot continuously turns while navigating the track.As time goes on, the differences between the two curves become increasingly apparent due to sensor drift, integration errors, and noise in the yaw-rate measurements.Despite these deviations, both systems capture the overall heading changes of the robot successfully.The yaw angle is cumulative, derived from the integration of the filtered yaw rate over time. As a result of the numerous bends on the track, the angle surpasses 360° even within single lap.

Technical Overview

The system is built on the waveshare AphaBot which is integrated with motors. A pixy2 camera is used for keeping the AlphaBot on track.

Gy-85 IMU used to record yaw rate and yaw angle.

Arduino r4 wifi is used for collecting the output wirelessly in real time.

Pin Assignment

Component	Module Pin	Connected Controller Pin
Pixy2 Camera (I2C)	SDA	A4
	SCL	A5
	VCC	5V
	GND	GND
GY-85 IMU Sensor (I2C)	SDA	A4 (shared I2C)
	SCL	A5 (shared I2C)
	VCC	3.3V / 5V
	GND	GND
Left Motor	PWMA	D6 (PWM)
	AIN1	A1
	AIN2	A0
Right Motor	PWMB	D5 (PWM)
	BIN1	A2
	BIN2	A3
Power Supply	Battery / VCC	AlphaBot power input; sensors powered via onboard voltage regulator

Measurement method

Measurement is done by using onboard sensors on AlphaBot. Pixy2 camera detects the lane position from which lateral error is calculated. The GY-85 IMU sensor measures the yaw rate which helps determine the yaw angle.

Formula for yaw angle.

\[ \theta(t) = \theta_0 + \int_{0}^{t} \omega(t)\, dt \]

\[ \theta_k = \theta_{k-1} + \omega_k \cdot \Delta t \]

Measuring Circuit

The measuring circuit consists of the AlphaBot and the GY-85 IMU Sensor for recording yaw rate and yaw angle.

Software

PixyMonV2 is used to teach Pixy2 camera to specifiy lane tracking and enhance its vision settings using color-based filtering.

AlphaBot is coded in Ardunio C/C++.

For output reults are monitored on Matlab with help of graph plots

Arduino IDE

The AlphaBot's software is created with the Arduino IDE, which offers a unified environment for writing, compiling, and uploading code to the microcontroller.

The program utilizes Arduino C/C++ and incorporates standard libraries like Wire.h for I2C communication and Pixy2I2C.h for the Pixy2 camera. The loop function operates continuously to read the sensors, compute the yaw rate and yaw angle, and manage the motors.

The Serial Monitor allows for the viewing o data in real-time (line error, yaw rate, yaw angle). This data can also be monitored via webserver and on matlab.

Simulink / MATLAB

The AlphaBot’s behavior can be modeled, simulated, and analyzed using Simulink, a visual programming environment based on MATLAB.

Matlab : It shows serial output data from Arduino r4 wifi such as (yaw rate, yaw angle ).MATLAB receives wireless data and plots withh added counter.

Measurement

The measurement system records the robot’s navigation performance using onboard sensor.

The Pixy2 Camera for lane detection .

The GY-85 IMU Sensor for measuring yaw rate and yaw angle.

Datasheet

Component	Model / Type	Key Specifications	Function in Project
AlphaBot	Waveshare AlphaBot	2 DC motors, motor driver, chassis, 6–12V power input	Mobile platform for autonomous navigation
Pixy2 Camera	Pixy2 CMUcam5	60 fps I2C/SPI/UART interface Line-tracking mode	Lane detection and position measurement
GY-85 IMU	GY-85 (HMC5883L + MPU6050)	3-axis gyroscope, accelerometer, magnetometer I2C interface	Measures yaw rate, heading, and motion data

Lighting Conditions

Lighting Conditions	Window Blinds	Switch Control

To create ideal ligthing condion windows blinds were lowered down and only the lab lights were kept turned on and the switch cintrols of these lights are also mentioned in the above image inside the red rectangle.

Implementation Challenges

AlphaBot Instability Due To Track Conditions	Track Condition

As you can see in above video and image there are some joints on cetrain track sections and when Alphabot crosses these at higher speeds it experienced vibrtations and bouncing which affects stability of Alphabot and pixy2 camera mounted on it. So it recieves distored and shifted vector input, causing wrong estimation of lane which reslut in loss of lane tracking and also loosening of wiring which stop the Alphabot later discussed in wiring section.

Wiring

Old Wiring	Wiring Problem	New Wiring

As you can see in first image the original old wiring connected to Alphabot consisted of pins conncetions which tended to lossen up by vibration and bounce of Aplhabot due to certain patches on track. As in above video you can see due to lossen wires the Alphabot can be seen powered on but does not moves until the wires are pressed back into place. To solve this issue we soldered the wiring of Alphabot.

Wheel

Old Wheel	Ball Wheel	Wheel Holder

The above images show Alphabot caster wheels. The old wheel was has fixed ball which can not be removed for cleaning over time the dust particles stucks inside the ball hardly moves and Alphabot could hardly reach its potential speed. To solve this issue we got the new replacement wheel which has a removalbe ball that can be taken out for cleaning and maintenance so that Alphabot can reach its potential speed.

Conclusion

The lane tracking with camera was successfully implemented to autonomously follow the lane using pixy2 camera. A GY-85 gyro sensor was intergrated to measure the yaw rate and calculate the yaw agnle. Topcon was used as a reference measurement system.The gyroscope and Topcon measurements were compared, revealing similar overall trends and effectively capturing the Alphabot's heading changes.The reliability of the Alphabot was enhanced through hardware upgrades, such as soldered wiring connections and mouonting a new caster wheel. At the same time, software optimizations allowed for greater operating speeds and better tracking performance.. The AlphaBot managed to finish the track and demonstrate stable lane-tracking performance, despite facing challenges like track irregularities and wiring issues.

Related Links

Waveshare AlphaBot Infromation and Demo Codes
https://www.waveshare.com/wiki/AlphaBot?srsltid=AfmBOopX1Pauk5MTo8QXPctFf7l_xvLlTFmiBLLC_auJVSvwZtxnXRu_

Connecting Pixy to Arduino guide:
https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:hooking_up_pixy_to_a_microcontroller_-28like_an_arduino-29

Pixy Tunning Lab:
https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:ccc_tuning

Line tracking through PixyMon:
https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:line_quickstart

Line tracking API:
https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:line_api

Topcon Robotic Total Station:
https://wiki.hshl.de/wiki/index.php/Referenzmessung_mit_der_Topcon_Robotic_Total_Station

SVN-Repository

https://svn.hshl.de/svn/HSHL_Projekte/trunk/AlphaBot

---

→ zurück zum Hauptartikel: Aufbau und Test eines Autonomen Fahrzeugs