Professor Howard M. Schwartz, Ph.D., PEng.

Graduate Thesis Topics:

1) Multi Robot Reinforcement Learning

We have done substantial research in the area of multi robot learning. This research topic is suitable for full time graduate students working towards their masters and Ph.D. degrees. The idea is to develop control and learning algorithms that will enable teams of robots to cooperate and compete. We use methods of Multi Agent Reinforcement Learning in combination with ideas from Game Theory and Genetic Algorithms. The work may be done using a grid or in continuous space using Fuzzy Logic Controllers and Estimators.

2) Guarding A Territory Game

We have done substantial research into the game of guarding a territory. The idea is for a guard agent or robot to capture an invading agent or robot before it reaches a target. In these games both the guard and the invader are learning their optimal strategies. A recent contribution has examined the case of the invader using deception to trick the guard. We simulate the case of a guard robot trying to defend up to 3 territories from invasion by an invading robot. The guard must learn how to guess which territory is the true target of the invader. 

Click here for video of guarding a territory with deception

3) Cooperating Swarm Robotics

We are investigating the use of relatively simple robots to achieve swarm based intelligence. We have developed a number of these robots that work together. Methods of adaptation and learning are implemented in each robot. The behaviour of each robot can change to reflect the group and individual goals of the swarm and robot respectively. Ideas from game theory, adaptive control and social colonies are integrated to develop these methods. This thesis topic is suitable for Doctoral and Masters students. Please click on the video below to see 3 of the robots leave a room.

Click here for video of swarm robots leaving a room

  In the next video we show a simulation of a swarm of robots coming together followed by obstacle avoidance. There is no leader and each robot is operating independently following very simple rules. These results can be applied to applications associated with crowd dynamics, automated highways, unmanned vehicles etc.

Click here for video of forming a swarm followed by obstacle avoidance.

    In the next video we show our experimental robots playing follow the leader and avoiding obstacles. The lead robot is being controlled over a bluetooth link and the two robots that follow are under local autonomous control. A video camera in used to locate the robots and the position of all the robots is then transmitted to each robot. The robots compute their own navigation solution.

Cooperative Robots playing follow the leaderVideo of follow the leader

Bachelor’s Honours Theses

1) Rubik's Cube Solving Robot

The video below shows  a robotic Rubik's Cube solver. It was constructed as a fourth year honours thesis project. The students constructed it out of parts from an old photocopy machine. Click on the link below.

Click here for video of Robotic Rubik's Cube Solver

2) Mobile Robotics

The video below shows a mobile robot being operating with a Handyboard microcontroller. It is using a sonar system to sense obstacles and then changes direction. This work was done as a fourth year honours thesis project.

Click here for video of mobile robot with sonar

3) Cooperating Robots

The video below shows two robots working together. They pass a part from one robot back to the other. The system is completely automated. The student who implemented this system combined real time control theory with real time software. The robots are communicating with each other over a serial connection.

Click here for video of cooperating robots (200 MB)

Thesis Supervision:

I am comfortable supervising theses in the area of machine learning for control systems and robotics. I am particularly interested in the field of fuzzy reinforcement learning and control and related intelligent control topics. My priorities in supervising graduate students are summarized in order of importance as:

1) The student completes an acceptable thesis and graduates in a timely fashion.
2) The student is interested in their chosen topic.
3) The student has a positive research experience.
4) The student publishes the research in some venue.

I like to start our first meeting by having a general discussion of the student's research interests. In our second meeting, I will propose research topics that may be of interest and I will ask the student to look at literature in the area. Shortly thereafter, the student and I will focus on a particular set of research papers and on a concise topic.

Back to Professor Schwartz's Home Page