Oct 2020 - Dec 2020, Group Project with Peter Sauer, Yuming Jin and Juntao He
Northwestern ME-495 Embedded Systems Final Project
Introduction
This project focuses on combining the techniques from computer vision and robotic arm manipulation to enable the sawyer robot arm to play connect-4 against a human player.
The project involves two cameras, on on the robot wrist and the other one on its head monitor. The wrist camera is used to accurately pick up and place the checkers using April tag position markers. And the head monitor camera is used to observe and determine the status of the game board.
Architecture
Robot
A sawyer robot arm (from the Center of Robotics and Biosystems at McCormick School of Engineering) was used in this project. Apart from the arm itself, we also utilized the cameras on the wrist and the head. Moveit (a ROS package) was used to control the movement of the robot arm so that it can pick up a checker and place it in corresponding slots. (This part was mainly done by Peter Sauer)
Computer Vision
Raw image data was aquired from the cameras on the robot’s wrist and head. Hough line transformation and april tags are then used to increase the accuracy of detecting the slots on the board. After that, a 2D-array was used to store the state of the board.
For this project, I mainly focused on the game board scanning and analyzing process. Since the head monitor camera views the game board from an upper angle, the image it records is tilted. In order to restore the image into a horizontal parallel view, four apriltags are placed on four corners of the board, and a perspective transformation is performed.
Artificial Intelligence
A perfect game strategy using the Minimax algorithm was implemented for the AI. As a result, as long as the robot is placing the checker first, it will always win. (This part was mainly done by Yuming Jin and Juntao He)
Result
With some fine tuning of the robot’s and board’s initial positions, the robot can now successfully pick a checker provided by the human player, assess the current state of the board, make a decision and play the checker.
Future Work
Currently the robot can only pick a checker from the human player’s hand. If given more time, it would be nice to make the robot pick up checkers from a place by itself.
Skills Involved:
ROS (Software Design) CV (AprilTag Recognition) Python (Software Programming) Laser Cutting (Part Fabrication) Adobe Illustrator (Materials Design)