Class gives CS students a chance to program an actual autonomous vehicle

David A. Forsyth couldn’t image 30 years ago that his career in academia would involve standing in front of a moving driverless vehicle with a traffic cone on his head. However, this spring that’s what the South African born professor of computer science did as part his University of Illinois class on artificial intelligence.

As the world closes in on the possibility of autonomous vehicles becoming a reality, artificial intelligence (AI) will play a big part in the development. In that light, Forsyth, in conjunction with AutonomouStuff, offered students a unique opportunity to program and test elements used in an actual autonomous vehicle.

The class, Autonomous Vehicles in AI (CS 598), was the result of collaboration between Forsyth and Bobby Hambrick, CEO for AutonomouStuff, a Morton, Ill., based company that supplies research platforms for autonomous vehicles around the world.

“The course was an impulse,” admitted Forsyth. “Bobby wanted to form a relationship with us and I felt that easiest way to do that was get a batch of students to mess around with their equipment and see what happens.”

The course, taken by some 35 students, included the usual mixture of lectures and reading, but a major component was writing software to be used on AutonomouStuff vehicles. The projects used a variety of standard technologies, including Yolo, a state-of-the-art real-time object detection system.

One group programmed a car to detect the distance to a stop sign from a video inside the vehicle then stop it in the right place.  “That’s harder than it sounds because using just one picture, it is difficult to determine the distance,” explained Forsyth. “It normally takes at least two images.”

For another project, students built a detective framework that could detect objects in real time from video on the car’s graphics processing unit.

Another student group used that framework to build a pedestrian detector, which when connected to the brakes of the car, would break autonomously if someone walked in front of it.

“For an old guy with a background in computer vision from the early 80s, back then nothing worked, so attaching them to motor cars was quite honestly a silly thing to do,” Forsyth said. “However, over the last 15 years or so, there have been absolute revolutions in computer vision, which are solving a lot of problems that were unsolvable 30 years ago. Now we’re very good at detecting objects in images and video and classifying them. What that means is we can start figuring out how to use these as tools. A natural tool is pedestrian detector.”

AutonomouStuff invested heavily in time and its resources of autonomous vehicles to make the class happen, but in Forsyth’s estimation, the experiment was a big success.

“I think the relationship for Engineering at Illinois to be connected to AutonomouStuff is so natural,” Forsyth said. “Illinois has some immensely strong students and we’re hoping that some of the technologies developed in class are going to help them in the internal development processes, either as proof-of-concept or guidelines.”

Forsyth’s team is just one of a handful of similar projects coming out of University of Illinois research labs. Tim Bretl, an associate professor in aerospace engineering, for example, is working on autonomous cars in  “smart” agriculture through a grant from the NSF’s National Robotics Initiative. Shubhankar Agarwal, an undergraduate in computer engineering, is part of a student team that is building controls for autonomous submarines.

While Forsyth couldn’t have imagined what was possible 30 years ago, he hesitates to make bold predictions about the future of autonomous vehicles.

“I don’t think we will see large number of autonomous cars anytime soon,” he surmised. “What I believe we are going to see are cars that are safer and easier to drive because of neat add-ons. They could regulate your speed more effectively; have better detection technology (such as spotting impaired drivers or pedestrians), and better avoidance technology. Hopefully that means fewer serious injuries and mortalities from motor cars.”

So, how did it feel standing in front of the moving driverless vehicle?

“Joe Buckner (director of engineering at AutonomouStuff) had both hands out the window, but his foot one millimeter from the brake,” Forsyth recalled. “While we were able to control the risk, it had its moments. In the last 20 frames, I was getting twitchy hoping that Joe was awake. In the end, it stopped closer to me than I realized was safely possible for a vehicle running student code.”