Ji Receives $12.3M DARPA Grant to Develop Second-Generation Event Understanding System

Using artificial intelligence to predict future events still seems a bit like something from science fiction. But that’s one of the goals of the Defense Advanced Research Project Agency’s (DARPA’s) Knowledge-directed Artificial Intelligence Reasoning Over Schemas (KAIROS) program, which recently awarded $12.3 million over four years to a team of researchers led by Illinois Computer Science Professor Heng Ji.

The team’s project, named RESIN—Reasoning about Event Schemas for Induction of kNowledge, seeks to create a framework for the next generation of event understanding systems, with an ambitious goal: being able to provide a comprehensive understanding of evolving situations, events, and trends.

To accomplish this, Ji will work alongside leading experts in natural language processing, computer vision, machine learning, and knowledge representation and reasoning, including Illinois CS Professor Jiawei Han, who is a co-PI on the grant.  Additional team members include professors Mohit Bansal, University of North Carolina Chapel Hill; Chris Callison-Burch, University of Pennsylvania; Shih-Fu Chang, Columbia University; Martha Palmer, University of Colorado at Boulder; Dan Roth, University of Pennsylvania; and Carl Vondrick, Columbia University; and Mohammed Zaki, Rensselaer Polytechnic Institute.

In attempting to predict the future, where do you begin?  As it turns out, an understanding of past events and their relationships is helpful in identifying and classifying current events. Events can be placed into schemas—units of knowledge used to organize events into commonly occurring narrative structures. In combination with background knowledge about the actors, location, and history, schemas can serve as a scaffolding to reason about unfolding events and to make predictions.

Thanks to progress in natural language understanding and computer vision, some parts of event understanding are already automated.  But current systems are limited, sequential, and flat. For example, most event extraction methods only focus on events in isolation; there is not a systematic approach to build sound universal schemas for complex events across data modalities and languages, and temporal event reasoning and prediction is limited to constrained situations. 

These are just a few of the challenges that the RESIN team plans to address as they look to develop a second-generation event understanding framework that is capable of producing long-term forecasts of complex events.  “I’m excited to collaborate with a world-class team to open up new research directions and do ground-breaking research on this important problem,” says Ji.

Beyond national security, Ji sees potential broader impacts for this research on fields like journalism.  “Our system will be able to automatically summarize what’s happening right now, what will happen next, and generate a news timeline for any particular situation.”