ML@GT Spring Seminar: Dipendra Misra, Cornell University

The Machine Learning Center at Georgia Tech invites you to attend a seminar featuring Dipendra Misra from Cornell University. The event will be held in Parker H. Petit Institute for Bioengineering and Biosciences at Georgia Tech from 2-3 p.m. on Friday, March 8. This event is open to the public.

Situated Natural Language Understanding

The goal of situated natural language understanding is to reason about language in a context. Examples include voice assistant applications (e.g., Siri, Alexa, Cortana), robots that follow instructions in the real world and dialogue bots that can book flight tickets. In this talk, I will present my work on the problem of instruction following: designing agents that follow natural language instructions by taking appropriate actions in an environment. While this problem has been studied extensively, previous approaches relied on expensive engineering that was difficult to scale.

In the first part of the talk, I will describe my work on learning a simple model that maps raw observations and instruction to actions. We train this model using a sample efficient contextual bandit learning algorithm. This allows us to learn to reason about language, complex visual cues, and action planning from a limited amount of natural language data. In the second part, I will describe recent work on designing agents that perform explicit spatial reasoning for navigation in 3D environments. We focus on the task of goal prediction and introduce LingUNET, a language conditioned image-to-image generation architecture. We use LingUNET in a model that separates goal prediction from action generation. This simplifies training and allows to examine the model decisions for better interpretability and safety.

Dipendra Misra is a Ph.D. candidate at Cornell University. His research focuses on developing models and learning algorithms for problems in natural language understanding.  His work spans instruction following, semantic parsing, question answering, and reinforcement learning theory. He is a member of the CLIC (Cornell Language in Context Lab) group and is advised by Yoav Artzi. He holds a bachelor’s degree from Indian Institute of Technology, Kanpur where he was an OPJEMS scholar. He is active in the program committees of natural language understanding and machine learning. In 2018, he co-organized the third workshop on representation learning for natural language processing (Rep4NLP) at ACL.

Event Details


  • Friday, March 8, 2019
    2:00 pm - Saturday, March 9, 2019
    2:59 pm
Location: Pettit Microelectronics Research Building Room 102 A&B

Related Media

Click on image(s) to view larger version(s)

  • Dipendra Misra is a Ph.D. candidate from Cornell University and will be visiting Georgia Tech as a part of the Machine Learning Center's Spring Seminar Series.

For More Information Contact

Kyla Hanson

Program Manager