| Pacific Time |
Event |
| 09:00 – 09:10 |
Opening Remarks |
| 09:10 – 09:55 |
Keynote 1 - Text Toys and Glitch Poetics - Lynn Cherny [link] |
|
Not just a bug, the glitch offers creative possibility -- especially in AI systems where we are travelers in a foggy latent space. The glitch is usually a visual metaphor, but it is alive and well in text encodings too. I'll talk about projects (mine and others') that explore neural spaces in poetic and game-like ways. Focusing on text play in this talk, we'll visit media collages, mistaken translations, cross-modal cutups, and the dusty bottoms of game databases in search of the uncanny glitch that make us laugh because it's true. |
| 10:00 – 10:30 |
Break |
| 10:30 – 11:15 |
Keynote 2 - Knowledge Intensive Reinforcement Learning - Tim Rocktäschel [link] |
|
Progress in Reinforcement Learning (RL) methods goes hand-in-hand with the development of challenging environments that test the limits of current approaches. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both these things. Moreover, research in RL has predominantly focused on environments that can be approached tabula rasa, i.e., without agents requiring transfer of any domain or world knowledge outside of the simulated environment. I will talk about the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for research based on the popular single-player terminal-based rogue-like game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. Interestingly, this game is extremely challenging even for human players who often need many years to solve it the first time and who generally consult external natural language knowledge sources like the NetHack Wiki to improve their skills. I will cover some of our recent work on utilizing language information in this challenging environment. |
| 11:15 – 12:00 |
Keynote 3 - Training Agents to Learn to Ask for Help in Virtual Environments - Hal Daumé III [link] |
|
"Agent" has largely become synonymous with "autonomous agent", but I'll argue that scoping our study of agents to those that are fully autonomous is a mistake: instead, we should aim to train agents that can assist humans, and be assisted by humans. In line with this goal, I will describe recent and ongoing work in the space of assisted agent navigation, where agents can ask humans for help, and where they can describe their own behaviors. This talk will largely be based on joint work with Sudha Rao, Khanh Nguyen, Lingjun Zhao, and Yonatan Bisk. |
| 12:00 – 13:30 |
Lunch Break |
| 13:30 – 14:15 |
Keynote 4 - Controllable Text Generation for Interactive Virtual Environments - Shrimai Prabhumoye [link] |
|
Since the dawn of the digital age, interactive virtual environments and electronic games have played a huge role in shaping our lives. Not only are they a source of entertainment but they also teach us important life skills such as strategic planning, collaboration, and problem solving. Therefore, online gamers expect their virtual environment to be aware of their situation (e.g., position in a game) and interact with them in natural language. In this talk, I describe novel techniques to generate text in a particular style. This talk provides an approach of generating engaging naturalistic conversation responses using knowledge generated by pre-trained language models, considering their recent success in a multitude of NLP tasks. The talk will conclude with exploring whether pretrained language models can be situated in these virtual spaces and generate dialogue in a zero-shot manner. |
| 14:15 – 15:00 |
Keynote 5 - ScienceWorld: Is your Agent Smarter than a 5th Grader? - Peter Jansen [link] |
|
Question answering models have rapidly increased their ability to answer natural language questions in recent years, due in large part to large pre-trained neural network models called Language Models. These language models have felled many benchmarks, including recently achieving an "A" grade on answering standardized multiple choice elementary science exams. But how much do these language models truly know about elementary science, and how robust is their knowledge? In this work, we present ScienceWorld, a new benchmark to test agents' scientific reasoning abilities. ScienceWorld is an interactive text game environment that tasks agents with performing 30 tasks drawn from the elementary science curriculum, like melting ice, building simple electrical circuits, using pollinators to help grow fruits, or understanding dominant versus recessive genetic traits. We show that current state-of-the-art language models that can easily answer elementary science questions, such as whether a metal fork is conductive or not, struggle when tasked to conduct an experiment to test this in a grounded, interactive environment, even with substantial training data. This presents the question of whether current models are simply retrieving answers to questions by way of observing a large number of similar input examples, or if they have learned to reason about concepts in a reusable manner. We hypothesize that agents need to be grounded in interactive environments to achieve such reasoning abilities. Our experiments provide empirical evidence supporting this hypothesis -- showing that a 1.5 million parameter agent trained interactively for 100k steps outperforms an 11 billion parameter model statically trained for scientific question answering and reasoning via millions of expert demonstrations. |
| 15:00 – 15:05 |
Closing Remarks |
| 15:05 – 15:30 |
Break |
| 15:30 – 16:30 |
Virtual Poster Session on GatherTown - All posters |
|
| Friday July 15th 15:30 – 16:30 |
Joint In-person Poster Session with Narrative Understanding workshop - Regency ballroom on the 7th |