Publications

Scaling behavior for motion forecasting and planning models in autonomous driving.

Multi-modality scene tokenization for motion prediction.

A raw sensor benchmark for motion forecasting.

Distillation methods for scaling motion forecasting models.

Multi-agent motion forecasting as language modeling.

Efficient attention architecture for motion forecasting.

Distillation for efficient and accurate scene-centric motion forecasting.

Soft prompt transfer for better adaptation of frozen models.

Token-free byte-to-byte language modeling at scale.

Prompt tuning improves substantially with model scale.

Parallel data and pre-training dynamics for massively multilingual LMs.

We present a system for bilingual Data-To-Text Generation and Semantic Parsing. We use a text-to-text generator to learn a single model …

Prior work on Data-To-Text Generation, the task of converting knowledge graph (KG) triples into natural text, focused on …

The recent “Text-to-Text Transfer Transformer” (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. We describe the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks. All of the code and model checkpoints used in this work are publicly available.

We propose a new multilingual language model benchmark that is composed of 40+ languages spanning several scripts and linguistic …

We present LAReQA, a challenging new benchmark for language-agnostic answer retrieval from a multilingual candidate pool. Unlike …

Purely character-based language models (LMs) have been lagging in quality on large scale datasets, and current state-of-the-art LMs …

LSTMs and other RNN variants have shown strong performance on character-level language modeling. These models are typically trained …

Can neural networks learn to compare graphs without feature engineering? In this paper, we show that it is possible to learn …

Graph embedding methods represent nodes in a continuous vector space, preserving different types of relational information from the …

This paper presents a computationally efficient machine-learned method for natural language response suggestion. Feed-forward neural …

We propose a new method for embedding graphs while preserving directed edge information. Learning such continuous-space vector …

We investigate the task of modeling open-domain, multi-turn, unstructured, multi-participant, conversational dialogue. We specifically …

We propose a new computational approach for tracking and detecting statistically significant linguistic shifts in the meaning and usage …

We present DeepWalk, a novel approach for learning latent representations of vertices in a network. These latent representations encode …