Elior Sulem - Learning with Less Data and Labels for Language Acquisition and Understanding

Natural Language Processing has attracted a lot of interest in recent years and has seen large improvements with the use of contextualized neural language models. However, the progress is limited to specific tasks where large datasets are available and models are often brittle outside of these datasets. Also, current models are usually pretrained on extremely large unlabeled datasets, which limits our understanding of low-resource scenarios and makes their use unfeasible for many in the academia and industry. On the other hand, children learn with much less data, which makes the study of child language acquisition an appealing research direction to address these issues. In the first part of the talk, I will focus on pretraining on unlabeled data and show what can be learned with an input both quantitatively and qualitatively comparable to that of an average English-speaking 6 year old. In the second part of the talk I will focus on Natural Language Understanding and show how zero-shot and transfer learning strategies can be used to go beyond task-specific training. In particular, I will present a zero-shot approach to Event extraction, which is usually based on the annotation of large domain-specific corpora, using Textual Entailments (TE) and Question-Answering (QA) tasks. I will show the challenge of missing arguments in this context, presenting new models that identify unanswerable questions,  leveraging different QA and TE tasks.