Invited Talks
The Theory and Practice of Compositional Distributed Semantics A combination of compositional and distributional/distributed representations has many potential advantages for computational semantics. From the distributed side: robustness, learnability from data, ease of handling ambiguity, and the ability to represent gradations of meaning. From the compositional side: the ability to handle the unbounded nature of natural language, and the existence of established accounts of semantic phenomena such as logical words, quantification and inference. The development of such a combination has many challenges. The first half of the talk will describe a complete mathematical framework for deriving distributed representations compositionally using Combinatory Categorial Grammar (CCG). The tensor-based framework extends that of Coecke et al., which was previously applied to pregroup grammars, and is based on the observation that tensors are functions (multi-linear maps) and hence can be manipulated by the combinators of CCG, including type-raising and composition. The existence of robust, broad-coverage CCG parsers opens up the possibility of applying the tensor-based framework to naturally occurring text. The second half will describe our ongoing efforts to implement the framework, for which there are considerable practical challenges. I will describe some of the sentence spaces we are exploring; some of the datasets we are developing; and some of the machine learning techniques we are using in an attempt to learn the values of the tensors from corpus data. This work is being carried out with Luana Fagarasan, Douwe Kiela, Jean Maillard, Tamara Polajnar, Laura Rimell, Eva Maria Vecchi, and involves collaborations with Mehrnoosh Sadrzadeh (Queen Mary) and Ed Grefenstette and Bob Coecke (Oxford). Bio: Stephen Clark is Reader in Natural Language Processing at the University of Cambridge. Previously he was a member of Faculty at the University of Oxford and a postdoctoral researcher at the University of Edinburgh. He holds a PhD in Computer Science and Artificial Intelligence from the University of Sussex and a Philosophy degree from Cambridge. His main research interest is the development of data-driven models for the syntactic and semantic analysis of natural language. He holds a 1M 5-year ERC Starting Grant to work on integrating distributional and compositional models of meaning, as well as coordinating a 1.5M 5-site EPSRC grant in this area. |
Talk to me in plain English, please! Explorations in Data-driven Text Simplification Recent years have witnessed increased interest in data-driven methods for text rewriting, e.g., the problem of reformulating a query to alternative queries, a document in a simpler style, or a sentence in more concise manner. In this talk I will focus on text simplification, one of the oldest and best-studied rewriting problems. The popularity of the simplification task stems from its potential relevance to various applications. Examples include the development of reading aids for people with aphasia, non-native speakers and more generally individuals with low literacy. A simplification component could be also used as a preprocessing step to improve the performance of parsers, summarizers and semantic role labelers. In this talk I will discuss the challenges involved in text simplification and describe recent progress in leveraging large-scale corpora for model training. I will then present a model that simplifies documents automatically based on quasi-synchronous grammar, a formalism that can naturally capture structural mismatches and complex rewrite operations. I will explain how such a grammar can be induced from Wikipedia (and its Simple English variant) and introduce an integer linear programming model for selecting the most appropriate simplification from the space of possible rewrites generated by the grammar. Finally, I will present experimental results on simplifying Wikipedia articles showing that this approach significantly reduces reading difficulty, while producing grammatical and meaningful output. Joint work with Kristian Woodsend Bio: Mirella Lapata is a Professor at the School of Informatics at the University of Edinburgh. Her research focuses on probabilistic learning techniques for natural language understanding and generation. She serves as an associate editor of the Journal of Artificial Intelligence Research (JAIR) and is an action editor for Transactions of the Association for Computational Linguistics (TACL). She is the first recipient (2009) of the British Computer Society and Information Retrieval Specialist Group (BCS/IRSG) Karen Sparck Jones award. She has also received best paper awards in leading NLP conferences and financial support from the EPSRC (the UK Engineering and Physical Sciences Research Council) |
Towards Universal Syntactic and Semantic Processing of Natural Language I will first describe some efforts towards a universal representation of morphology and syntax that makes it possible to model language phenomena across language boundaries in a consistent way. I will then highlight several examples of how we have successfully used syntax at Google to improve downstream applications like question answering and machine translation. Finally, I will outline some ongoing and future work at the interface of syntactic and semantic processing. This work is carried out at Google by teams in New York, Mountain View and London. Bio: Slav Petrov is a researcher in Google's New York office, leading a team that works on syntactic parsing and its applications to information extraction, question answering and machine translation. He holds a PhD degree from UC Berkeley, where he worked with Dan Klein. Before that he completed a Master's degree at the Free University of Berlin and was a member of the FU-Fighters team that won the RoboCup world championship in 2004. His work on fast and accurate multilingual syntactic analysis has been recognized with best paper awards at ACL 2011 and NAACL 2012. Slav also teaches Statistical Natural Language Processing at New York University. |
