---

Schedule
Content
Examination
Assignments
Reading

---

Language Technology: Research and Development

Note that this page has been migrated from a previous server. There is thus a risk that not all links work correctly

Credits: 15 hp
Syllabus: 5LN714
Teachers: Sara Stymne, Mats Dahllöf, Joakim Nivre
Course coordinator and examiner: Sara Stymne

News

• 200108: The schedule for the final presentations are now available.
• 190925: The deadline for the written proposals is changed from October 4 to October 7.
• 190920: Note that there are schedule changes. The seminars of October 2 have been moved. The seminars on October 8 have been cancelled and the semianrs on October 9 now strats at 9.15.
• 190906: The research groups, including presenters of research papers are now available!

Schedule

	Date	Time	Room	Content	Reading
L1	4/9	10-12	2-1077	Introduction Introduction, Universal Dependencies, Multilinguality Author id and profiling
L2	5/9	10-12	Turing	Science and research	Okasha
L3	9/9	13-15	22-0031	Language technology research and development	Cunningham, Lee
S1	13/9	10-12	2-0025, 2-0026, 2-0028	Seminar - research papers
S2	24/9	14-16	3-0012, 9-3068, 16-2044	Seminar - research papers
L4	25/9	10-12	6-K1031	R&D projects - from proposal to implementation	Zobel 10-11, 13
S2	27/9	9/11	UD: 9-2029	Seminar - research papers
S3	27/9	15-17	aip: 7-0015	Seminar - research papers
S3	2/10	10-12	22-1005, 22-1006, 22-1008	Seminar - research papers
S3	3/10	10-12	mling: 9-1016	Seminar - research papers
S3	3/10	15-17	UD: 9-3068	Seminar - research papers
S4a	8/10	10-12	9-2029, 22-1005, 22-1008	Seminar - project proposals
S4b	9/10	9-12	22-1005, 22-1008, 22-1009	Seminar - project proposals (Note: starts at 9.15!)
S5	23/10	10-12	2-0023, 2-0026, 22-0025	Seminar - progress report
L5	6/11	10-12	16-2043	Dissemination of research results	Zobel 1-9, 14
S6	13/11	10-12	2-0023, 2-0025, 2-0026	Seminar - progress report
S7	27/11	10-12	2-0025, 2-0026, 2-0028	Seminar - progress report
L6	27/11	13-15	Turing	Review of scientific articles	Zobel 14
S8	11/12	10-12	2-0025, 2-0026, 2-0028	backup Seminar - progress report
S9	8/1	10-12	2-0025, 2-0026, 2-0027	backup Seminar - progress report
FS	15/1	9-16	7-0042 and 7-0043	Seminar - term paper presentations
	15/1	16-	Department staff room	Social event

All lectures will be given by Sara. The seminars will be led by the seminar leader for each research group. Unless otherwise stated for the seminars, group 1 (UD) will be in the first room, group 2 (mling) in the second room, and group 3 (aip) in the third room. Note that attendance is obligatory at all seminars. If you miss one, contact your seminar leader to discuss how to compensate.

Content

The course gives a theoretical and practical introduction to research and development in language technology. The theoretical part covers basic philosophy of science, research methods in language technology, project planning, and writing and reviewing of scientific papers. The practical part consists of a small project within a research area common to a subgroup of course participants, including a state-of-the-art survey in a reading group, the planning and implementation of a research task, and the writing of a paper according to the standards for scientific publications in language technology. The research areas, with teachers, for 2019 are:

1. Universal Dependencies (UD) - Joakim Nivre
2. Multilinguality (mling) - Sara Stymne
3. Author identification and profiling (aip) - Mats Dahllöf

Research Groups

Below groups and articles for the research seminars will appear.

Groups	Members	Papers
1: Universal Dependencies	Ivan	Sep 13: Nivre. (2015)
	Rex	Sep 13: Nivre et al. 2016
	Shorouq	Sep 13: Nivre et al. 2018
	Artur	Sep 24: Reddy et al. 2017
	Giuseppe	Sep 24: Straka et al. 2016
	Tiantian	Sep 24: Tiedemann 2015
	Marsida	Oct 2: Berzak et al. 2016
	(Joakim)	Oct 2: Dyer 2018
	Camilla	Oct 2: Futrell et al. 2015
2: Multilinguality	(Sara)	Sep 13: Yarowsky et al. 2001
	George	Sep 13: Tiedemann 2015
	Elsa	Sep 13: Smith et al. 2018
	Natalia	Sep 24: Plank & Agić 2018
	Xiajing	Sep 24: Östling & Tiedemann 2017
	Arra'di Nur	Sep 24: Glavaš et al. 2019
	Jun	Oct 2: Tiedemann 2012
	Hartger	Oct 2: Zoph et al. 2016
	Ugo	Oct 2: Lin et al. 2019
3: Author identification and profiling	Drew	Sep 13: Kestemont et al. 2018
	Shifei	Sep 13: Rocha et al. 2017
	Luise	Sep 24: Karmen et al. 2015
	Xuemei	Sep 24: Park et al. 2015
	Evangelia	Sep 24: Peersman et al. 2011
	Anne Fleur	Oct 2: Overdorf & Greenstadt 2016
	Adam	Oct 2: Potthast et al. 2017
	Renfei	Oct 2: Preoţiuc-Pietro et al. 2015

Examination

The course is examined by means of five assignments with different weights (see below). In order to pass the course, a student must pass each of one of these. In order to pass the course with distinction, a student must pass at least 50% of the weighted graded assignments with distinction.

Assignments

1. Take home exam on philosophy of science (15%)
   • This assignment will be based on your reading of Okasha's book. You will be asked to discuss issues in the philosophy of science and (sometimes) relate them to the area of language technology. The questions will be handed out September 13, and the report should be handed in September 20.
2. Research paper presentation and discussion (15%)
   • You will present one of the papers discussed in the seminars. The task is to introduce the paper and lead the discussion, not to make a formal presentation. In addition you shall take active part in the discussion of all other papers discussed in the seminars. The seminars thus have obligatory attendance; if you miss a seminar, you have to write a short report instead. This assignment is not graded and does not qualify for distinction.
3. Project proposal (15%)
   • You will put together a 3-page proposal describing the project you are going to work on for the rest of the course, using an adapted version of the Swedish Research Council's guidelines for research plans. You will also give a short presentation of the proposal in a seminar (10 minutes with slides). The deadline for the written proposal is October 4, and the seminars will take place October 9.
4. Review of term papers (15%)
   • You will review two term papers written by your course mates using the guidelines of Transactions of the Association for Computational Linguistics. You will receive the papers on December 16 and the reviews are due December 20.
5. Term paper (40%)
   • You will report your project in a paper following the guidelines of Transactions of the Association for Computational Linguistics (except that the page limit for your papers is 4-7 pages + references). The deadline is December 13 for the first version and January 17 for the revised version. On January 15, you will also give an oral presentation of the paper. As part of your work on the project, it is obligatory to attend the progress report seminars. If you miss a seminar, you have to write a short report instead.

Submitting and Reviewing Term papers

Information to appear!

Final Seminar/Workshop

The final seminar will be organized as a workshop with term paper presentations.

Research Groups

Your first task in the course is to make a wish for which research topic to work on. Send a ranked list of your preference for the three topics by email to Sara, at the latest Friday September 6, at 13.00. If you fail to make a wish by this deadline you will be arbitrarily assigned to a topic. We will try our best to respect everyone's wishes, but if it turns out not to be possible, we will resort to random decisions.

Deadlines

Here is a summary of all deadlines in the course.

Task	Deadline	Extra deadline
Choose your preferred topics	September 6, 13:00	-
Hand in take home exam	September 20	November 15
Project proposal	October 4	November 8
Present project proposal	October 8-9	By agreement
First version of project report	December 13	January 17
Reviews on peer's project papers	December 20	February 21
Final seminar	January 15	By agreement
Final project report	January 17	February 21

Note that it is important for you to finish the course on time, since it is a requirement for starting your master thesis. So try to avoid resorting to the backup deadlines!

Reading

Science and Research

• Cunningham, H. (1999) A definition and short history of Language Engineering. Natural Language Engineering 5 (1), 1-16.
• Hovy, D. and Spruit, S. L. (2016) The Social Impact of Natural Language Processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 591-598.
• Lee, L. (2004) "I'm sorry Dave, I'm afraid I can't do that": Linguistics, Statistics, and Natural Language Processing circa 2001. In Computer Science: Reflections on the Field, Reflections from the Field, 111-118.
• Okasha, S. (2002) Philosophy of Science: A Very Short Introduction. Oxford University Press. Chapters 1-3 and 5.
• Zobel, J. (2004) Writing for Computer Science. Second Edition. Springer.

Universal Dependencies

• Berzak, Y., Kenney, J., Spadine, C., Wang, J.-X., Lam, L., Mori, K.S., Garza, S. and Katz, B. (2016) Universal Dependencies for Learner English. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 737-746. [Sem 3]
• De Marneffe, M.-C., Dozat, T., Silveira, N., Haverinen, K., Ginter, F., Nivre, J. and Manning, C.D. (2014) Universal Stanford Dependencies: A Cross-Linguistic Typology In Proceedings of the Ninth International Conference on Language Resources and Evaluation, 4585-4592.
• Dozat, T., Qi, P. and Manning, C. D. (2017) Stanford's Graph-based Neural Dependency Parser at the CoNLL 2017 Shared Task. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, 20-30
• Dyer, W. (2018) Integration Complexity and the Order of Cosisters. In Proceedings of the Universal Dependencies Workshop [Sem 3]
• Futrell, R., Mahowald, K. and Gibson, E. (2015) Quantifying Word Order Freedom in Dependency Corpora. In Proceedings of the Third International Conference on Dependency Linguistics, 91–100. [Sem 3]
• Levshina, N., (2017) Does Syntactic Informativity Predict Word Length? A Cross-linguistic Study Based on the Universal Dependencies Corpora In Proceedings of the NoDaLiDa Workshop on Universal Dependencies (UDW 2017).
• McDonald, R., Nivre, J., Quirmbach-Brundage, Y., Goldberg, Y., Das, D., Ganchev, K., Hall, K., Petrov, S., Zhang, H., Täckström, O., Bedini, C., Bertomeu Castelló, N. and Lee, J. (2013) Universal Dependency Annotation for Multilingual Parsing. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 92-97.
• McDonald, R., Petrov, S. and Hall, K. (2011) Multi-Source Transfer of Delexicalized Dependency Parsers. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 62-72.
• Naseem, T., Chen, H., Barzilay, R. and Johnson, M. (2010) Using Universal Linguistic Knowledge to Guide Grammar Induction. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, 1234-1244.
• Nivre, J. (2015) Towards a Universal Grammar of Natural Language Processing. In Computational Linguistics and Intelligent Text Processing, 3-16. [Sem 1]
• Nivre, J., de Marneffe, M.-C., Ginter, F., Goldberg, Y., Hajic, J., Manning, C.D., McDonald, R., Petrov, S., Pyysalo, S., Silveira, N., Tsarfaty, R. and Zeman, D. (2016) Universal Dependencies v1: A Multilingual Treebank Collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016) [Sem 1].
• Nivre, J. and Fang, C. (2017) Universal Dependency Evaluation In Proceedings of the NoDaLiDa Workshop on Universal Dependencies (UDW 2017).
• Nivre, J., Marongiu, P., Ginter, F., Kanerva, J., Montemagni, S., Schuster, S. and Simi, M. (2018) Enhancing Universal Dependency Treebanks: A Case Study. In Proceedings of the Universal Dependencies Workshop (UDW). [Sem 1]
Joakim Nivre, Paola Marongiu, Filip Ginter, Jenna Kanerva, Simonetta Montemagni, Sebastian Schuster and Maria Simi
• Östling, R. (2015) Word Order Typology through Multilingual Word Alignment. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers), 205–211.
• Reddy, S., Täckström, O., Petrov, S., Steedman, M. and Lapata, M. (2017) Universal Semantic Parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 89-101. [Sem 2]
• Straka, M., Hajic, J. and Straková, J. (2016) UDPipe: Trainable Pipeline for Processing CoNLL-U Files Performing Tokenization, Morphological Analysis, POS Tagging and Parsing In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016) [Sem 2].
• Swanson, B. and Charniak, E. (2014) Data Driven Language Transfer Hypotheses. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, 169–173.
• Tiedemann, J. (2015) Cross-Lingual Dependency Parsing with Universal Dependencies and Predicted PoS Labels. In Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), 340--349. [Sem 2]
• Waleed, A., Mulcaire, G., Ballesteros, M., Dyer, C., Smith and N. (2016) Many Languages, One Parser. Transactions of the Association for Computational Linguistics 4, 431-444.
• Wisniewski, G. and Lacroix, O. (2017) A Systematic Comparison of Syntactic Representations of Dependency Parsing In Proceedings of the NoDaLiDa Workshop on Universal Dependencies (UDW 2017).
• Zeman, D., Marecek, D., Popel, M., Ramasamy, L., Stepánek, J., Zabokrtský, Z., Hajic, J. (2012) HamleDT: To Parse or Not to Parse?. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, 2735-2741.
• Zeman, D. (2017) Core Arguments in Universal Dependencies. In Proceedings of the Fourth International Conference on Dependency Linguistics, 287-296.

See also proceedings from the Universal Dependencies Workshops (UDW):

• UDW 2017
• UDW 2018
• UDW 2019

Multilinguality

• Wasi Uddin Ahmad, Zhisong Zhang, Xuezhe Ma, Eduard Hovy, Kai-Wei Chang, and Nanyun Peng. (2019) On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing. In arXiv preprint, arXiv:1811.00570v3
• Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah Smith. (2016) Many languages, one parser.. In TACL, 4:431.444.
• Lauriane Aufrant. (2018) Training parsers for low-resourced languages: improving cross-lingual transfer with monolingual knowledge. PhD thesis, Paris Saclay.
• A. Conneau, G. Lample, L. Denoyer, MA. Ranzato, and H. Jégou. (2017) Word Translation Without Parallel Data. arXiv preprint arXiv:1710.04087
• Goran Glavaš, Robert Litschko, Sebastian Ruder, and Ivan Vulić. (2019) How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 710-721.[Sem 2]
• Rob van der Goot, Nikola Ljubešić, Ian Matroos, Malvina Nissim, and Barbara Plank. (2018) Bleaching Text: Abstract Features for Cross-lingual Gender Prediction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 383-389.
• Jiang Guo, Wanxiang Che, Haifeng Wang, and Ting Liu. (2016) A universal framework for inductive transfer parsing across multi-typed treebanks.. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 12-22.
• Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. (2017) Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. In TACL, 5:339-351.
• Miryam de Lhoneux, Johannes Bjerva, Isabelle Augenstein, and Anders Søgaard. (2018) Parameter sharing between dependency parsers for related languages. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 4992-4997.
• Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. (2019) Choosing Transfer Languages for Cross-Lingual Learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3125-3135.[Sem 3]
• Stephen Mayhew, Chen-Tse Tsai and Dan Roth. (2017) Cheap Translation for Cross-Lingual Named Entity Recognition. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2536-2545.
• Phoebe Mulcaire, Swabha Swayamdipta, and Noah A. Smith. (2018) Polyglot Semantic Role Labeling. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Short Papers), 667-672.
• Tahira Naseem, Regina Barzilay, and Amir Globerson. (2012) Selective sharing for multilingual dependency parsin. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, 629-637.
• Robert Östling and Jörg Tiedemann. (2017) Continuous multilinguality with language vectors. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, 644-649.[Sem 2]
• Barbara Plank and Željko Agić. (2018) Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 614-620.[Sem 2]
• Edoardo Maria Ponti, Roi Reichart, Anna Korhonen, and Ivan Vulič. (2018) Isomorphic Transfer of Syntactic Structures in Cross-Lingual NLP. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1531-1542.
• Aaron Smith, Bernd Bohnet, Miryam de Lhoneux, Joakim Nivre, Yan Shao, and Sara Stymne. (2018) 82 Treebanks, 34 Models: Universal Dependency Parsing with Multi-Treebank Model. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, 113-123.[Sem 1]
• Jörg Tiedemann. (2012) Character-Based Pivot Translation for Under-Resourced Languages and Domains. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, 141-151.[Sem 3]
• Jörg Tiedemann. (2015) Cross-Lingual Dependency Parsing with Universal Dependencies and Predicted PoS Labels. In Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), 340--349.[Sem 1]
• David Yarowsky, Grace Ngai, and Richard Wicentowski. (2001) Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the first international conference on Human language technology research, 1-8.[Sem 1]
• Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. (2018) CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, 1-22.
• Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. (2016) Transfer Learning for Low-Resource Neural Machine Translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1568-1575.[Sem 3]
• Yanyan Zou and Wei Lu. (2018) Learning Cross-lingual Distributed Logical Representations for Semantic Parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 673-679.

Author Identification and Profiling

• Abbasi, M., and Liu, H. (2013) Measuring user credibility in social media. In SBP'13 Proceedings of the 6th international conference on Social Computing, Behavioral-Cultural Modeling and Prediction, 441-448.
• Cutler, A. and Kulis, B. (2018) Inferring Human Traits from Facebook Statuses. In Social Informatics, Staab, S., Koltsova, O., and Ignatov, D.I. (Eds.), 167–195.
• Havigerová, J.M. Haviger, J., Kučera, D., and Hoffmannová, P. (2019) Text-Based Detection of the Risk of Depression. Frontiers in Psychology.
• Hinds, J., and Joinson, A.N. (2018) What demographic attributes do our digital footprints reveal? A systematic review. PLoS ONE 13(11).
• Karmen, C., Hsiung, R.C., and Wetter, T. (2015) Screening Internet forum participants for depression symptoms by assembling and enhancing multiple NLP methods. Computer Methods and Programs in Biomedicine 120(1), 27-36. [Sem 2]
• Mike Kestemont, M., Tschuggnall, M., Stamatatos, E., Daelemans, W., Specht, G., Stein, B., and Potthast, M. (2018) Overview of the Author Identification Task at PAN-2018 Cross-domain Authorship Attribution and Style Change Detection. In Working Notes of CLEF 2018 – Conference and Labs of the Evaluation Forum (PAN Lab on Digital Text Forensics). [Sem 1]
• Long, Y., Lu, Q., Xiang, R., Li, M., and Huang, C. (2017) Fake News Detection Through Multi-Perspective Speaker Profiles. In Proceedings of the The 8th International Joint Conference on Natural Language Processing, 252–256.
• Mishra, P., Del Tredici, M., Yannakoudakis, H., and Shutova, E. (2018) Author profiling for abuse detection. In Proceedings of the 27th International Conference on Computational Linguistics, 1088–1098.
• Overdorf, R., and Greenstadt, R. (2016) Blogs, Twitter Feeds, and Reddit Comments: Cross-domain Authorship Attribution. In Proceedings on Privacy Enhancing Technologies 2016 (3), 155–171. [Sem 3]
• Park, G., Schwartz, H.A., Eichstaedt, J.C., Kern, M.L., Kosinski, M., Stillwell, D.J., Ungar, L.H., and Seligman, M.E.P. (2015) Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108(6), 934-952. [Sem 2]
• Peersman, C., Daelemans, W., and van Vaerenbergh, L. (2011) Predicting age and gender in online social networks. In 3rd International Workshop on Search and Mining User-generated Contents, SMUC '11, 37–44. [Sem 2]
• Potthast, M., Kiesel, J., Reinartz, K., Bevendorff, J., Stein, B. (2017) A Stylometric Inquiry into Hyperpartisan and Fake News. arXiv:1702.05638. [Sem 3]
• Preoţiuc-Pietro, D., Volkova, S., Lampos, V., Bachrach, Y., and Aletras, N. (2015) Studying User Income through Language, Behaviour and Affect in Social Media PLoS ONE 10(9), 37–44. [Sem 3]
• Rocha, A., Scheirer, W.J., Forstall, C.W., Cavalcante, T., Theophilo, A., Shen, B., Carvalho, A.R.B., and Stamatatos, E. (2017) Authorship attribution for social media forensics. IEEE Transactions on Information Forensics and Security, 12(1), 5-33. [Sem 1]
• Sundararajan, K. and Woodard, D.L. (2018) What constitutes "style" in authorship attribution?. In Proceedings of the 27th International Conference on Computational Linguistics, 2814–2822.