Current Topics in Natural Language Processing (WS 2019-2020)

Summary

Deep Learning is an interesting new branch of machine learning where neural networks consisting of multiple layers have shown new generalization capabilities. The seminar will look at advances in both general deep learning approaches, and at the specific case of Neural Machine Translation (NMT). NMT is a new paradigm in data-driven machine translation. In Neural Machine Translation, the entire translation process is posed as an end-to-end supervised classification problem, where the training data is pairs of sentences and the full sequence to sequence task is handled in one model.

Here is a link to last semester's seminar.

There is a Munich interest group for Deep Learning, which has an associated mailing list, the paper announcements are sent out on this list. See the link here.

Instructors

Alexander Fraser

Email Address: SubstituteLastName@cis.uni-muenchen.de

CIS, LMU Munich


Hinrich Schütze

CIS, LMU Munich

Schedule

Thursdays 14:45 (s.t.), location is room 115

Click here for directions to CIS.

New attendees are welcome. Read the paper and bring a paper or electronic copy with you, you will need to refer to it during the discussion.

If this page appears to be out of date, use the refresh button of your browser

Date Paper Links Discussion Leader
Thursday, October 17th Gonzalo Mena (2017). The Gumbel-Softmax Trick for Inference of Discrete Variables. Blog (Columbia). blogpost Nina Pörner
Thursday, October 24th *at 2:15pm* BRIEF TALK AND DISCUSSION: NLP in Sri Lanka, Prof Gihan Dias (Moratuwa) and K. Sarveswaran (Konstanz)
Thursday, October 31st Sachin Kumar, Yulia Tsvetkov (2019). Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs. ICLR 2019 paper Benjamin Roth
Thursday, November 14th R. Thomas McCoy, Ellie Pavlick, Tal Linzen (2019). Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. ACL 2019
Also look at: Timothy Niven and Hung-Yu Kao (2019). Probing Neural Network Comprehension of Natural Language Arguments. ACL 2019
mccoy
niven
Alex Fraser
Thursday, November 21st Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis (2019). Generalization Through Memorization: Nearest Neighbor Language Models. arXiv 2019. paper Helmut Schmid
Thursday, November 28th Anonymous. You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings. ICLR 2020 submission. openreview Marina Speranskaya
Thursday, December 12th Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu (201). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv. paper
You can optionally skip 3.2 to 3.6 (or 3.1 to 3.6), not 3.7
Kerem Şenel
Thursday, December 19th Emma Strubell, Ananya Ganesh, Andrew McCallum (2019). Energy and Policy Considerations for Deep Learning in NLP. ACL 2019. paper Alex Fraser
Thursday, January 9th Steven C.Y. Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, Chu-Song Chen (2019). Compacting, Picking and Growing for Unforgetting Continual Learning. NeurIPS 2019. paper Mengjie Zhao
Thursday, January 16th (Some) Research Happening at Google Translate, talk by Markus Freitag
Thursday, January 23rd Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya (2020). Reformer: The Efficient Transformer. ICLR 2020 paper Nina Pörner
Thursday, January 30th Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu (2019). MASS: Masked Sequence to Sequence Pre-training for Language Generation. ICML 2019 paper Alexandra Chronopoulou
Thursday, February 6th Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, Adam Tauman Kalai (2019). What's in a Name? Reducing Bias in Bios without Access to Protected Attributes. NAACL 2019 (best paper) paper Ben Roth
Thursday, February 13th Eric Malmi, Sebastian Krause, Sascha Rothe, Daniil Mirylenka, Aliaksei Severyn (2019). Encode, Tag, Realize: High-Precision Text Editing. EMNLP 2019 paper Jindřich Libovický
Thursday, February 20th Parker Riley, Daniel Gildea (2018, 2020). Orthographic Features for Bilingual Lexicon Induction. Unsupervised Bilingual Lexicon Induction Across Writing Systems. ACL and arXiv. paper1 paper2 Silvia Severini
Thursday, February 27th Junxian He, Xinyi Wang, Graham Neubig, Taylor Berg-Kirkpatrick (2020). A Probabilistic Formulation of Unsupervised Text Style Transfer. ICLR 2020. paper Dario Stojanovski
Thursday, March 5th Stephen Merity (2019). Single Headed Attention RNN: Stop Thinking With Your Head. arXiv. paper Martin Schmitt


Further literature:

You can go back through the previous semesters by clicking on the link near the top of the page.