Syllabus

Date	Topic	Readings (starred=graduate level)
Aug 26	Introduction [Video] [Youtube]	Koehn (2010), Statistical Machine Translation, Chapter 1.2 Weaver (1949), Translation Kay, Translation
Aug 28	Basics in Language and Probability [Video] [Youtube]	Koehn (2010), Statistical Machine Translation, Chapter 2.1 and 3
Sep 02	Language Models [Video] [Youtube]	Koehn (2010), Statistical Machine Translation, Chapter 7
Sep 04	IBM Model 1 and the EM Algorithm [Video] [Youtube]	Koehn (2010), Statistical Machine Translation, Chapter 4.1-4.2 Knight (1997), Automating Knowledge Acquisition for Machine Translation
Sep 09	Phrase-Based Models and Decoding [Video] [Youtube] [Video 2] [Youtube 2] [Demo]	Koehn (2010), Statistical Machine Translation, Chapter 5, 6
Sep 11	Evaluation [Video] [Youtube]	Koehn (2010), Statistical Machine Translation, Chapter 8 Koehn (2020), Neural Machine Translation, Chapter 4 Graham et al. (2014), Is Machine Translation Getting Better over Time?
Sep 16	Introduction to Neural Networks [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 5 Makin (2006), Backpropagation LeCun et al. (1998), Efficient BackProp Srivastava et al. (2014), Dropout - A Simple Way to Prevent Neural Networks from Overfitting
Sep 18	Computation Graphs [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 6 Tensorflow, Neural network online demo
Sep 23	Neural Language Models [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 7 Bengio et al. (2002), A Neural Probabilistic Language Model Tomas Mikolov (2012), Statistical Language Models Based on Neural Networks, Chapter 3 Karpathy et al. (2015), Visualizing and Understanding Recurrent Networks
Sep 25	Neural Translation Models [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 8 Bahdanau et al. (2015), Neural Machine Translation by Jointly Learning to Align and Translate
Sep 30	Decoding in Neural Translation Models [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 9 Freitag et al. (2022), Minimum Bayes Risk Decoding with Neural Metrics Post and Vilar (2018), Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation
Oct 02	Machine Learning Tricks [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 10
Oct 07	Alternative Architectures [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 11 Vaswani et al. (2017), Attention Is All You Need Gehring et al. (2017), Convolutional Sequence to Sequence Learning
Oct 09	Beyond Parallel Data [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 14 Fan (2020), Introducing the First AI Model That Translates 100 Languages Without Relying on English
Oct 14	Analysis and Visualization [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 17
Oct 21	Linguistic Intermezzo [Video] [Youtube] [Video 2] [Youtube 2]	Koehn (2010), Statistical Machine Translation, Chapter 10.2 Koehn (2020), Neural Machine Translation, Chapter 12, 15 Cook et al. (2011), The World Color Survey Database - History and Use Banarescu et al. (2013), Abstract Meaning Representation for Sembanking
Oct 23	Large Language Models	Radford et al. (2019), Language Models are Unsupervised Multitask Learners Llama Team, Meta AI (2024), The Llama 3 Herd of Models Briakou et al. (2023), On the Role of Incidental Bilingualism in PaLM’s Translation Capability
Oct 28	Adaptation [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 13 Axelrod et al. (2011), Domain adaptation via pseudo in-domain data selection
Oct 30	Preference Training	Rafailov et al. (2023), Direct Preference Optimization - Your Language Model is Secretly a Reward Model Rei et al. (2023), Scaling up COMETKIWI
Nov 04	Speech Translation
Nov 06	Corpus Acquisition from the Internet [Video] [Youtube]	Uszkoreit et al. (2010), Large Scale Parallel Document Mining for Machine Translation Resnik and Smith (2003), The Web as a Parallel Corpus Buck and Heafield (2014), N-gram Counts and Language Models from the Common Crawl Koehn et al. (2018), Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering
Nov 11	Current Challenges [Video] [Youtube]	Koehn (2020), Neural Machine Translation, Chapter 16 Koehn and Knowles (2017), Six Challenges for Neural Machine Translation Khayrallah and Koehn (2018), On the Impact of Various Types of Noise on Neural Machine Translation
Nov 13	Computer Aided Translation [Video] [Youtube]	Barrachina et al. (2009), Statistical Approaches to Computer-Assisted Translation Knowles and Koehn (2017), Neural Interactive Machine Translation
Nov 18	No class - project work
Nov 20	No class - project work
Dec 02	Project Presentations (Hackerman Hall 2nd floor)
Dec 04	Project Presentations (Hackerman Hall 2nd floor)