Συντάχθηκε 16-10-2014 11:50
από Esthir Gelasaki
Email συντάκτη: egelasaki<στο>tuc.gr
Ενημερώθηκε:
16-10-2014 13:50
Ιδιότητα: υπάλληλος.
ΠΟΛΥΤΕΧΝΕΙΟ ΚΡΗΤΗΣ
Σχολή Ηλεκτρονικών Μηχανικών και Μηχανικών Υπολογιστών
Πρόγραμμα Προπτυχιακών Σπουδών
ΠΑΡΟΥΣΙΑΣΗ ΔΙΠΛΩΜΑΤΙΚΗΣ ΕΡΓΑΣΙΑΣ
ΦΥΤΟΠΟΥΛΟΥ ΝΙΚΟΛΑΟΥ
με θέμα
Discriminative Training of Language Models
Παρασκευή 17 Οκτωβρίου 2014, 13.00
Αίθουσα 137.Π39, Κτίριο Επιστημών, Πολυτεχνειούπολη
Εξεταστική Επιτροπή
Καθηγητής Διγαλάκης Βασίλειος(επιβλέπων)
Αναπληρωτής Καθηγητής Λαγουδάκης Μιχαήλ
Δρ. Διακολουκάς Βασίλειος
Abstract
The present thesis implements MMI training on continuous Language Models. The main motivation for dealing with continuous language models was the fact that they overcome the drawbacks of N-gram based models. N-gram models have been widely used in Language Modeling, but suffer from lack of generalizability and contain a very large number of parameters that are hard to adapt. Another flaw of N-gram models is the need for a large amount of training data, in order to cover as many N-grams as possible. Continuous Gaussian Mixture Language Models (GMLMs) for Speech Recognition have proven to be effective in terms of smoothing unseen events and adapting efficiently while using relatively small amount of data when compared to N-gram models.
The language models being built consist of a vocabulary of 2700 words. The training and testing data were extracted from the Wall Street Journal Corpus. Data has the form of a continuous-space vector and consists of the history of each word in the corpus. The dimensions of these vectors were reduced by using SVD and LDA techniques.
As far as the main objective of the thesis is concerned, attempts focus on improving the performance of GMLMs that have been previously trained by using the ML criterion on Language Models by adapting and using the MMI Discriminative Method previously deployed in training HMMs for acoustic models. MMI acoustic models have proven to perform better than ML models, therefore MMI training gave a strong incentive in order to apply it on continuous language models. Perplexity is the metric being used to measure the effectiveness of the presented method. The experiments of the thesis focus on testing MMI models that are smoothed with their correspondent baseline ML model and MMI models that are unsmoothed, with mixed results.
