Title page for ETD etd-08242004-012316


Type of Document Master's Thesis
Author Pereira, Dennis V.
Author's Email Address dpereira@vt.edu
URN etd-08242004-012316
Title Automatic Lexicon Generation for Unsupervised Part-of-Speech Tagging Using Only Unannotated Text
Degree Master of Science
Department Computer Science
Advisory Committee
Advisor Name Title
Egyhazy, Csaba J. Committee Chair
Belli, Gabriella M. Committee Member
Frakes, William B. Committee Member
Keywords
  • lexicon generation
  • part-of-speech
  • term categorization
  • lexicon
  • automatic
Date of Defense 2004-08-13
Availability unrestricted
Abstract
With the growing number of textual resources available, the ability to understand them becomes critical. An essential first step in understanding these sources is the ability to identify the parts-of-speech in each sentence. The goal of this research is to propose, improve, and implement an algorithm capable of finding terms (words in a corpus) that are used in similar ways--a term categorizer. Such a term categorizer can be used to find a particular part-of-speech, i.e. nouns in a corpus, and generate a lexicon. The proposed work is not dependent on any external sources of information, such as dictionaries, and it shows a significant improvement (~30%) over an existing method of categorization. More importantly, the proposed algorithm can be applied as a component of an unsupervised part-of-speech tagger, making it truly unsupervised, requiring only unannotated text. The algorithm is discussed in detail, along with its background, and its performance. Experimentation shows that the proposed algorithm performs within 3% of the baseline, the Penn-TreeBank Lexicon.
Files
  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
  08242004_Dennis_Pereira_ETD.pdf 1.60 Mb 00:07:23 00:03:48 00:03:19 00:01:39 00:00:08

Browse All Available ETDs by ( Author | Department )

dla home
etds imagebase journals news ereserve special collections
virgnia tech home contact dla university libraries

If you have questions or technical problems, please Contact DLA.