Lecture 3a: Chuks Aoucemets Code Freeze Day! From here o out, do t chage your code Exceptios: Bug fixes : do t tell me why your code crashed, just fix it. Checkpoited before-ad-after: If you tell me how the froze versio of your code works The you ca also say how a (small) modificatio chages performace CS540 4/7/8 Material borrowed (with permissio) from James Pustejovsky & Marc Verhage of Bradeis. Mistakes are mie. Aoucemets (II) Paper : Due oe week from today Four pages (o loger) Three parts Itroductio : describe your motivatio ad program Why did you desig it the way you did How is it supposed to work Predicted limitatios ad risks Performace How well does it work o the problems give? Report relevat quatitative metrics for your desig You may supplemet with additioal problems of your ow desig Coclusio Do your predictios match the results? If ot, why ot? Aoucemets (III) I-class presetatio : oe week from today 5 miutes maximum (timed) Powerpoit or pdf slides Email to me (draper@colostate.edu) the ight before Same 3-part structure as the paper Project #3 Preview Project #3 is OPTIONAL Do it if uhappy with existig grades Accepted through Wedesday, May 9 th I will try to be quick about Project #2 grades Task: Build a NLP iterface to your Blocks World Simulator Cotrol it through a series of Eglish laguage commads Ca use existig parsers ad other tools Pipelie of NLP Tools Scrapig (ot covered here) Setece splittig Tokeizatio (Stemmig) Part-of-speech taggig Shallow parsig Named etity recogitio Sytactic parsig (Sematic Role Labelig) Thursday Today 6
POS as HMM What are the states? POS tags CC/CD/ /NN/ /WRB Because the goal is to fid the most likely sequece of tags What are the observatios Words What is i the Trasitio Table? Maps POS tags to POS tags Probabilities how likely is a sigular ou (NN) to be followed by a adjective (JJ)? Traied usig a labeled corpus What about the observatio matrix? Maps words (observatios) to states (POS tags) Etries: P(POS tag word) Traied o labeled corpus POS taggig with Hidde Markov Models P ( t... t w... w ) tags words P = µ P» ( w... w t... t ) P( t... t ) P( w... w ) ( w... w t... t ) P( t... t ) Õ i= P ( w t ) P( t t ) i i i i- output probability trasitio probability 8 POS taggig algorithms Performace o the Wall Street Joural corpus Traiig Cost Speed Accuracy Depedecy Net (2003) Low Low 97.2 Coditioal Radom Fields High High 97. Support vector machies (2003) 97. Bidirectioal MEMM (2005) Low 97. Brill s tagger (995) Low 96.6 HMM (2000) Very low High 96.7 Chukig (shallow parsig) He reckos the curret accout deficit will arrow to VP VP PP oly #.8 billio i September. PP A chuker (shallow parser) segmets a setece ito o-recursive phrases. 9 0 The Nou Phrase () Examples: He Barak Obama The Presidet The former Cogressma from Illiois They ca all appear i a similar cotext: was bor i Hawaii. Prepositioal Phrases Examples: the ma i the white suit Come ad look at my paitigs Are you fod of aimals? Put that thig o the floor 2 2
Verb Phrases Examples: He wet He was tryig to keep his temper. She quickly showed me the way to hide. Chukig Text chukig is dividig seteces ito ooverlappig phrases. Nou phrase chukig deals with extractig the ou phrases from a setece. While chukig is much simpler tha parsig, it is still a challegig task to build a accurate ad very efficiet chuker. 3 4 What is it good for Chukig is useful i may applicatios: Iformatio Retrieval & Questio Aswerig Machie Traslatio Preprocessig before full sytactic aalysis Text to speech What kid of structures should a partial parser idetify? Differet structures useful for differet tasks: Partial costituet structure [ I] [ VP saw [ a tall ma i the park]]. Prosodic segmets [I saw] [a tall ma] [i the park]. Cotet word groups [I] [saw] [a tall ma] [i the park]. 5 6 Chuk Parsig Goal: divide a setece ito a sequece of chuks. Chuks are o-overlappig regios of a text: [I] saw [a tall ma] i [the park]. Chuks are o-recursive a chuk ca ot cotai other chuks Chuks are o-exhaustive ot all words must be icluded i chuks Chuk Parsig Examples Nou-phrase chukig: [I] saw [a tall ma] i [the park]. Verb-phrase chukig: The ma who [was i the park] [saw me]. Prosodic chukig: [I saw] [a tall ma] [i the park]. 7 8 3
Chuks ad Costituecy Costituets: [a tall ma i [the park]]. Chuks: [a tall ma] i [the park]. Chuks are ot costituets Costituets are recursive Chuks are typically subsequeces of Costituets Chuks do ot cross costituet boudaries Chuk Parsig: Accuracy Chuk parsig achieves higher accuracy Smaller solutio space Less word-order flexibility withi chuks tha betwee chuks Better locality: Fewer log-rage depedecies Less cotext depedece No eed to resolve attachmet ambiguity Less error propagatio 9 20 Chuk Parsig: Domai Specificity Chuk parsig is less domai specific: Depedecies o lexical/sematic iformatio ted to occur at levels "higher" tha chuks: Attachmet Argumet selectio Movemet Fewer stylistic differeces withi chuks Chuk Parsig: Efficiecy Chuk parsig is more efficiet Smaller solutio space Relevat cotext is small ad local Chuks are o-recursive Chuk parsig ca be implemeted with a fiite state machie 2 22 Psycholiguistic Motivatios Chuk parsig is psycholiguistically motivated: Chuks as processig uits Humas ted to read texts oe chuk at a time Eye-movemet trackig studies Chuks are phoologically marked Pauses, Stress patters Chukig might be a first step i full parsig Chuk Parsig Techiques Chuk parsers usually igore lexical cotet Oly eed to look at part-of-speech tags Techiques for implemetig chuk parsig: Regular expressio matchig / Fiite State Machies (see ext) Trasformatio Based Learig Memory Based Learig Other taggig-style methods 23 24 4
Regular Expressio Matchig Defie a regular expressio that matches the sequeces of tags i a chuk A simple ou phrase chuk regexp: <DT>? <JJ>* <NN>* <NN> Chuk all matchig subsequeces: the/dt little/jj cat/nn sat/vbd o/in the/dt mat/nn [the/dt little/jj cat/nn] sat/vbd o/in [the/dt mat/nn] If matchig subsequeces overlap, the first oe or logest oe gets priority Chukig as Taggig Map Part of Speech tag sequeces to {I,O,B}* I tag is part of a chuk O tag is ot part of B the first tag of a chuk which immediately follows aother chuk Alterative tags: Begi, Ed, Outside Example: Iput: The teacher gave Sara the book Output: I I O I B I 25 26 Chukig State of the Art Whe addressed as taggig methods similar to POS taggig ca be used HMM combiig POS ad IOB tags TBL rules based o POS ad IOB tags Depedig o task specificatio ad test set: 90-95% for chuks Chukig with Machie learig Chukig performace o Pe Treebak Wiow (with basic features) (Zhag, 2002) Recall Precisi o F- score 93.60 93.54 93.57 Perceptro (Carreras, 2003) 93.29 94.9 93.74 SVM + votig (Kudoh, 2003) 93.92 93.89 93.9 SVM (Kudo, 2000) 93.5 93.45 93.48 Bidirectioal MEMM (Tsuruoka, 2005) 93.70 93.70 93.70 27 28 Named-Etity Recogitio We have show that iterleuki- (IL-) ad IL-2 cotrol protei protei protei IL-2 receptor alpha (IL-2R alpha) gee trascriptio i DNA CD4-CD8-murie T lymphocyte precursors. cell_lie Recogize amed-etities i a setece. Gee/protei ames Protei, DNA, RNA, cell_lie, cell_type Sytactic Parsig S VP QP VBN NN VBD DT JJ CD CD NNS. Estimated volume was a light 2.4 millio ouces. 29 30 5
Phrase Structure + Head Iformatio Depedecy relatios S VP QP VBN NN VBD DT JJ CD CD NNS. Estimated volume was a light 2.4 millio ouces. VBN NN VBD DT JJ CD CD NNS. Estimated volume was a light 2.4 millio ouces. 3 32 Parse Tree S o Sematic Structure DT 2 4 VP 6 VP 5 VP 2 S o VP 5 Predicate argumet relatios A AJ 5 ormal 7 8 0 VP 7 does AV 9 ot VP 22 exclude 25 24 DT 2 4 A AJ 5 7 ARG2 ormal 8 0 VP 6 VP 2 VP 7 AV 9 VP 22 25 ARG2 does ot exclude 24 serum CRP 3 measuremet AJ 26 deep 29 vei 28 3 thrombosis 33 MOD serum MOD CRP 3 mesuremet AJ 26 28 deep 29 3 MOD vei thrombosis 34 Feature-Based Parsig Lexical etry HEAD: ou SUBJ: <> HEAD: verb SUBJ: <> Subject-head schema HEAD: verb SUBJ: <ou> HEAD: verb SUBJ: <ou> Head-modifier schema HEAD: Mary walked slowly adv MOD: verb HPSG A few schema May lexical etries Deep sytactic aalysis Grammar Corpus-based grammar costructio (Miyao et al 2004) Parser Beam search (Tsuruoka et al.) 35 6