LING/C SC 581: Advanced Computa9onal Linguis9cs. Lecture Notes Jan 15 th

Transcription

1 LING/C SC 581: Advanced Computa9onal Linguis9cs Lecture Notes Jan 15 th

2 Course Webpage for lecture slides and Panopto recordings: hap://dingo.sbs.arizona.edu/~sandiway/ling581-15/ Mee6ng informa6on

3 Course Objec9ves Follow- on course to LING/C SC/PSYC 438/538 Computa(onal Linguis(cs: con9nue with selected material from the 538 textbook (J&M): 25 chapters, a lot of material not covered in 438/538 And gain more extensive experience with new stuff not in textbook dealing with natural language sowware packages Installa9on, input data formaxng opera9on project exercises useful real- world computa9onal experience abili9es gained will be of value to employers

4 Computa9onal Facili9es Use your own laptop/desktop can also make use of the computers in this lab (Shantz 338) PlaMorms but you don t have installa9on rights on these computers Plus the alarm goes off awer hours and campus police will arrive Windows is maybe possible but you really should run some variant of Unix (for your task #1 for this week) Linux (separate bootable par99on or via virtualiza9on sowware) de facto standard for advanced/research sowware haps:// (free!) Cygwin on Windows hap:// Linux- like environment for Windows making it possible to port sowware running on POSIX systems (such as Linux, BSD, and Unix systems) to Windows. OSX Not quite Linux, some por9ng issues, especially with C programs, can use Virtual Box (Linux under OSX)

5 Grading Comple9on of all homework tasks will result in a sa9sfactory grade (A) Tasks should be completed before the next class. me your work (sandiway@ .arizona.edu). also be prepared to come up and present your work (if called upon).

6 Today's Topics Homework Task 1: Install tregex

7 Homework Task 1: Install Tregex hap://nlp.stanford.edu/sowware/tregex.shtml Computer language: java (538: Perl regex on strings) 581: regex for trees

8 Homework Task 1: Install Tregex We ll use the program tregex from Stanford University to explore the Penn Treebank current version:

9 Penn Treebank Availability Source: Linguis9c Data Consor9um (LDC) U. of Arizona is a (fee- paying) member of this consor9um Resources are made available to the community through the main library URL hap://sabio.library.arizona.edu/search/x

10 Penn Treebank (V3) Call Record Have it on a usb drive here that I will pass around TREEBANK_3.zip (65.2MB)

11 Raw data: Penn Treebank (V3)

12 tregex Tregex is a Tgrep2- style u9lity for matching paaerns in trees. wri5en in Java run-tregex-gui.command shell script -mx flag, the 300m default memory size may need to be increased depending on the platform

13 tregex Select the PTB directory TREEBANK_3/parsed/mrg/wsj/ Browse Deselect any unwanted files

14 Part 2 Textbook: sec9on 3.11

15 general string comparison edit opera9ons are inser9on, dele9on and subs9tu9on not just limited to distance defined by a single opera9on away we can ask how different is string a from b by the minimum edit distance 15

16 applica6ons could be used for mul9- typo correc9on used in Machine Transla9on Evalua9on (MTEval) example Source: 生産工程改善について Transla6ons: (Standard) For improvement of the produc9on process (MT- A) About a produc9on process beaerment (MT- B) About the produc9on process improvement method compute edit distance between MT- A and Standard and MT- B and Standard in terms of word inser9on/subs9tu9on etc. 16

17 cost models Levenshtein inser9on, dele9on and subs9tu9on all have unit cost Levenshtein (alternate) inser9on, dele9on have unit cost subs9tu9on is twice as expensive subs8tu8on = one insert followed by one delete Typewriter inser9on, dele9on and subs9tu9on all have unit cost modified by key proximity 17

18 Dynamic Programming divide- and- conquer to solve a problem we divide it into sub- problems sub- problems may be repeated don t want to re- solve a sub- problem the 2nd 9me around idea: put solu9ons to sub- problems in a table and just look up the solu9on 2nd 9me around, thereby saving 9me memoiza(on we ll use a spreadsheet

19 Consider a simple case: xy yx Minimum # of opera9ons: insert and delete cost = 2 Minimum # of opera9ons: swap cost =?

20 Generally

21 Programming Practice: could be easily implemented in Perl

22 Generally

23 Computa9on Or in MicrosoW Excel, file: eds.xls (on course webpage) $ in a cell reference means don t change when copied from cell to cell e.g. in C$1 1 stays the same in $A3 A stays the same

24 Task: transform string s 1..s i into string t 1..t j each s n and t n are leaers string s is of length i, t is of length j Example: s = leader, t = adapter i = 6, j = 7 Let s say you re allowed just three opera9ons: (1) delete a leaer, (2) insert a leaer, or (3) subs9tute a leaer for another leaer What is one possible way to generate t from s?

25 Example: s = leader, t = adapter What is one possible way to generate t from s? leader!! adapter! adapter! adapte! lead! l! leader! leade! lea! le! ada! adap! adapt! Simplest method cost: 13 operations cost is 2 deletes and 3 inserts, total 5 opera9ons Ques9on: is this the minimum possible?

26 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r!

27 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r! cell (2,3) cost of transforming le into ada

28 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r! cell (2,3) cost of transforming le into ada cell (6,7) cost of transforming leader into adapter

29 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r! cell (3,0) cost of transforming lea into (empty)

30 0 1 l! 2 e! 3 a! 4 d! cell (0,4) cost of transforming (empty) into adap 5 e! 6 r!

31 0 1 l! 2 e! 3 a! 4 d! 5 e! k 6 r! cell (5,6) cost of transforming leade into adapte

32 0 1 l! 2 e! 3 a! 4 d! 5 e! k 6 r! cell (5,6) cost of transforming leade into adapte

33 0 1 l! 2 e! 3 a! 4 d! cell (5,6) cost of transforming leade into adapte 5 e! k 6 r! k

34 0 1 l! 2 e! k 3 a! 4 d! 5 e! 6 r! cell (2,3) cost of transforming le into ada

35 0 1 l! 2 e! k 3 a! 4 d! 5 e! 6 r! cell (2,3) cost of transforming le into ada cell (2,4) cost of transforming le into adap

36 0 1 l! 2 e! k k+1 3 a! 4 d! 5 e! 6 r! cell (2,3) cost of transforming le into ada cell (2,4) cost of transforming le into adap l! e! a! d! a! p!

37 0 1 l! k 2 e! 3 a! 4 d! 5 e! 6 r! cell (1,4) cost of transforming l into adap

38 0 1 l! k 2 e! k+1 3 a! 4 d! 5 e! 6 r! cell (1,4) cost of transforming l into adap a! d! a! p! l! e!

39 0 1 l! k 2 e! 3 a! 4 d! 5 e! cell (1,3) cost of transforming l into ada 6 r!

40 0 1 l! k 2 e! k+2 3 a! 4 d! 5 e! 6 r! cell (1,3) cost of transforming l into ada assuming the cost of swapping e for p is 2 l! e! a! d! a! p!

41 0 1 l! k 1,3 k 1,4 2 e! k 2,3? 3 a! 4 d! 5 e! 6 r! cell (2,4) minimum of the three costs to get here in one step

42 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r! cell (3,0) cost of transforming lea into (empty)

43 0 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r!

44 0 0 1 l! 1 2 e! 3 a! 4 d! 5 e! 6 r!

45 0 0 1 l! 1 2 e! 2 3 a! cost of le à! =! cost of l à, plus the cost of delet 4 d! 5 e! 6 r!

46 0 0 1 l! 1 2 e! 2 3 a! 3 4 d! 4 5 e! 5 6 r! 6

47 0 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r!

48 l! 2 e! 3 a! 4 d! 5 e! 6 r!

49 0 1 l! 2 e! 3 a! 4 d! 5 e! 6 r!

50 0 1 l! 1 2 e! 2 3 a! 3 4 d! 4 5 e! 5 6 r! 6

51 0 1 l! 1 2 e! 2 3 a! 3 4 d! 4 5 e! 5 6 r! 6

52 0 1 l! 1 2 e! 2 3 a! 3 4 d! 4 5 e! 5 6 r! 6

53 0 1 l! e! 2 3 a! 3 4 d! 4 5 e! 5 6 r! 6

54 0 1 l! 1 2 e! 2 3 a! 3 4 d! e! r! 6

55 0 1 l! 1 2 e! 2 3 a! 3 4 d! e! r! 6

56 0 1 l! 1 2 e! 2 3 a! 3 4 d! e! r! 6

57 0 1 l! 1 2 e! a! d! 4 5 e! 5 6 r! 6

58 0 1 l! 1 2 e! a! d! 4 5 e! 5 6 r! 6

59 0 1 l! 1 2 e! a! d! 4 5 e! 5 6 r! 6

60 0 1 l! 1 2 e! 2 3 a! 3 4 d! 4 5 e! r! 6 7

61 0 1 l! 1 2 e! 2 3 a! 3 4 d! 4 5 e! r! 6 7

62 0 1 l! 1 2 e! 2 3 a! 3 4 d! 4 5 e! r! 6 7 6

63