Compressed Suffix Trees in Practice

Size: px

Start display at page:

Download "Compressed Suffix Trees in Practice"

June Payne
6 years ago
Views:

1 Introdction CST design CST in practice Copressed Sffix Trees in Practice Sion Gog Copting and Inforation Systes The University of Melborne Febrary 13th 213

2 Introdction CST design CST in practice Otline 1 Introdction Basic data strctres The sffix tree 2 CST design NAV (tree topology and navigation) CSA (lexicographic inforation) LCP (longest coon prefixes) 3 CST in practice The sdsl library

3 Introdction CST design CST in practice Sccinct data strctres (1) Data strctre D representation of an object X Exaple: Rank-bit-vector bit vector b of length n (,1,,1,1,,1,1) (,,1,1,2,3,3,4) in n bits space + + operations on X access b[i] in O(1) tie rank(i) = i 1 j= b[j] in O(n) tie Sccinct data strctre D Space of D is close the inforation theoretic lower bond to represent X, while operations can still be perfored efficient.

4 Introdction CST design CST in practice Sccinct data strctres (1) Data strctre D representation of an object X Exaple: Rank-bit-vector bit vector b of length n (,1,,1,1,,1,1) (,,1,1,2,3,3,4) in n bits space + + operations on X access b[i] in O(1) tie rank(i) = i 1 j= b[j] in O(n) tie Sccinct data strctre D Space of D is close the inforation theoretic lower bond to represent X, while operations can still be perfored efficient.

5 Introdction CST design CST in practice Sccinct data strctres (1) Data strctre D representation of an object X Exaple: Rank-bit-vector bit vector b of length n (,1,,1,1,,1,1) (,,1,1,2,3,3,4) in n + n log n bits space + + operations on X access b[i] in O(1) tie rank(i) = i 1 j= b[j] in O(1) tie Sccinct data strctre D Space of D is close the inforation theoretic lower bond to represent X, while operations can still be perfored efficient.

6 Introdction CST design CST in practice Sccinct data strctres (1) Data strctre D representation of an object X Exaple: Rank-bit-vector bit vector b of length n (,1,,1,1,,1,1) (,,1,1,2,3,3,4) in n + n log n bits space + + operations on X access b[i] in O(1) tie rank(i) = i 1 j= b[j] in O(1) tie Sccinct data strctre D Space of D is close the inforation theoretic lower bond to represent X, while operations can still be perfored efficient.

7 Introdction CST design CST in practice Sccinct data strctres (2) Can sccinct data strctres replace classic ncopressed data strctres in practice? Less eory fewer CPU cycles!? Less eory less costs!? Probles: in theory develop sccinct data strctres in practice constants in O(1)-tie ters are large o(n)-space ter is not negligible coplex data strctres are hard to ipleent

8 Introdction CST design CST in practice Sccinct data strctres (2) Can sccinct data strctres replace classic ncopressed data strctres in practice? Less eory fewer CPU cycles!? 1 CPU cycle 5 CPU cycles CPU L1-Cache L2-Cache L3-Cache DRAM Disk 1 B 1 KB 512 KB 1-8 MB 4 GB x 1 GB Less eory less costs!? Probles: in theory develop sccinct data strctres in practice constants in O(1)-tie ters are large o(n)-space ter is not negligible

9 Introdction CST design CST in practice Sccinct data strctres (2) Can sccinct data strctres replace classic ncopressed data strctres in practice? Less eory fewer CPU cycles!? Less eory less costs!? Instance nae ain eory price per hor Micro 613. MB.2 US High-Meory Qadrple Extra Large 68.4 GB 2. US Pricing of Aazons Elastic Clod Copting (EC2) service in Jly 211. Probles: in theory develop sccinct data strctres in practice constants in O(1)-tie ters are large o(n)-space ter is not negligible coplex data strctres are hard to ipleent

10 Introdction CST design CST in practice Sccinct data strctres (2) Can sccinct data strctres replace classic ncopressed data strctres in practice? Less eory fewer CPU cycles!? Less eory less costs!? Probles: in theory develop sccinct data strctres in practice constants in O(1)-tie ters are large o(n)-space ter is not negligible coplex data strctres are hard to ipleent

11 Introdction CST design CST in practice The classic index data strctre: The sffix tree (ST) Let T be a text of length n over alphabet Σ of size σ. Sffix tree index data strctre for T (constrction O(n)) can be sed to solve any probles in optial tie coplexity bioinforatics data copression ses O(n log n) bits! In practice (ASCII-alphabet) 17 ties the size of T Can not handle The Attack of Massive Data DNA seqencing data (NGS)...

12 Introdction CST design CST in practice Exaple: ST of T=lndl 15 7 dl ndl l 3 ndl l 1 ndl ndl ndl l 13 ndl l 5 n = 16 Σ ={,d,l,,n,} σ = ndl Classic ipleentation ses pointers each of size 4 or 8 bytes!

13 Introdction CST design CST in practice Exaple: ST of T=lndl 15 7 dl ndl l l ndl ndl ndl ndl l 13 ndl 8 l ndl Operations root() is leaf (v) parent(v) degree(v) 5 child(v, c) select child(v, i) depth(v) edge(v, d) lca(v, w) sl(v) wl(v, c)

14 Introdction CST design CST in practice CSTs Goal of a CST ipleentation Replace fastest ncopressed ST ipleentations in different scenarios (a) both fit in RAM and we easre tie (b) both fit in RAM and we easre resorce costs (c) only CST fits in RAM and we easre tie Proposals Sadakane s CST cst_sada Flly Copressed Sffix Tree (Rsso et al.) CSTs based on interval representation of nodes (Fischer et al. csty, Ohlebsch et al. cst_sct3)

15 Introdction CST design CST in practice CSTs Goal of a CST ipleentation Replace fastest ncopressed ST ipleentations in different scenarios (a) both fit in RAM and we easre tie (b) both fit in RAM and we easre resorce costs (c) only CST fits in RAM and we easre tie Proposals which ight work for (a) and (b) Sadakane s CST cst_sada Flly Copressed Sffix Tree (Rsso et al.) CSTs based on interval representation of nodes (Fischer et al. csty, Ohlebsch et al. cst_sct3)

16 Introdction CST design CST in practice Otline 1 Introdction Basic data strctres The sffix tree 2 CST design NAV (tree topology and navigation) CSA (lexicographic inforation) LCP (longest coon prefixes) 3 CST in practice The sdsl library

17 Introdction CST design CST in practice Big pictre of CST design Ψ Wavelet Tree T BWT Hffan excess CSA NAV Min-Max-Tree LF Pioneer RMQ arentheses Seq 2n bits first child LCP PLCP

18 Introdction CST design CST in practice Big pictre of CST design 15 7 dl 14 l ndl l ndl 11 ndl 9 l 3 ndl ndl ndl l ndl (()()(()())(()((()())()()))()((()())(()(()()))()))

19 Introdction CST design CST in practice Exaple: Copressing NAV 15 7 dl l l ndl ndl l l ndl ndl 6 9 ndl 3 12 ndl ndl 11 1 tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

20 Introdction CST design CST in practice Exaple: Copressing NAV 15 7 dl l l ndl ndl l l ndl ndl 6 9 ndl 3 12 ndl ndl 11 1 tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

21 Introdction CST design CST in practice Exaple: Copressing NAV dl l l ndl ndl l l ndl ndl 6 9 ndl 3 12 ndl ndl 11 1 tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

22 Introdction CST design CST in practice Exaple: Copressing NAV dl l l ndl ndl l l ndl ndl 6 9 ndl 3 12 ndl ndl 11 1 tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

23 Introdction CST design CST in practice Exaple: Copressing NAV dl l l ndl ndl l l ndl ndl 6 9 ndl 3 12 ndl ndl 11 1 tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

24 Introdction CST design CST in practice Exaple: Copressing NAV dl l l ndl ndl l l ndl ndl 6 9 ndl 3 12 ndl ndl 11 1 tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

25 Introdction CST design CST in practice Exaple: Copressing NAV 15 1 dl l l 5 ndl ndl ndl 11 l l ndl ndl 6 9 ndl 3 12 ndl tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

26 Introdction CST design CST in practice Exaple: Copressing NAV 15 1 dl l l 5 ndl ndl ndl 11 l 6 l ndl ndl 6 9 ndl 3 12 ndl tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

27 Introdction CST design CST in practice Exaple: Copressing NAV 15 1 dl l l 5 ndl ndl ndl 11 l 6 l ndl ndl 6 9 ndl 3 12 ndl tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

28 Introdction CST design CST in practice Exaple: Copressing NAV 15 1 dl l l 5 ndl ndl ndl 11 l 6 l ndl ndl 6 9 ndl ndl tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

29 Introdction CST design CST in practice Exaple: Copressing NAV 15 1 dl l l 5 ndl ndl ndl 11 l 6 l ndl ndl 6 9 ndl ndl tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

30 Introdction CST design CST in practice Exaple: Copressing NAV 15 1 dl l l 5 11 ndl ndl ndl 11 l 36 6 l 15 ndl ndl ndl ndl tree ncopressed O(n log n) bits BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) copressed 4n bits

31 Introdction CST design CST in practice NAV data strctres (1) 4n bits 2n bits 15 1 dl l l 5 11 ndl ndl ndl 11 l 36 6 l 15 ndl ndl ndl ndl BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) -[, 15] 15 dl 1-[1, 15] l 73-[2, 3] 1-[4, 14 8] l ndl ndl 2-[5, 8] 4-[1, 11] ndl 11 l 2-[12, 14] 5-[5, 6] l ndl ndl 6-[13, 14] 6 9 ndl 3 12 ndl BPS sct= ( ( ( ( )( ( ( )( ( )))( ( ( )( ( ( ))( )))))))) LCP= o(n) bits to answer find open(i), find close(i), enclose(i), doble enclose(i, j),rank(i, c), select(i, c),... in constant tie

32 Introdction CST design CST in practice NAV data strctres (2) Coparison of different NAV strctres cst_sada cst_sct cst_sct3 space in bits 4n + o(n) 2n + o(n) 3n + o(n) root() O(1) O(1) O(1) degree(v) O(σ) O(t LCP log σ) O(1) depth(v) O(t LCP ) O(t LCP ) O(t LCP ) parent(v) O(1) O(t LCP log σ) O(1) select child(v, i) O(i) O(t LCP ) O(1) sibling(v) O(1) O(t LCP ) O(1) sl(v), lca(v, w) O(1) O(t LCP log σ) O(1) child(v, c) O(t SA σ) O(t SA log σ) O(t SA log σ)

33 Introdction CST design CST in practice Exaple operations: select leaf (i) and lca(v, w) on NAV 15 7 dl l l ndl ndl l l ndl ndl 6 9 ndl ndl ndl 11 BPS dfs = (()()(()())(()((()())()()))()((()())(()(()()))())) 1 select leaf (4) = select(4, 1 ) = 8 select leaf (5) = select(5, 1 ) = 12 lca(8,12) = doble enclose(8,12) =

34 Introdction CST design CST in practice Virtes of a CSA based on BWT Sall size: CSA = BWT + n log n s SA where BWT can be chosen to be n log σ bits nh (T) bits nh k (T) + O(σ k ) bits bits pattern atching in tie O( P log σ) (even O( P ) for σ polylog(n)) by backward search (Ferragina & Manzini)

35 Introdction CST design CST in practice H k of the Pizza&Chili 2MB test cases dblp.xl dna english proteins rand_k128 sorces k H k CT /n H k CT /n H k CT /n H k CT /n H k CT /n H k CT /n

36 Introdction CST design CST in practice Backward search i T BWT n l l d F dl l lndl l lndl ndl ndl l lndl l lndl ndl (a) T BWT n l l d F dl l lndl l lndl ndl ndl l lndl l lndl ndl (b) T BWT n l l d F dl l lndl l lndl ndl ndl l lndl l lndl ndl (c)

37 Introdction CST design CST in practice Wavelet tree: rank for character seqences d 1 nlld lld 11 1 ll 1 n n 1 1 nlld nlld nlld d 1 n d 1 1 nll (a) d ll n (b)

38 Introdction CST design CST in practice Exaple: Copressing CSA (practical approach) i SA LF T BWT T n l l d dl l lndl l lndl ndl ndl l lndl l lndl ndl

39 Introdction CST design CST in practice Exaple: Copressing CSA (practical approach) SA ncopressed n log n bits n l l d copressed d l l n s SA = 3 access LF[i] in tie O(log σ) access CSA[i] in tie O(s SA log σ) CSA = T BWT + SA saples: n log σ + o(n log σ) bits + n log n s SA bits

40 Introdction CST design CST in practice Exaple: Copressing CSA (practical approach) SA[13]= n l d l l n s SA = 3 access LF[i] in tie O(log σ) access CSA[i] in tie O(s SA log σ) l d SA ncopressed n log n bits copressed CSA = T BWT + SA saples: n log σ + o(n log σ) bits + n log n s SA bits

41 Introdction CST design CST in practice Exaple: Copressing CSA (practical approach) SA[13]=SA[1]+1 n l d l l n s SA = 3 access LF[i] in tie O(log σ) access CSA[i] in tie O(s SA log σ) l d SA ncopressed n log n bits copressed CSA = T BWT + SA saples: n log σ + o(n log σ) bits + n log n s SA bits

42 Introdction CST design CST in practice Exaple: Copressing CSA (practical approach) SA[13]=SA[9]+2 n l d l l n s SA = 3 access LF[i] in tie O(log σ) access CSA[i] in tie O(s SA log σ) l d SA ncopressed n log n bits copressed CSA = T BWT + SA saples: n log σ + o(n log σ) bits + n log n s SA bits

43 Introdction CST design CST in practice Exaple: Copressing CSA (practical approach) SA[13]=6+2=8 n l d l l n s SA = 3 access LF[i] in tie O(log σ) access CSA[i] in tie O(s SA log σ) l d SA ncopressed n log n bits copressed CSA = T BWT + SA saples: n log σ + o(n log σ) bits + n log n s SA bits

44 Introdction CST design CST in practice Overview of LCP data strctres data strctre ses access eory in bits lcp_ncopressed - O(1) n log n lcp_spport_sada CSA O(t SA ) 2n + o(n) lcp_krtz - O(log n) or O(1) 8n 1 + 2n 2 log n lcp_spport_tree NAV O(log σ ) H q 1 + q 2 log n lcp_spport_tree2 NAV & LF O(s LCP log σ ) H q 1 + (q 2 log n)/s LCP with n 1 + n 2 = n and q 1 + q 2 = q < n q nber of inner nodes of the ST

45 Introdction CST design CST in practice Rntie for rando access to the LCP array (1) dblp.xl.2mb proteins.2mb Tie in nanoseconds per operation lcp_ncopressed lcp_wt lcp_dac lcp_krtz lcp_spport_sada lcp_spport_tree lcp_spport_tree Meory in bits per character Meory in bits per character

46 Introdction CST design CST in practice Rntie for rando access to the LCP array (2) dna.2mb rand_k128.2mb Tie in nanoseconds per operation lcp_ncopressed lcp_wt lcp_dac lcp_krtz lcp_spport_sada lcp_spport_tree lcp_spport_tree Meory in bits per character Meory in bits per character

47 Introdction CST design CST in practice Rntie for rando access to the LCP array (3) english.2mb sorces.2mb Tie in nanoseconds per operation lcp_ncopressed lcp_wt lcp_dac lcp_krtz lcp_spport_sada lcp_spport_tree lcp_spport_tree Meory in bits per character Meory in bits per character

48 Introdction CST design CST in practice Otline 1 Introdction Basic data strctres The sffix tree 2 CST design NAV (tree topology and navigation) CSA (lexicographic inforation) LCP (longest coon prefixes) 3 CST in practice The sdsl library

49 Introdction CST design CST in practice The sccinct data strctre library sdsl Provides basic and advanced sccinct data strctres Easy to se (very siilar to C++ STL) Fast and space-efficient constrction of data strctres 64-bit ipleentation Well-optiized ipleentation (e.g. now sing hardware POPCOUNT operation,.. ) Easy configration of yriads of CSAs, CSTs with any tie-space trade-offs Fast prototyping of other coplex sccinct data strctres

50 Introdction CST design CST in practice cst_sada<csa_sada<>,lcp_.. cst_sct3<csa_wt<wt_hff<>.. vector has eber of class type vector has eber of class type int_vector int_vector enc_vector has teplate paraeter enc_vector has teplate paraeter rrr_vector of concept rrr_vector of concept rank_spport wavelet_tree select_spport rank_spport wavelet_tree select_spport rank_spport_v wt select_spport_cl rank_spport_v wt select_spport_cl rank_spport_v5 wt_int select_spport_bs rank_spport_v5 wt_int select_spport_bs rrr_rank_spport wt_hff rrr_select_spport rrr_rank_spport wt_hff rrr_select_spport wt_rln wt_rln wt_rlg wt_rlg csa csa csa_ncopressed csa_ncopressed csa_sada csa_sada csa_wt csa_wt lcp bp_spport lcp bp_spport lcp_ncopressed bp_spport_g lcp_ncopressed bp_spport_g lcp_dac bp_spport_gg lcp_dac bp_spport_gg lcp_wt bp_spport_sada lcp_wt bp_spport_sada lcp_krtz lcp_krtz lcp_spport_sada rq lcp_spport_sada rq lcp_spport_tree rq_spport_sparse_table lcp_spport_tree rq_spport_sparse_table lcp_spport_tree2 rq_sccinct_sct lcp_spport_tree2 rq_sccinct_sct rq_sccinct_sada rq_sccinct_sada cst cst cst_sada cst_sada cst_sct3 cst_sct3

51 Introdction CST design CST in practice CST space in practice (for english.2mb) MB 27% CSA: 18.8 MB wt: 148 MB data: 114 MB rank: 7.1 MB select 1: 14.4 MB select : 12.5 MB sa_saple: 21.9 MB isa_saple: 1.9 MB lcp: 215 MB lcp vales: 17.4 MB overflow ark: 41.9 MB rank: 2.6 MB nav: MB BPS dfs (bit_vector): 8.9 MB bp_spport: 37.1 MB sall block: 5.7 MB edi block: 1.6 MB bp rank: 2.2 MB bp select: 9.6 MB rank_spport1: 2.2 MB select_spport1: 6.1 MB cst_sada<csa_wt<wt_hff<> >,lcp_dac<> >

52 Introdction CST design CST in practice CST space in practice (for english.2mb) MB 23% 114. CSA: 18.8 MB wt: 148 MB data: 114 MB rank: 7.1 MB select 1: 14.4 MB select : 12.5 MB sa_saple: 21.9 MB isa_saple: 1.9 MB lcp: 96.1 MB sall lcp: 72.1 MB data: 67.9 MB rank: 4.2 MB big lcp: 24 MB nav: MB bp: 8.9 MB bp_spport: 21.9 MB sall block: 5.7 MB edi block: 1.6 MB bp rank: 5.1 MB bp select: 9.6 MB rank_spport1: 2.2 MB select_spport1: 6.1 MB cst_sada<csa_wt<wt_hff<> >,lcp_spport_tree2<> >

53 Introdction CST design CST in practice CST space in practice (for english.2mb) MB 167% CSA: 18.4 MB wt: 75.6 MB data: 75.6 MB bt: 3.4 MB btnr: 31.4 MB btnrp: 6.7 MB rank saples: 6.9 MB invert:.2 MB rank: MB select 1: MB select : MB sa_saple: 21.9 MB isa_saple: 1.9 MB lcp: 96.1 MB sall lcp: 72.1 MB data: 67.9 MB rank: 4.2 MB big lcp: 24 MB nav: MB BPS dfs (bit_vector): 8.9 MB bp_spport: 21.9 MB sall block: 5.7 MB edi block: 1.6 MB bp rank: 5.1 MB bp select: 9.6 MB rank_spport1: 2.2 MB select_spport1: 6.1 MB cst_sada<csa_wt<wt_hff<rrr_vector<> > >,lcp_spport_tree2<> >

54 Introdction CST design CST in practice CST space in practice (for english.2mb) MB 147% CSA: 18.4 MB wt: 75.6 MB data: 75.6 MB bt: 3.4 MB btnr: 31.4 MB btnrp: 6.7 MB rank saples: 6.9 MB invert:.2 MB rank: MB select 1: MB select : MB sa_saple: 21.9 MB isa_saple: 1.9 MB lcp: 96.1 MB sall lcp: 72.1 MB data: 67.9 MB rank: 4.2 MB big lcp: 24 MB nav: 88.4 MB bp: 5 MB bp_spport: 13.4 MB sall block: 3.5 MB edi block:.8 MB cst_sct3<csa_wt<wt_hff<rrr_vector<> > >,lcp_spport_tree2<> >

55 Introdction CST design CST in practice Experiental setp ˆ= cst_sada<csa_sada<>, lcp_dac<> > 1 ˆ= cst_sada<csa_sada<>, lcp_spport_tree2<> > 2 ˆ= cst_sada<csa_wt<>, lcp_dac<> > 3 ˆ= cst_sada<csa_wt<>, lcp_spport_tree2<> > 4 ˆ= cst_sct3<csa_sada<>, lcp_dac<> > 5 ˆ= cst_sct3<csa_sada<>, lcp_spport_tree2<> > 6 ˆ= cst_sct3<csa_wt<>, lcp_dac<> > 7 ˆ= cst_sct3<csa_wt<>, lcp_spport_tree2<> > The sae basic data strctres are sed, i.e. its a very fair coparison

56 Introdction CST design CST in practice Rntie of operations of cst_sada csa[i]: psi[i]: lcp[i]: id(v): depth(v): depth(v) : parent(v): sibling(v): sl(v): child(v, c): select child(v, 1): lca(v, w): dfs and id(v): dfs and depth(v): stats : µs 2µs 4µs 6µs µs 2µs 4µs 6µs

57 Introdction CST design CST in practice Rntie of operations of cst_sct3 csa[i]: psi[i]: lcp[i]: id(v): depth(v): depth(v) : parent(v): sibling(v): sl(v): child(v, c): select child(v, 1): lca(v, w): dfs and id(v): dfs and depth(v): stats : µs 2µs 4µs 6µs µs 2µs 4µs 6µs

58 Introdction CST design CST in practice Tie-space trade-off for select child(v, 1) english.2mb sorces.2mb Tie in nanoseconds per operation Meory in bits per character Meory in bits per character

59 Introdction CST design CST in practice Tie-space trade-off for child(v, c) english.2mb sorces.2mb Tie in nanoseconds per operation Meory in bits per character Meory in bits per character

60 Introdction CST design CST in practice Constrction of a CST #inclde < sdsl / s f f i x t r e e s. hpp> #inclde < sdsl / t i l. hpp> sing naespace sdsl ; typedef cst_sct3 <> tcst ; i n t ain ( i n t argc, char argv [ ] ) { tcst c s t ; c o n s t r c t _ c s t ( argv [ 1 ], c s t ) ; }

61 Introdction CST design CST in practice Rntie for constrction (prefixes of english text) 1 4 cstv ST cst_sada () cst_sct3 (4) Tie in seconds Text size in MB

62 Introdction CST design CST in practice Meory in bytes per character CST constrction resorces coparison text: english.2mb σ: 226 CST type: cst_sada LCP type: lcp_sada CSA type: csa_wt 5 first CST ipleentation (27) sdsl (21) NAV LCP 1 Tie in seconds CSA

63 Introdction CST design CST in practice Detailed resorces for the constrction of CSTs text : english.2mb CST type: cst_sada CSA type: csa_wt text : english.2mb CST type: cst_sada CSA type: csa_wt text : english.2mb CST type: cst_sct3 CSA type: csa_wt 1 Tie in seconds LCP: 67.2 CSA: 18.3 NAV: 9.9 wt: 9.9 SA: 57.4 bp_spport: 2.3 T BWT : 29.4 NAV: LCP: 65.9 CSA: 18.5 wt: 9.9 SA: 56.5 bp_spport: 23.5 T BWT : 29.6 NAV: LCP: 98.8 CSA: 16.8 wt: 9.9 SA: 56.7 bp_spport: 24.3 ISA: 33.9 T BWT : Meory in bytes per inpt character

64 Introdction CST design CST in practice Depth first search traversal in a CST teplate <class Cst > void t e s t _ c s t _ d f s _ i t e r a t o r _ a n d _ d e p t h ( Cst &c s t ) { typedef typenae Cst : : c o n s t _ i t e r a t o r i t e r a t o r ; long long cnt = ; for ( i t e r a t o r i t = c s t. begin ( ) ; i t!= c s t. end ( ) ; + + i t ) { i f (! c s t. i s _ l e a f ( i t ) ) cnt += c s t. depth ( i t ) ; } cot << cnt << endl ; }

65 Introdction CST design CST in practice Rntie of dfs on CST (prefixes of english text) 1 4 cstv ST cst_sada () cst_sct3 (4) Tie in nanoseconds Text size in MB

66 Introdction CST design CST in practice Conclsion CSTs can be bild fast and space-efficient... provide a rich set of fnctionality fast operations: basic navigation, access LF or Ψ slow operations: child(v, c), access to SA Yo can se the sdsl library to configre a CSTs which fits yor needs

67 Introdction CST design CST in practice Thank yo!

68 Introdction CST design CST in practice Rntie of int_vector access Tie in nanoseconds per operation rando access read rando access write seqential write int_vector<32> bit_vector int_vector<> v(..,..,32) int_vector<> v(..,..,27)

69 Introdction CST design CST in practice Operation rntie of basic data strctres Tie in nanoseconds per operation int_vector<32> rank_spport_v rank_spport_v5 select_spport_cl (rando access)

70 Introdction CST design CST in practice Rntie of CSA access (1) Tie in nanoseconds per operation dblp.xl.2mb proteins.2mb csa_wt<wt<>> csa_wt<wt_hff<>> csa_wt<wt_rln<>> csa_wt<wt_rlg<8>> csa_sada<δ> csa_sada<φ> Meory in bits per character Meory in bits per character

71 Introdction CST design CST in practice Rntie of CSA access (2) Tie in nanoseconds per operation dna.2mb rand_k128.2mb csa_wt<wt<>> csa_wt<wt_hff<>> csa_wt<wt_rln<>> csa_wt<wt_rlg<8>> csa_sada<δ> csa_sada<φ> Meory in bits per character Meory in bits per character

72 Introdction CST design CST in practice Rntie of CSA access (3) Tie in nanoseconds per operation english.2mb sorces.2mb csa_wt<wt<>> csa_wt<wt_hff<>> csa_wt<wt_rln<>> csa_wt<wt_rlg<8>> csa_sada<δ> csa_sada<φ> Meory in bits per character Meory in bits per character

73 Introdction CST design CST in practice Rntie of CSA operations (1) Tie in nanoseconds per operation dblp.xl.2mb proteins.2mb csa_wt<wt<>> csa_wt<wt_hff<>> csa_wt<wt_rln<>> csa_wt<wt_rlg<8>> csa_sada<δ> csa_sada<φ> Meory in bits per character Meory in bits per character =psi[i], =psi(i)=lf[i], + =bwt[i]

74 Introdction CST design CST in practice Rntie of CSA operations (2) Tie in nanoseconds per operation dna.2mb rand_k128.2mb csa_wt<wt<>> csa_wt<wt_hff<>> csa_wt<wt_rln<>> csa_wt<wt_rlg<8>> csa_sada<δ> csa_sada<φ> Meory in bits per character Meory in bits per character =psi[i], =psi(i)=lf[i], + =bwt[i]

75 Introdction CST design CST in practice Rntie of CSA operations (3) Tie in nanoseconds per operation english.2mb sorces.2mb csa_wt<wt<>> csa_wt<wt_hff<>> csa_wt<wt_rln<>> csa_wt<wt_rlg<8>> csa_sada<δ> csa_sada<φ> Meory in bits per character Meory in bits per character =psi[i], =psi(i)=lf[i], + =bwt[i]

MODELLING THE EFFECTS OF PEDESTRIANS ON INTERSECTION CAPACITY AND DELAY WITH ACTUATED SIGNAL CONTROL

MODELLING THE EFFECTS OF PEDESTRIANS ON INTERSECTION CAPACITY AND DELAY WITH ACTUATED SIGNAL CONTROL ABSTRACT Zong Tian, Ph.D., P.E. Feng Xu, Graduate Assistant Departent of Civil and Environental Engineering