IA-64: Advanced Loads Speculative Loads Software Pipelining

Size: px

Start display at page:

Download "IA-64: Advanced Loads Speculative Loads Software Pipelining"

Cecily Watts
5 years ago
Views:

1 Presentation stolen from the web (with changes) from the Univ of Aberta and Espen Skoglund and Thomas Richards (47 alum) and Our textbook s authors : Advanced Loads Speculative Loads Software Pipelining

2 28 64-bit registers Use a register window similarish to SPARC bit fp registers 64 bit predicate registers 8 64-bit branch target registers

3 Explicit Parallelism Groups Instructions which could be executed in parallel if hardware resources available. Bundle Code format. 3 instructions fit into a 28-bit bundle. 5 bits of template, 4*3 bits of instruction.» Template specifies what execution units each instruction requires.

4 Instruction groups instructions are bound in instruction groups No read-after-write dependencies No write-after-write dependencies Any instruction in the group may be executed in parallel New processors can easily take advantage of the existing ILP in the instruction group Instruction groups indicated by stop bits in template Instruction groups may end dynamically on branches

5 Instruction bundles Instruction bundles contain 3 instructions A template field which maps instructions to execution units Processor dispatches all three instruction in parallel Instruction group may end in middle of bundle Bundles are aligned on 6 byte boundaries Slot 3 Slot 2 Slot Template Instruction bundle

6 Predication Use predicates to eliminate branches Predicates are one bit registers (total of 64) Most instructions can be predicated (qp) mnemonic dest = source Predicates are set by compare instructions (qp) cmp.crel px,py = source C code: if (a == b) y += 3; else y += 4; x86 assembly: cmp a, b beq.eq add $4, y jmp.done.eq: add $3, y.done: assembly: cmp.eq p,p2 = a,b (p) add y = y, 3 (p2) add y = y, 4

7 Advanced loads and speculative loads Advanced loads Used to address data dependencies Speculative loads Used to address control dependencies advanced load speculative load st st (p) br (p) br ld check load ld check speculation

8 Advanced loads Addr and addr2 in example might point to same address If different: Datum in addr2 can be prefetched If same: Datum in addr2 can not be prefetched C code example: int foo (int *addr, int *addr2) { int h; *addr = 4; h = *addr2; return h+; }

9 Advanced loads Insert advanced loads (ld.a) to prefetch data (store in ALAT) Use check data instruction (ld.c) in place of original load If memory contents has changed, perform real load Advanced loads do not defer exceptions (e.g., page-faults) Regular load: add r3 = 4,r ;; st4 [r32] = r3 ld4 r2 = [r33] regular load add r5 = r2,r3 use data Advanced Load: ld4.a r2 = [r33] advanced load add r3 = 4,r ;; st4 [r32] = r3 ld4.c r2 = [r33] ;; verify data add r5 = r2,r3 use data

10 Speculative loads If addr in example is legal, we can prefetch its value If addr is illegal, prefetching the value would cause exception Any exception should be delayed until code path has been resolved C code example: int add5 (int *addr) { if (addr == NULL) return (-); else return (*addr+5); }

11 Speculative loads Insert speculative loads (ld.s) to prefetch data Verify load using check instruction (chk.s) NaT-bit/NaTVal is used track success of load Might also be combined with advanced loads (ld.sa and chk.a) Assembly code: add5: ld8.s r = [r32] cmp.eq p6,p5 = r32,r ;; (p6) add r8 = -,r (p6) br.ret (p5) chk.s r,return_error add r8 = 5,r br.ret ;; return_error: recovery code

12 Code example Why hoist loads? add r5 = r2,r3 mult r4 = r5,r2 mult r4 = r4,r4 st8 [r2] = r4 ld8 r5 = [r5] div r6 = r5,r7 add r5 = r6,r2 Assume latencies are: add, store: + mult, div: +3 ld: +4 //A //B //C //D //E //F //G 2 A: B:4 E:5 C:4 F:4 D: G:

13 Advanced Loads Recovery Case A Hoist just the load.» In this case, if there is a memory dependency we just re-execute the load. Case B Hoist the load and dependent instructions.» In this case, we need to reexecute all of the dependent instructions. // Case A: Advanced Load ld.a r2 = [r] st8 [r] = r9 ld.c r2 = [r] add r5 = r2, r3 st8 [r8] = r9 // Case B: Advanced Load // With Speculative Add ld.a r2 = [r] add r5 = r2, r3 st8 [r] = r9 ld.c r2 = [r] // Wrong st8 [r8] = r9 A ld.c will only re-execute the load, r5 is still wrong after the ld.c!

14 Advanced Load-Use Recovery: Compiler Generated Recovery Code // Solution: Using the chk.a instruction ld8.a r2 = [r] add r5 = r2, r3 st8 [r] = r9 chk.a r6, fixup return: // Return Point st8 [r8] = r fixup: // Re-execute load and all speculative uses ld8 r2 = [r] add r5 = r2, r3 br return Use ld.c if JUST a load is speculative. Use chk.a if a load and an instruction that is dependant on the load are both speculative.

15 The Advanced Load Address Table (ALAT) The ALAT tells us if we need to recover from an Advanced Load. When an advanced load is executed Save the type of load, size of load, and load address to the ALAT (indexed by PR). When we execute a ld.c or chk.a look for the entry in the ALAT. If it is missing, run the recovery code. Remove an entry from the ALAT if A store address overlaps an ALAT entry. Capacity/Associatively evictions. Other advanced load indexes the same PR.

16 Control Speculation and Recovery What if we want to move a load above a branch? Problem is that the load maybe shouldn t have executed and might have thrown a spurious exception. Similar to Advanced Load, but no ALAT. Instead, check NaT bit for deferred exceptions.» See next slide. Use chk.s for recovery (instead of chk.a or ld.a). // Control Speculation and Recovery ld8.s r = [r] // load moved outside of branch st8 [r] = r9 (p)br.cond branch_label // (p) is a predication bit chk.s r,recovery return: add r2 = r, r2 chk.s checks r to see if the NaT bit is set. If so, branch to recovery code (re-execute instructions if necessary).

17 Not a Thing Bit (NaT) IA64 register 64bits + NaT If a control speculative load causes an exception, the processor can set this bit, which defers the exception. NaT bits propagate. Propagation allows a single check for multiple ld.s. ld8.s r = [r] ld8.s r2 = [r] add r3 = r, r2 ld8.s r4 = [r3] st8[r] = r9 (p)br.cond branch_label chk.s r4, recovery

18 Software pipelining on Lots of tricks Rotating registers Special counters Often don t need Prologue and Epilog. Special counters and prediction lets us only execute those instructions we need to.

19 r3=r3-8 r4=mem[r2+] r=r4*2 r4=mem[r2+4] Loop: MEM[r2+]=r r=r4*2 r4=mem[r2+8] r2=r2+4 bne r2 r3 Loop MEM[r2+]=r r=r4*2 MEM[r2+]=r r3=r3+8 Prolog and epilog From before!!!!! // Needed to check legal! //A() //B() //A(2) //C(n) //B(n+) //A(n+2) //D(n) //E(n) // C(x-) // B(x) // C(x) // Could have used tmp var.

20 There are three special purpose registers used in for software pipelining There are three special purpose registers used in for software pipelining Loop counter () indicates how many times to run through loop (prolog/kernel) Initialized to N- before starting loop code Decremented until == Epilog counter () indicates how many times to run loop after loop counter exhausted (epilog) Needed to flush the software pipeline Initialized to num-stages before entering loop code Decremented if ==, and >

21 And (Register Rename Base) Add internal counter to register number to get actual used register Counter decreased by special loop branch instructions May be reset by clrrrb instruction Use modular lookup (so we wrap around!) Rotated predicate registers Initially reset using: mov pr.rot = value pr 63 is reset before every rotation

22 How does register rotation work? (Basics) Rotated registers: General: gr 32 - gr N (as specified by alloc instruction) Predicate: pr 6 - pr 63 Floating point: fr 6 - fr 27 Registers are rotated to higher numbers Register r n is renamed to r n+, r max is renamed to r min Registers are rotated by specific loop branch instructions br.ctop, br.cexit (for counted loops) br.wtop, br.exit (for while loops)

23 ctop, cexit How they relate? == (epilog) (prolog/kernel)!= >? (special unrolled loops) == == -- = = = = = PR[63]= PR[63]= PR[63]= PR[63]= = ctop: branch cexit: fall-thru ctop: fall-thru cexit: branch

24 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35,

25 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35,

26 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35,

27 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35,

28 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35,

29 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35,

30 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35,

31 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y

32 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y

33 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y

34 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y

35 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y

36 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y

37 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y

38 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y

39 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y

40 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y

41 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y

42 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y

43 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y

44 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y

45 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y

46 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y4 y

47 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y y4 y

48 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y y4 y

49 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y y4 y

50 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y y4 y

51 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y y5 y y4 y

52 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y y5 y y4 y

53 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y y5 y y4 y

54 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y y5 y y4 y

55 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y y5 y y4 y

56 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y y5 y y4 y

57 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y4 y y5 y y4 y

58 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y4 y y5 y y4 y

59 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y4 y y5 y y4 y

60 (p6) ldl r32 = [r2], (p7) add r34 =, r33 (p8) stl [r3] = r35, y y3 y4 y y5 y y4 y

61 Software pipelining Review No prolog or epilog in code But we execute a lot of noops. Rotated registers help In this case, we just didn t have to reverse the code ordering» But in general, better still. Could move load from use more than one loop iteration apart. Looks good at least in this case

62 review Some problems ALAT difficult for compliers to use.» Recall Colwell talking about once we figure out how to do this 28/3 instruction size makes I-cache worse. Big register file has disadvantages» Context switch mainly. So many dependencies with special purpose instructions, dynamic OoO is unlikely. But If the complier could do a good job, there really does look like the potential for a big win.

CS650 Computer Architecture. Lecture 5-1 Branch Hazards

CS650 Computer Architecture. Lecture 5-1 Branch Hazards CS65 Computer Architecture Lecture 5- Branch Hazards Andrew Sohn Computer Science Department New Jersey Institute of Technology Lecture 5-: Branch Hazards 5--/6 /5/24 A. Sohn Simple 5-Stage Computer 4