CS3350B Computer Architecture. Lecture 6.2: Instructional Level Parallelism: Hazards and Resolutions

Transcription

1 CS3350B Compute Achitectue Winte 2015 Lectue 6.2: Instuctional Level Paallelism: Hazads and Resolutions Mac Moeno Maza [Adapted fom lectues on Compute Oganization and Design, Patteson & Hennessy, 5 th edition, 2011] 0

2 Recap: Pipelining fo Pefomance All moden day pocessos use pipelining Pipelining doesn t help latency of single task, it helps thoughput of entie wokload Potential speedup: CPI=?, and a faste CC Recall CPU time = CPI * CC * IC Pipeline ate limited by slowest pipeline stage Unbalanced pipe stages make fo inefficiencies The time to fill pipeline and time to dain it can impact speedup fo deep pipelines and shot code uns Must detect and esolve hazads Can always esolve hazads by waiting (Stalling) Stalling negatively affects CPI (makes CPI moe than the ideal of 1) 1

3 Thee Types of Pipeline Hazads Stuctual hazads Attempt to use the same esouce by two diffeent instuctions at the same time Data hazads (fom what types of instuctions?) Attempt to use data befoe it is eady in instuctions involving aithmetic and data tansfes - An instuction s souce opeand(s) ae poduced by a pio instuction still in the pipeline Contol hazads Attempt to make a decision about pogam contol flow befoe the condition has been evaluated and the new PC taget addess calculated; banch instuctions Can always esolve hazads by waiting (makes CPI > 1) Bette to have pipeline contol to detect the hazads and take action to esolve hazads moe efficiently 2

4 Stuctual Hazad #1: in case of Single Memoy Time (clock cycles) I n s t. lw Inst 1 Mem Reg Mem Reg Mem Reg Mem Reg Reading data fom memoy O d e Inst 2 Inst 3 Mem Reg Mem Reg Mem Reg Mem Reg Inst 4 Reading instuction fom memoy Mem Reg Mem Reg Read same memoy twice in same clock cycle 3

5 Stuctual Hazad #1: Fix with sepaate instuction and data memoies (I$ and D$) I n s t. O d e lw Inst 1 Inst 2 Inst 3 Inst 4 Time (clock cycles) I$ Reg D$ Reg 4

6 Stuctual Hazad #2: Registes (1/2) I n s t. O d e lw Inst 1 Inst 2 Inst 3 Inst 4 Time (clock cycles) I$ Reg D$ Reg Can we ead and wite to egistes simultaneously? 5

7 Stuctual Hazad #2: Registes (2/2) Two diffeent solutions have been used: (1) RegFile access is vey fast: takes less than half the time of stage - Wite to Registes duing fist half of each clock cycle - Read fom Registes duing second half of each clock cycle (2) Build RegFile with independent ead and wite pots Result: can pefom egiste Read and Wite duing same clock cycle 6

8 Data Hazad Type 1 (1/2) Conside the following sequence of instuctions add $t0, $t1, $t2 sub $t4, $t0, $t3 and $t5, $t0, $t6 o $t7, $t0, $t8 xo $t9, $t0, $t10 Q1: What ae the dependences? $t0 of sub depends on $t0 of add; Read Afte Wite (RAW) $t0 of and depends on $t0 of add; RAW $t0 of o depends on $t0 of add; RAW $t0 of xo depends on $t0 of add; RAW Q2: Ae thee any hazads? We use pipeline diagam to analyze it. 7

9 Data Hazad Type 1 (2/2) Data-flow backwad in time ae hazads. This case is a ead befoe wite data hazad. I n s t. O d e Time (clock cycles) add $t0,$t1,$t2 sub $t4,$t0,$t3 and $t5,$t0,$t6 CC1 CC2 CC3 CC4 CC5 CC6 CC7 CC8 CC9 IF ID/RF EX MEM WB o $t7,$t0,$t8 I$ xo $t9,$t0,$t10 Reg D$ Reg 8

10 Data Hazad Solution 1: Stall (Waiting) Stall, o bubble, o nop; no backwad data flow anymoe I n s t. O d e add $t0,$t1,$t2 stall stall sub $t4,$t0,$t3 and $t5,$t0,$t6 CC1 CC2 CC3 CC4 CC5 CC6 IF ID/RF EX MEM WB bubble bubble bubble bubble bubble bubble bubble bubble bubble bubble CC7 CC8 CC9 CC10 CC11 o $t7,$t0,$t8 I$ xo $t9,$t0,$t10 Reg D$ Reg How many cycles? What s the CPI now? 11/5 9

11 Data Hazad Solution 2: Fowading (aka Bypassing) Hadwae fowads esult to the stage needed as soon as it is available (bypassing the egiste) - - fowading in this case - Hadwae: hazad detection unit; fowad unit o hazad solved by egiste hadwae add $t0,$t1,$t2 sub $t4,$t0,$t3 CC1 CC2 CC3 CC4 CC5 CC6 IF ID/RF EX MEM WB CC7 CC8 CC9 and $t5,$t0,$t6 o $t7,$t0,$t8 I$ xo $t9,$t0,$t10 Reg D$ Reg How many cycles? What is the CPI now? 9/5 10

12 Yet Anothe Complication! Anothe potential data hazad can occu when thee is a conflict between the esult of the WB stage instuction and the MEM stage instuction which should be fowaded? I n s t. O d e add $1,$1,$2 add $1,$1,$3 add $1,$1,$4 IM Reg DM Reg IM Reg DM Reg IM Reg DM Reg

13 Data Hazad Type 2: Load/Use (1/2) Dataflow backwads in time ae hazads CC1 CC2 CC3 CC4 CC5 CC6 CC7 lw $t0,0($t1) sub $t3,$t0,$t2 IF ID/RF EX MEM WB 12

14 Data Hazad Type 2: Load/Use (2/2) Is it feasible to fix it by just fowading? i.e. when the data is loaded fom D$ befoe witing to the egiste, fowad it to fo sub. CC1 CC2 CC3 CC4 CC5 CC6 CC7 lw $t0,0($t1) sub $t3,$t0,$t2 IF ID/RF EX MEM WB Oops! Still a backwad data flow! Can we go back in time? Must stall instuction dependent on load, then fowad (moe hadwae) 13

15 Load/Use Data Hazad: Solution Option 1 Hadwae detects hazad, stalls pipeline (Called intelock ), and fowad (MEM- fowading). CPI =? 9/4 lw $t0, 0($t1) sub $t3,$t0,$t2 and $t5,$t0,$t4 IF ID/RF EX MEM WB bub ble bub ble CC9 bub ble o $t7,$t0,$t6 I$ Reg D$ Reg Not in MIPS: (MIPS = Micopocesso without Intelocked Pipeline Stages) 14

16 Load/Use Data Hazad Solution Option 2 Inset nop (equivalent to stall) and fowad lw $t0, 0($t1) nop bub ble bub ble bub ble bub ble bub ble sub $t3,$t0,$t2 and $t5,$t0,$t4 o $t7,$t0,$t6 I$ Reg D$ 15

17 Remaks on Load/Use Data Hazad Instuction slot afte a load is called load delay slot If that instuction uses the esult of the load, then the hadwae intelock will stall it fo one cycle. Altenative: If the compile puts an unelated instuction in that slot, then no stall Letting the hadwae stall the instuction in the delay slot is equivalent to putting a nop in the slot (except the latte uses moe code space) 16

18 Load/Use Data Hazads: Code Scheduling to Avoid Stalls Reode code to avoid use of load esult in the next instuction (load delay slot) C code fo A = B + E; /* $t3 = $t1 + $t2 */ C = B + F; /* $t5 = $t1 + $t4 */ stall stall lw $t1, 0($t0) lw $t2, 4($t0) add $t3, $t1, $t2 sw $t3, 12($t0) lw $t4, 8($t0) add $t5, $t1, $t4 sw $t5, 16($t0) 13 cycles lw $t1, 0($t0) lw $t2, 4($t0) lw $t4, 8($t0) add $t3, $t1, $t2 sw $t3, 12($t0) add $t5, $t1, $t4 sw $t5, 16($t0) 11 cycles 17

19 Memoy-to-Memoy Copies Fo loads immediately followed by stoes (memoy-tomemoy copies) can avoid a stall by adding fowading hadwae fom the MEM/WB egiste to the data memoy input (MEM-MEM fowading) Would need to add a Fowad Unit and a mux to the memoy access stage I n s t. O d e lw $1,4($2) sw $1,4($3)

20 Contol Hazads Banch detemines flow of contol Fetching next instuction depends on banch outcome The delay in detemining the pope instuction to fetch is called a contol hazad o banch hazad. Pipeline can t always fetch coect instuction - Still woking on ID stage of banch beq, bne in MIPS pipeline 19

21 Contol Hazads Simple Solution Option 1: two Stalls Stall on evey banch until have new PC value; Would add 2 bubbles/clock cycles fo evey Banch! I n s t. O d e beq nop nop Inst Inst Time (clock cycles) EX bubble bubble bubble bubble bubble bubble bubble bubble bubble bubble I$ Reg D$ Reg Whee do we do the compae fo the banch? 20

22 Contol Hazad: Banching Optimization #1: Inset special banch compaato in Stage 2 (Dec) As soon as instuction is decoded (i.e. Opcode identifies it as a banch), immediately make a decision and set the new value of the PC Benefit: since banch is complete in Stage 2, only one unnecessay instuction is fetched, so only one no-op is needed Side Note: means that banches ae idle in Stages 3, 4 and 5 21

23 Special Banch Compaato with One Clock Cycle Stall Time (clock cycles) I n s t. O d e beq nop Inst Inst Inst ID/RF bubble bubble bubble bubble bubble I$ Reg D$ Reg Banch compaato moved to Decode stage 22

24 Pefomance of Stall on Banch Assume banches ae 17% of the instuctions executed in SPECint2006. Since the othe instuctions un have a CPI of 1, and banches took one exta clock cycle fo the stall, then we would see a CPI of 1.17 and hence a slowdown of 1.17 vesus the ideal case. 23

25 Contol Hazads: Banch Delay Slot Optimization #2: Redefine banches Old definition: if we take the banch, none of the instuctions afte the banch get executed by accident New definition: whethe o not we take the banch, the single instuction immediately following the banch gets executed (the banch-delay slot) Delayed Banch means we always execute the instuction afte banch This optimization is used with MIPS. 24

26 Example: Nondelayed vs. Delayed Banch Nondelayed Banch o $8, $9, $10 add $1, $2, $3 sub $4, $5, $6 beq $1, $4, Exit xo $10, $1, $11 Delayed Banch add $1, $2,$3 sub $4, $5, $6 beq $1, $4, Exit o $8, $9, $10 xo $10, $1, $11 Exit: Exit: 25

27 Notes on Banch-Delay Slot Wost-Case Scenaio: put a no-op in the banch-delay slot Bette Case: place some instuction peceding the banch in the banch-delay slot as long as the changed doesn t affect the logic of pogam - Re-odeing instuctions is common way to speed up pogams - Compile usually finds such an instuction 50% of time - Jumps also have a delay slot Since delayed banches ae useful when the banches ae shot, no pocesso uses a delayed banch of moe than one cycle. Fo longe banch delays, hadwae-based banch pediction is usually used. The delayed banch always executes the next sequential instuction, with the banch taking place afte that one instuction delay. It is hidden fom the MIPS assembly language pogamme because the assemble can automatically aange the instuctions to get the banch behavio desied by the pogamme. MIPS softwae will place an instuction immediately afte the delayed banch instuction that is not affected by the banch, and a taken banch changes the addess of the instuction that follows this safe instuction. 26

28 Contol Hazads: Banch Pediction Opt #3: Pedict outcome of a banch, fix up if guess wong Must cancel all instuctions in pipeline that depended on wongguess This is called flushing the pipeline Opt 3.1: Assume banches ae NOT taken, continue execution down the sequential instuction steam. If the banch is taken, the instuctions that ae being fetched and decoded must be discaded. Execution continues at the banch taget. If banches ae untaken half the time, and if it costs little to discad the instuctions, this optimization halves the cost of contol hazads. Opt3.2: Dynamic banch pediction: Pediction of banches at untime using untime infomation. banch pediction buffe o banch histoy table 27

29 In Summay: Hazads and Resolutions Stuctual Hazads Memoy: I$ and D$ ae sepaated Registe: ead and wite can be done in same clock cycle Data Hazads load followed by stoe: MEM-MEM fowading load/use - Hadwae intelock (stall pipeline) and MEM- fowading - load delay slot: put a nop o a valid instuction afte load (MIPS) othe cases: one stall (nop) plus - fowading Hadwae suppot: hazad detection unit and fowad unit Contol hazads stall two cycles if banch execution done in EX stage stall one cycle if banch execution done in ID stage banch delay slot: put a nop (one cycle waste) o a valid instuction afte banch (MIPS) (banch execution in ID) banch pedication: banch not taken o dynamic pedication How does a hazad solution impact the pipeline pefomance? 28

30 Execise 1 Fo the following code sequence in MIPS, Indicate the dependences Indicate the potential hazads and types Povide you hazad esolution methods and show how many exta clock cycles you have to pay. sub $2, $1,$3 # Registe $2 witten by sub and $12,$2,$5 # 1st opeand($2) depends on sub o $13,$6,$2 # 2nd opeand($2) depends on sub add $14,$2,$2 # 1st($2) & 2nd($2) depend on sub sw $15,100($2) # Base ($2) depends on sub 29

31 Execise 2 Show what happens when the banch is taken in this instuction sequence, assuming the pipeline is optimized fo banches that ae not taken and that we moved the banch execution to the ID stage. The numbes to the left of the instuction (40, 44,... ) ae the addesses of the instuctions. 36 sub $10, $4, $8 40 beq $1, $3, 7 # PC-elative banch to * 4 = and $12, $2, $5 48 o $13, $2, $6 52 add $14, $4, $2 56 slt $15, $6, $7 72 lw $4, 50($7) 30