Real-World Performance Training Hands-On Exercise

Transcription

1

2 Real-World Performance Training Hands-On Exercise Real-World Performance Team

3 Solution Progress, Step 1 Initial Observations

4 RWP Training Extreme DW Hands-On Exercise: Baseline q.low.sql Runs in 5 seconds Nested Loops joins and B*Tree index access on fact table Almost all the time on random I/O against fact table index and table Cardinality estimate problem on PART

5 RWP Training Extreme DW Hands-On Exercise: Baseline q.med.sql Runs in 183 seconds Nested Loops joins and B*Tree index access on fact table Almost all the time on random I/O against fact table index and table Cardinality estimate problem on PART

6 RWP Training Extreme DW Hands-On Exercise: Baseline q.high.sql Runs in 474 seconds Nested Loops joins and B*Tree index access on fact table Almost all the time on random I/O against fact table index and table Cardinality estimate problem on PART

7 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds

8 Solution Progress, Step 2 Gather Stats on PART

9 RWP Training Extreme DW Hands-On Exercise: Stats on PART q.low.sql Runs in 3 seconds Nested Loops joins and B*Tree index access on fact table Now performing hash join to DATE_DIM Better cardinality estimates on PART, but still off

10 RWP Training Extreme DW Hands-On Exercise: Stats on PART q.med.sql Runs in 3 seconds Now using Bloom Filter for PART and HASH JOIN Hash Joins to other dimensions Better cardinality estimates on PART, but still off

11 RWP Training Extreme DW Hands-On Exercise: Stats on PART q.high.sql Runs in 6 seconds Filtering on SUPPLIER Bloom Filter with Hash Join Hash Joins to other dimensions

12 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds

13 Solution Progress, Step 3 Extended Stats on PART

14 RWP Training Extreme DW Hands-On Exercise: Extended Stats on PART q.low.sql Runs in 3 seconds Now doing NL join between DATE_DIM and LINEORDER Hash joining to PART and SUPPLIER Reference to CUSTOMER indicates lack of Join Elimination (and constraints) Accurate cardinality estimates on PART changed join order

15 RWP Training Extreme DW Hands-On Exercise: Extended Stats on PART q.med.sql Runs in 3 seconds Accurate cardinality estimate on PART yield a slightly different right-deep plan Reference to CUSTOMER indicates lack of Join Elimination (and constraints) Accurate cardinality estimates on PART changed join order

16 RWP Training Extreme DW Hands-On Exercise: Extended Stats on PART q.high.sql Runs in 6 seconds No plan change from previous step Reference to CUSTOMER indicates lack of Join Elimination (and constraints)

17 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds

18 Solution Progress, Step 4 Constraints (PK, FK, NOT NULL)

19 RWP Training Extreme DW Hands-On Exercise: Constraints q.low.sql Runs in 3 seconds No reference to CUSTOMER; Join Elimination takes place Still doing NL join & index access for DATE_DIM/LINEORER join

20 RWP Training Extreme DW Hands-On Exercise: Constraints q.med.sql Runs in 4 seconds No reference to CUSTOMER; Join Elimination takes place Now doing Hash Joins with Bloom Filters for both DATE_DIM and PART dimensions Scanning all partitions; not performing Bloom pruning

21 RWP Training Extreme DW Hands-On Exercise: Constraints q.high.sql Runs in 4 seconds No reference to CUSTOMER; Join Elimination takes place Now doing Hash Joins with Bloom Filters for both DATE_DIM and SUPPLIER dimensions Scanning all partitions; not performing Bloom pruning

22 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds

23 Solution Progress, Step 5 Partition LINEORDER on LO_ORDERDATE

24 RWP Training Extreme DW Hands-On Exercise: Partitioning q.low.sql Runs in 2 seconds Using BF and Hash Join with PART dimension Now iterating over fact table partitions for DATE_DIM NL join

25 RWP Training Extreme DW Hands-On Exercise: Partitioning q.med.sql Runs in 31 seconds No Bloom Filters used Execution time partition iteration due to data type mismatch between DATE_DIM and LINEORDER No Bloom Pruning

26 RWP Training Extreme DW Hands-On Exercise: Partitioning q.high.sql Runs in 2 seconds Bloom Filtering, Bloom Pruning, and Hash Joins

27 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds Partitioning 2 seconds 31 seconds 2 seconds 35 seconds

28 Solution Progress, Step 6 Fix DATE_DIM Data Type

29 RWP Training Extreme DW Hands-On Exercise: Data Types q.low.sql Runs in 3 seconds With proper data types, reverted to index access path and NL joins Performing partition range scan after NL join between PART/LINEORDER

30 RWP Training Extreme DW Hands-On Exercise: Data Types q.med.sql Runs in 0.2 seconds Right-deep, BF plan with hash joins Bloom pruning possible with proper data types

31 RWP Training Extreme DW Hands-On Exercise: Data Types q.high.sql Runs in 1 seconds Bloom Filtering, Bloom Pruning, and Hash Joins same as previous test

32 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds Partitioning 2 seconds 31 seconds 2 seconds 35 seconds Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds

33 Solution Progress, Step 7 Bitmap Indexes and Star Transformation

34 RWP Training Extreme DW Hands-On Exercise: Star Transformation q.low.sql Runs in 0.2 seconds Star transformation yielded decent response time

35 RWP Training Extreme DW Hands-On Exercise: Star Transformation q.med.sql Runs in 5 seconds Here, the transformation isn t engaged but we re still accessing the fact table via a bitmap index Spending most of our time in random I/O

36 RWP Training Extreme DW Hands-On Exercise: Star Transformation q.high.sql Runs in 45 seconds Star transformation Spending most of our time in random I/O. Clearly, star transformation & bitmap indexes isn t the right choice for this workload. This is the risk with small bitmap indexes; Oracle often will cost these types of plans in when full scans are faster

37 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds Partitioning 2 seconds 31 seconds 2 seconds 35 seconds Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds

38 Solution Progress, Step 8 Exadata, No Star Transformation, No Bitmap Indexes, DoP=4 on LINEORDER

39 RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4 q.low.sql Runs in 1 second In parallel, query runs quite fast

40 RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4 q.med.sql Runs in 0.55 seconds In parallel, query runs quite fast

41 RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4 q.high.sql Runs in 1 second In parallel, query runs quite fast

42 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds Partitioning 2 seconds 31 seconds 2 seconds 35 seconds Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds

43 Solution Progress, Step 9 Database In-Memory, DoP=4 on LINEORDER

44 RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4 q.low.sql Runs in 0.03 seconds TABLE ACCESS STORAGE FULL becomes TABLE ACCESS INMEMORY FULL

45 RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4 q.med.sql Runs in 1 second TABLE ACCESS STORAGE FULL becomes TABLE ACCESS INMEMORY FULL

46 RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4 q.high.sql Runs in 0.52 seconds

47 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds Partitioning 2 seconds 31 seconds 2 seconds 35 seconds Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds DBIM with DoP= seconds 1 second 0.52 seconds 1.55 seconds

48 Solution Progress, Step 10 In-Memory Aggregation (DoP = 1)

49 RWP Training Extreme DW Hands-On Exercise: IMA q.low.sql Runs in 0.03 seconds The IMA transformation

50 RWP Training Extreme DW Hands-On Exercise: IMA q.med.sql Runs in 0.09 seconds The IMA transformation

51 RWP Training Extreme DW Hands-On Exercise: IMA q.high.sql Runs in 1 second

52 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds Partitioning 2 seconds 31 seconds 2 seconds 35 seconds Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds DBIM with DoP= seconds 1 second 0.52 seconds 1.55 seconds IMA with DoP= seconds 0.09 seconds 1 seconds 1.12 seconds

53 Solution Progress, Step 11 Materialized Views In-Memory, Clustered

54 RWP Training Extreme DW Hands-On Exercise: MJV DBIM q.low.sql Runs in seconds

55 RWP Training Extreme DW Hands-On Exercise: MJV DBIM q.med.sql Runs in seconds

56 RWP Training Extreme DW Hands-On Exercise: MJV DBIM q.high.sql Runs in seconds You can make things really fast if you take it this far. A total of 0.14 seconds for each of the 3 typical star queries!

57 Progress q.low.sql q.med.sql q.high.sql Total Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds Constraints 3 seconds 4 seconds 4 seconds 11 seconds Partitioning 2 seconds 31 seconds 2 seconds 35 seconds Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds DBIM with DoP= seconds 1 second 0.52 seconds 1.55 seconds IMA with DoP= seconds 0.09 seconds 1 seconds 1.12 seconds MJV with DBIM seconds seconds seconds 0.17 seconds

58 Appendix A Instructor Notes

59 RWP Training Extreme DW Hands-On Exercise Instructor Notes Objective: Demonstrate multiple steps to achieve query optimization 1. Download SQL*Monitor reports from on_sqlmon.zip 2. Show reports in baseline and constraints section just open a browser to show them. The demo application will take too long to run them 3. Go to the demo system and launch Star Query Demo. Use Windows for display/vga purposes and Chrome 4. Click through each of the numbered queries on the list. There 8 steps with 3 queries each; the steps are Constraints and Stats, Extended Stats, Partitioning on the right column, Star transform w/ bitmap indexes, Exadata DoP=4, DBIM DoP=4, IMA with DoP=1, and MV In-Memory DoP=4 5. You can maximize any window and click on the Description to generate the SQL*Monitor report If things go wrong: 1. Login to scao08adm01 as oracle 2. Go to /home/oracle/users/jclarke/training/hands_on/solution 3. SQL*plus as DW_JC/DW_JC 4. Run solution.sql

60 Appendix B Code Examples

61 Constraints NOT NULL, Primary Key, and Foreign Key Constraints

62 Statistics Unlocking, Gathering Stats & Extended Stats

63 Star Transformation Bitmap Indexes and Star Transformation

64 Star Transformation Partitioning LINEORDER on correct column

65 Star Transformation Mark Tables for In-Memory Column Store and Populate

66 Star Transformation Check In-Memory Population Status

67