Agenda. 2 sessions 13:30-15:00 Game Analysis. 15:00-15:30 Ask A Head Referee

Transcription

1 FIRST Game Analysis

2 Agenda 2 sessions 13:30-15:00 Game Analysis 15:00-15:30 Ask A Head Referee

3 Who are we? Anne Shade 22nd year in FIRST Former FIRST Chesapeake Regional Director Former 2377 General Manager and Coach Former Mechanical and Systems Engineer, Leidos Steve Shade 21nst year in FIRST FIRST Chesapeake Head Referee Former 2377 Lead Engineer and Coach Senior Engineer Modeling and Simulation, JHU Applied Physics Laboratory

4 Background Knowledge Basic Understanding of FRC Games Basic Understanding of Statistics

5 Common Themes Write it down Post it up Questions What do you want to accomplish? Who is responsible? How are you going to accomplish it? Repeat

6 Making Decisions Every team should have a method to make decisions Popular vote is not always practical for robot building purposes, often misses the quiet contributors We like Decision Matrices for comparing ideas Voting can be used to break ties

7 Session Overview Assumptions Establish team goals Create event goals Form match goals Data Review Review statistical models Examine 2018, 2017, and 2016 data Compare results to models Game Analysis Understand how to play the game Determine how many points is needed to win a match Construct a game strategy to meet the point goal Implementation Select a strategy Evaluate options Strategic design Competition execution

8 Assumptions

9 Assumptions Your team wants to advance to the District Championship and World Championship Be a contributing alliance member Consistent and improving on-field performance A great robot can win matches, but two good robots with great strategy win more frequently

10 Big Picture Process Robot Requirements On Field Performance Robot Design Robot Construction Coaching Game & Team Strategy Scouting Robot Testing Prioritization Decision Making

11 District Ranking District Championship qualification is determined by points your team earns at first two Chesapeake district events plus team age correction FIRST Championship qualification includes points earned at Chesapeake District Championship District championship weighted 3x district events Qualifying Points Alliance Selection Order Playoff Performance Awards Event Ranking Points Chesapeake District Event 1 Chesapeake District Event 2 Chesapeake District Champ. Team Age Correction (1st & 2nd year teams) Total District Ranking Points

12 District Ranking Qualifying Points Alliance Selection Order Playoff Performance Awards Event Ranking Points Chesapeake District Event 1 Chesapeake District Event 2 Chesapeake District Champ. Team Age Correction (1st & 2nd year teams) Total District Ranking Points Must score points to maximize these categories!

13 Statistics Review and Historical Data

14 80/20 Rule (Pareto Principle) For many events, roughly 80% of the effects come from 20% of the causes In other words, if we can find the meaningful 20% of the game and make it great, we can contribute to achieve 80% of the points and win about 80% of the time Pareto works best with power distribution Scores in 2011 & 2012 followed power distribution Scores have become more normally distributed over time, but principles still apply

15 The Normal Distribution Distribution defined by mean (µ) and standard deviation (σ) calculated from raw data Probability of a win given a score can be found by calculating the area under the pdf. P(win score) = blue area Raw data can be modeled to reflect probability of score occuring. Σ P(all scores) = 1 Probability Density Function (pdf) The Cumulative Distribution Function (cdf) can be generated for by calculating the area under the pdf at each score. Looking for the score where the cdf = 0.8 (score = 0.85)

16 What are typical FRC scores? Depends on the game Depends on the event Our analysis ignores the effects of Foul points Fouls are included in recorded scores Fouls not included in Kickoff analysis Known error source Scores generally increase each week, Big steps at DCMP and CMP FIRST Chesapeake is not FIRST in Michigan, Indiana or Ontario, but districts are making a huge difference

17 Ranking Strategy Goal to win about 80% of your matches 12 matches x 0.8 = 9 to 10 wins per event Districts - Top 6 (estimated ranking) DCMP - Top 12 (estimated ranking) CMP - Top 24 (estimated ranking) Ignores bonus Ranking Points (e.g. 3 robot climb 2018, 4 Rotor Matches 2017)

18 2018 Game Overview

19 2018 Score Data

20 2018 Score Model - Normal Distribution

21 2018 Data Summary 2018 Probability of Win - CHS+CMP P(win score > 366 [kickoff guess]) = P(win score > 395 [80% win score]) = P(win score > 420 [match breakpoint]) = P(win score > 500 [max score]) = Kickoff Analysis 420 points breakpoint 366 points to win 80% matches 2018 Probability of Win - All FRC P(win score > 366 [kickoff guess]) = P(win score > 378 [80% win score]) = P(win score > 420 [match breakpoint]) = P(win score > 500 [max score]) = Fouls significantly inflate scores! 420 points -> 89% matches won 366 points -> 77% matches won Team Examples GDC District (4) CHS DCMP (5), 422 (11), 1599 (12) CMP ARC (6), 2016 (32)

22 2017 Game Overview

23 2017 Score Data

24 2017 Score Model - Normal Distribution

25 2017 Data Summary 2017 Probability of Win - CHS+CMP P(win score > 265 [first kickoff guess]) = P(win score > 315 [week 1 update]) = P(win score > 333 [some fuel added]) = P(win score > 355 [4 rotor match]) = Kickoff Analysis 265 points initial guess 315 points with easier climb 2017 Probability of Win - All FRC P(win score > 265 [first kickoff guess]) = P(win score > 315 [week 1 update]) = P(win score > 333 [some fuel added]) = P(win score > 355 [4 rotor match]) = % win point included just a little fuel. 315 points -> 89% matches won All FRC 315 points -> 73% matches won CHS+CMP Team Examples NMD District (1) 240 max avg points CHS DCMP (8) 311 max avg points CMP ARC (5), 217 (6), 2590 (12)

26 2016 Game Overview

27 2016 Score Data

28 2016 Score Model - Normal Distribution

29 2016 Data Summary 2016 Probability of Win - CHS+CMP P(win score > 118 [80% CHS win rate]) = P(win score > 124 [kickoff guess]) = P(win score > 165 [typical max score]) = Probability of Win - All FRC P(win score > 118 [80% CHS win rate]) = P(win score > 124 [kickoff guess]) = P(win score > 165 [typical max score]) = Kickoff Analysis 124 points 124 points -> 95% matches won All FRC 124 points -> 84% matches won CHS+CMP This is within 5% of the actual point for 80% win rate Team Examples CMD District (1) (only averaged 113) CHS DCMP (6) CMP GAL (18), 2468 (19)

30 P(Win??) > 0.8 How many points does my Alliance have to score to make this equation true? How do you figure this out at Kickoff?

31 Game Analysis

32 Overall Process Evaluate the Game Estimate Match Scores Develop Strategies Arena Scoring Opportunities Fouls Ranking Process Alliance Team Qualifying Playoffs Brainstorm methods to achieve the target match score WHAT will our robot do? Design Robot Develop Robot Requirements Select a Strategy Goal - Achieve Strategy Robot Requirements act as design guide Specify HOW robot will achieve strategy Demands Wishes Decision Matrix to select a strategy

33 Analyzing the Game READ THE RULES!!!!! Goal: Understand the game Examine every possible way to score points, no matter how obscure How much is each object/robot/action worth? Are there bonus points? Is there a maximum match score? Limited number of objects or do objects cycle? Examine every possible way to prevent opponents from scoring Stealing balls (2000), Capping robots (2004) Understand the ranking system Win-loss-tie, loser s score, own score plus double the loser s score, Coop points, bonus ranking points What are the key Fouls?

34 The Big Questions How many points does an alliance need to score to win a match? How many points does our team need to score to help our alliance win a match?

35 Strategy Development Examples 2011 (55-60 pts/match/team goal) 1 Ubertube top row and 2 top row Logos + 4th place minibot (58 points) 1 Ubertube top row and 1 top row Logo + 1st or 2nd place minibot (60 points) 1 Ubertube middle row and 2 middle row Logos + 1st place minibot (58 points) 2013 (~72 pts/match/team goal) 3 High Goal Auto Disc + 4 Pyramid Discs + 30 point Climb (68 points) 5 High Goal Auto Disc High Goal Discs + 10 point Climb (76 points) 3 High Goal Auto Disc + 20 High Goal Discs + 10 point Climb (88 points)

36 Strategy Development Examples 2014 (~165 pts/match goal) 15 Auto Mobility + 30 Auto Ball Ball + 10 Truss (165 points) 15 Auto Mobility + 30 Auto Ball Ball + 10 Truss (165 points) 15 Auto Mobility + 30 Auto Ball Ball (165 points) 15 Auto Mobility + 0 Auto Ball Ball (170 points) 2016 (~125 pts/match goal + 1 or 2 RP) 4 Auto Reach + 1 Auto Auto Boulder + 35 Cross + 5 Low High Challenge + 15 Scale (129 points) 1 Auto Reach + 2 Auto Auto Boulder Low Goals + 3 High Goals + 15 Challenge (120 points) 3 Auto Auto Boulder Low Goals + 3 High Goals + 15 Challenge (125 points) 3 Auto Auto Boulder Low Challenge + 15 Scale (113 points)

37 Strategy Development Examples 2017 (~ pts/match goal) 5 Auto Mobility + 60 Auto Gear + 80 Rotors (5 Gear Cycles) + 50 Climb (195 points) 5 Auto Mobility + 0 Auto Gear Rotors (6 Gear Cycles) + 50 Climb (175 points) 5 Auto Mobility + 60 Auto Gear + 40 Rotors (4 Gear Cycles) + 50 Climb (155 points) 5 Auto Mobility + 60 Auto Gear + 0 Rotors (0 Gear Cycles) + 50 Climb + 40 kpa Fuel (155 points)

38 Strategy Development Examples (2018) Theoretical Alliance Max Actions Occurred Auto Teleo p Time Held 3 Red Alliance Total Actions Occurred 15 3 Time Held Blue Alliance Total Actions Occurred 15 3 Time Held Total Run 5 per robot Scale 2 per own 2 per sec Switch 2 per own 2 per sec Scale 1 per own 1 per sec Switch 1 per own 1 per sec Vault 5 per cube Parking 5 per robot 0 0 Climbing 30 per robot Levitate 30 per robot Force 10 per use Boost 10 per use Dark Force -10 Total

39 Implementation

40 Next Steps Develop 3-6 Strategies Select 1 Strategy Define Robot Requirements

41 Game Strategy Thoughts Your team does not have to score all of the alliance points, just assist the alliance offense OFFENSIVE strategy should be priority Defensive maneuvers should not be primary strategy, but can be incorporated Best to drive and do one thing really well, than do many tasks averagely or worse Need to do the right thing well Example (2014) pickup and manipulate the ball well

42 If football is a game of inches robots is a game of seconds. Drive Have a purpose Practice makes perfect Acquire One touch and hold Don t let go until you want to Not enough resources, prototyping, time spent on this element 148 JVN Blog, 2363 Triple Helix Blog, 3847 Spectrum Blog Score Usually easier when acquire is robust (can t score without acquire) Too much time/resources spent here

43 Acquisition Accuracy Develop high accuracy, quick acting pickup mechanisms based on your abilities Scoring location is often secondary factor 2018 Switch/Vault Robot 2014 Low Goal Underused (31 points/cycle vs. 40 points/cycle) 2017 Gear/Hanging Robot

44 Cost-Benefit Analysis For each task you must compare the difficulty of accomplishment to the reward for doing so Balancing easier than scoring (2012) Small balls (2001) This is where the strategic value vs. coolness factor decision often pops up Vision tetra vs. hanging tetras (2005) Scoring Fuel vs. Gears (2017) The best tasks to perform are those which are relatively easy, yet provide big points Climbing the rope (2017) Remember denying your opponents 10 points is just as good as scoring 10 points (at least in terms of win/loss)

45 Prioritization Two separate lists Desired robot qualities Things like speed, power, agility, center of gravity Desired robot functionality The things you want your robot to be able to do Shoot balls, climb pyramid, traverse field Also consider necessary versus desired requirements Write these down and post so all can see At this point you can merge the two lists, and decide on a drive system and functionalities

46 Strategic Design Designing and building a cool robot is a lot of fun Designing and building a cool robot that does well in competition is even more fun Difficult to go through build without a goal Success in competition is clear choice It s not just about robots but there s a reason why the program is called FIRST Robotics COMPETITION Secondary objectives: aesthetics, design elegance, coolness, etc. Beware of the cool factor Can be fun, but sacrificing effectiveness hurts your partners

47 Simplicity & Golden Rules Golden Rule #1: Always build within your team s limits Evaluate your abilities and resources honestly and realistically Limits are defined by manpower, budget, experience Avoid building unnecessarily complex functions On the other hand, as you get more experienced start cautiously pushing a few boundaries Consider building limits/resources during the offseason instead of sacrificing performance Golden Rule #2: If a team has 30 units of robot and functions have maximum of 10 units capability, better to have 3 functions at 10/10 instead of 5 at 6/10 More important to do a few tasks well than a bunch of tasks mediocre

48 Once Built Re-evaluate strategy How close are you to your initial goals? Be realistic with what you can do Know the machine limits Watch how game is played at other events Game play evolves, very different each week Keep improving design Apply good ideas from what you watched Expand horizons Style of play is different in other parts of country

49 Competition Execution Scouting Goal is to collect useful data of actual performance to feed into match strategy Many resources available Effective Coaching Communication Negotiating and Consensus Building Leadership Visible team representative Teamwork with other Coaches during matches Alliance selection factor Robot Maintenance Checklists, Checklists, Checklists Preventive actions more effective than reactive fixes

50 Final Comments Read the rules!! Develop a clear, consistent strategy for how your robot will play the game Remember the Golden Rules The role of the coach cannot be understated Scouting is the easiest way to make your team more successful at competition Each FIRST match is like a high-speed game of chess: You need to have a well thought-out plan, but be prepared to counter your opponents moves Have fun!!