FIRST LEGO League Judged Sessions Guide Core Values Judging Session 2 Core Values Judging Session Format 2 Core Value Activity 2 Core Values Poster 2 Questions and Answers 3 Tips for Preparing for Core Values Judging 3 Example Core Value Judging Sessions 3 Core Values Judging Rubric 4 Project Judging Session 5 Project Judging Session Format 5 Project Presentation 5 Questions and Answers 5 Tips for Preparing for Project Judging 5 Example Core Value Judging Sessions 5 Project Judging Rubric 6 Robot Design Judging Session 7 Robot Design Judging Session Format 7 Robot Design Discussion 7 Tips for Preparing for Robot Design Judging 7 Example Core Value Judging Sessions 7 Robot Design Judging Rubric 8 Team Information Sheet 9 What is a Team Information Sheet? 9 What do we do with a Team Information Sheet? 9 How to Create a Team Information Sheet 9 Sample Team Information Sheet Template 10 Additional Information Judging, Deliberating, and Scoring 11 FLL Advancement Policy: Update for 2017 HydroDynamics 15 1
Core Values Judging Session Note to those reading this as a printed document: all hyperlinks in this section are for YouTube videos. Search YouTube using the title or team name as a guide. Core Values Judging Session Format 3-5 minutes Core Value Activity (Described below) 3-5 minutes Core Values Poster and Presentation (Described below) 3-5 minutes Questions and Answers (Described below) Core Value Activity The Core Value Judges will give your team a task/activity. Every team is given the same task. The judges want to see how the team works together, not whether or not they can complete the task. Example Activity: Straw Transfer Materials : Bendable straws (one per team member), A cardboard ring (4-5cm diameter) or lightweight ring of any type Instructions to Team: Take a straw and bend it. Place the long end in your mouth. One student should add the ring to their straw. Now transfer the ring to each person without the use of any hands What the Judges are Looking For: How did the team communicate and coordinate to move the ring from person to person? Core Values Poster Why Have a Core Values Poster? To help the Core Values judges learn more about your team and its unique story, teams should complete a Core Values tri-fold or poster. The topics highlighted on the poster are typically the most difficult to discuss during judging sessions. The poster is intended to help your team consider in advance how best to share these elements with the judges. How to Make a Core Values Poster Board for FLL (FLL Team Astromechs) Suggested Template (see picture on the right.) The poster layout can follow this template. The overall size of the poster should be as shown, although it may be smaller if required by travel needs. The poster may also be rolled or assembled on site. 2
Defined areas: There are four areas on the poster where specific information should be provided. These are the MOST IMPORTANT areas of the poster and should be the main focus of your work. Discovery: provide examples from the season about things your team discovered that were more about gaining knowledge than about gaining an advantage in the competition or winning an award. Integration: provide specific examples of how your team applied Core Values and other things you learned through FLL, to situations outside of FLL. Tell the judges your stories about how you integrate new knowledge, skills and abilities into your everyday life. Inclusion: describe how your team incorporated ideas from everyone and made each team member feel like a valued part of the team. Help the judges understand how working together you accomplished more than you could by working alone. Coopertition: describe how your team honors the spirit of friendly competition, including any assistance provided or received from other teams. Tell the judges how your team helps each other, and other teams, prepare for and approach potentially stressful competition experiences during your season. The Middle: The area in the middle of the poster is for your team to highlight anything else you choose about the remaining criteria on the Core Values rubric: Team Spirit, Respect or any of the Teamwork elements. You may also include examples of how your team has fun or shares with others how amazing science, technology, engineering and math can be. Feel free to use your creativity and expression! Questions and Answers This time is reserved for the Core Values judges to ask questions about the Poster and the season. Possible Questions could include but are not limited to: What is the biggest challenge your team have overcome? How do you work out problems as a team? What is your favorite thing about being a team? What have you learned this year? Tips for Preparing for Core Values Judging Determine how much time to spend preparing your poster and in participating in Core Value activities. A realistic amount of time for most teams is 4 to 8 hours (initial discussion, poster and activities). The poster is NOT intended to be on the same scale as your Project presentation. Remember the judges are only interested in your team s story, not fancy graphics. You can ask a parent to help a few of your students create a Core Values poster. Excellent Extras: Include a picture of your Core Value Poster or picture of your team on your Team Introduction Page. (See Team Introduction Page section) Example Core Value Judging Sessions FLL team LMS Robotics Nature s Fury FLL team LMS Robotics World Class FLL team Lego Mages FLL team FLL Eyases Core Values Judging Rubric is included on the next page. 3
4
Project Judging Session Note to those reading this as a printed document: all hyperlinks in this section are for YouTube videos. Search YouTube using the title or team name as a guide. Project Judging Session Format 5-7 minutes Project Presentation 5-7 minutes - Project Q&A Project Presentation Teams will have 5 minutes to present their project to the Judges. The five minutes includes setup time. Teams can be creative about how they present their project. Presentation styles can include a skit, PowerPoint presentation, song, puppet show or whatever the team can imagine. Presentations must be live, they CANNOT be pre-recorded. It is expected that the youth will handle all setup. The project should cover the elements outlined in the Project Rubric (see page ): Research, Innovative Solution, and Presentation. Questions and Answers Each team will have 5 minutes of Q&A with the judges. The team will be asked about components on the rubric, items in their project notebook (optional), and how they found inspiration for their idea. Tips for Preparing for Project Judging Low-tech or no-tech is the best tech during the Project Presentation. The five minutes presentation time includes setup time, and if audiovisual equipment fails, there is no do over. Your five minutes is your five minutes, no exceptions. It is expected that the youth will handle all setup. Find a way to include every team member. If someone doesn t want to speak, they can hold a sign, be a prop, etc. You can ask parents to help with costumes, props, practices, makeup, etc. Excellent Extras: Include a picture of something that presents your Project for your Team Introduction Page. (See Team Introduction Page section) Example Core Value Judging Sessions FLL team Team 4197 Splat FLL team Team 544 FLL team Cybots FLL team Team 2766 Is Lost Project Judging Rubric is included on the next page. 5
6
Robot Design Judging Session Note to those reading this as a printed document: all hyperlinks in this section are for YouTube videos. Search YouTube using the title or team name as a guide. Robot Design Judging Session Format This session is 10-12 minutes long. The team is expected to run at least 1 mission and discuss some of their programming. There will be a mission table set up for the team to use. Robot Design Discussion Teams can have a prepared presentation, but keep in mind the judges want a conversation, not a speech. Use your prepared material as a guide and reference, not as a presentation. The judges are ascertaining the team s understanding of their robot, programming and strategy. A memorized presentation does not provide the opportunity to delve into the team s understanding. Teams are judged in three areas: Mechanical Design, Programming and Strategy & Innovation. Tips for Preparing for Robot Design Judging Have the robot on and ready to run when you enter the room. Plan to run 1 mission at a minimum. It is a good idea to have a 2nd mission ready, in case there is time to show a 2nd mission. Bring a printout of the programming to share with the judges, or show them your programs using a device. Bonus: It is never a bad idea to keep a technical binder documenting your robot design process and changes. If you keep a binder, be sure to bring it to Robot Design Judging. If you do bring it, make sure team members are familiar with it and can address questions about it. Make sure that everyone has something they can add to the conversation. The judges want to hear from more people than just the lead programmer or builder. Examples of conversation contributions: I built this attachment so we could We decided to combine these to missions since I worked on building the bot I haven t programmed until this year. I learned how to We thought this was a good robot design because... Excellent Extras Include a picture of your robot for your Team Introduction Page. (See Team Introduction Page section) 2017 Hydrodynamics How to Annotate Your Mission code For Robot Design Judging Example Core Value Judging Sessions FLL team tyrk0820 FLL team Ice Falcons FLL team Nano Beasts Robot Design Judging Rubric is included on the next page. 7
8
Team Information Sheet What is a Team Information Sheet? Team Information sheets are used to help judges remember teams. Information Sheets are not required, but are very helpful during Judge Deliberations. What do we do with a Team Information Sheet? Take 3 copies of the information sheet into each judging room Core Values, Project and Robot Design. The team will leave copies of the Information Sheet with the judges. How to Create a Team Information Sheet You can make your Team Information Sheet look anyway you wish, as long as it includes the following information: FLL Team Number Team Name: School/Organization/City Robot Name Team Members Names Coaches Names Mentors Names Core Values Information Project Information Robot Design Information Team Photo You can search the internet for FLL Team Information Sheet Templates for different Information Sheet styles. A sample template is explained on the right and included for your convenience. 9
10
Judging, Deliberating, and Scoring Original Document by FL Roboticss Judging Process Before judge deliberation begins, the Head Judge or Judge Advisor should check for conflicts of interest. If a judge has any relationship or alliance with a certain team, he or she should refrain from the judging process and award determination regarding that team. Judge groups meet with FLL Teams Each judging team assesses teams and determines top teams (one or two) that they have seen Each Head Judge hosts a deliberation session to determine top teams for each award for call backs if being used Judges provide reasons why they would like to bring team forward for a call back or consider for an award. Judges receive feedback from Head Referee and Volunteer Coordinator on any teams they are considering to help provide rounded picture of team, as well as highlight any teams that they reviewed that judges may want to consider further. Judges determine if any of the remaining teams should be added to the list and note reasons Head Judge will check in frequently with judges to ensure they are satisfied with the choices. Callbacks, additional review or discussion of top teams occurs After callbacks or additional review by judges, a ranking of the top five or six candidates for each category are determined through the same process. Robot Design Judges review Robot Performance scores before final deliberations. For Robot Dependability Award winners, teams in the top third of Robot Performance scores should be considered for the final award determination. This ensures that robots that perform only one or two missions perfectly are not given top honors, and that the consistently strong robots are the winners. Final Deliberations and Champion s Award Determination After the Champion s Award Winner is chosen by process listed below, teams that do not win a Champion s Award are considered for awards under the categories where they were nominated initially. No team may be awarded more than two awards. The only time that two awards may be given is if one is the Robot Performance Award. Judges use criteria to determine the winner of the Champion s Award. The team s ranking for Project, Teamwork and Robot Design, in conjunction with the Robot Performance Score and the assessment of FLL Values, are all equally important in determining the award. All four required competition categories and Core Values are of equal importance in FLL. Head Judge of each award area provides ranking of top teams in their award area, and presents information on why each team was chosen. Judge Advisor identifies which teams appeared as top teams across all awards categories. Judge Advisor runs the voting process. Final deliberations start with the Champion s Award winners, then moves on to other awards categories Top candidates for each award are discussed before voting Each judge votes for as many teams as they like 11
For each team, judges are asked by show of hands if they would be happy if team received first place This process is repeated for each team For each team the number of votes it receives for first place are recorded The team with the most votes is given first place, team with the next highest votes is given second and if third place is awarded then team with the third highest amount of votes is given third place. Before final award assignments are made, the Judge Advisor checks with the judges to ensure that judges are comfortable with results Repeat this process for each required award, then continue with optional awards. Once votes are tallied, if a team is listed under several awards team is given an award based on the highest number of votes it received, with required awards given priority. Example: Team listed in three areas with votes as follows: Team Spirit second Project Presentation second Robot Design first Team would receive first place for Robot Design and then be removed from other awards and team with next highest amount of votes would be given second place. If the team ranks first in two categories, as follows: Team Spirit first Project Presentation first Robot Design second The team will be given the Project Presentation award, because it is the required award. The team with the next highest votes for Team Spirit would receive that award. If the team ranks first in two required categories, as follows: Team Spirit second Project Presentation first Robot Design first Project and Robot Design judges would discuss the team s performance in each category, to determine which of the two awards is most appropriate for the team. The team will win the award the judges deem most suitable, and the next ranked team will win the award for the other category. Judge Advisor should check with judges to ensure they are comfortable with the choices. This judging process is designed to allow for normalization of judging scores. Some judges naturally score higher; others lower. This process allows the judges to consider teams in terms of their overall ranking or achievement, rather than according to their judging scores. Awards Descriptions for Required Awards at Championship and Qualifying Tournaments Champion s Award (not required at Qualifying Tournaments) The Champion s Award is the most prestigious award that any team can win. It celebrates the ultimate success of the FIRST mission and FLL values. It measures how the team members design, program and score with their robot, the quality of their research and project presentation, their ability to work as a team, solve problems, and demonstrate respect and gracious professionalism. To be considered for the Champion s Award, teams must perform well in both technical and team performance categories, which are equally weighted. 12
Once teams are selected, judges convene and review the results of the teams FLL values assessment, as well as their overall impressions of each team s performance and participation at the tournament. Using these additional parameters for determination, judges decide which team receives this highest honor. The team that wins the Champion s Award will not receive any additional awards in the other categories, with the possible exception of the Robot Performance award. Team Performance Awards Project Award (Depending on tournament, may be broken into three distinct awards.) FLL presents the Project Award to the team whose quality research, innovative solutions, and creative presentation best reflect an in-depth understanding of the various scientific disciplines and issues involved with the Challenge Project. Tournaments may choose to break the Project Award into three separate awards: Research Quality The use and understanding of diverse resources to formulate an in-depth and thorough explanation of the team s point of view and solution to the Challenge Project. Innovative Solution Thought-provoking and innovative resolution, including how and why it was chosen. Creative Presentation An imaginative, creative presentation demonstrating the team s research and solution. Teamwork Award Teamwork is critical to succeed in FIRST LEGO League and is the key ingredient in any team effort. FLL presents this award to the team that best demonstrates extraordinary enthusiasm, an exceptional partnership, and FLL values. Technical Awards Robot Design Award (Depending on tournament, may be broken into three distinct awards.) Judges look for teams whose work stands out for innovation and/or dependability. To assess innovation, the judges watch the robots work and look for things that make them say Wow! They interview team members to reveal the less obvious unique and inventive ideas. To assess dependability, the judges interview the teams to learn what solid principles and best practices they used to reduce variability and errors, with preference to robots that best back it up throughout the matches. Tournaments may choose to break the Robot Design Award into three separate awards: Innovative Robot Award This award goes to the team best demonstrating its ability to think out of the box. Judges consider the most original robot design approach to solving the Challenge missions. Robot Dependability Award The best designs make products that are consistent over time and dependable under changing conditions. This award goes to the team whose robot most consistently and dependably works every time. Programming Award FLL presents this award to the team that understands outstanding programming principles. This team s robot demonstrated programming mastery. 13
Robot Performance Award This award goes to the team whose robot achieves the best score on the competition field, or in the elimination round, at the tournament. There are several options judges use to determine the winner: If no elimination round is held, the team with the single highest score receives the trophy. If elimination rounds are held, the team(s) whose robot achieved the highest score in the elimination round receives the trophy. If elimination rounds are held, the highest scoring team (using high score between two teams as the factor for advancement to the next round) receives the award. If elimination rounds are held, participants are the top teams according to each team s single highest score after three robot performance rounds. Special Recognition Awards (Not required, but recommended) Outstanding Volunteer Award This award honors the dedication of the volunteer(s) whose assistance and devotion helps change the lives of children. Adult Coach/Mentor Award Many teams reach significant milestones of success thanks to their close relationship with an adult mentor. This award goes to the coach or mentor whose wisdom, guidance, and devotion are most clearly evident in her team s discussion with the judges. Young Adult Mentor Award FLL presents this award to the young adult, high school or college mentor, whose support, impact, inspiration, and guidance are most clearly evident in the team s discussion with the judges. Optional Awards (At the discretion of the FLL Partner) Against All Odds Award This award goes to the team that improvises and overcomes a difficult situation while still making a respectable showing. We can overcome incredible odds if we never give up, no matter what! (Some tournaments prefer to call this award the Persistence Award.) Team Spirit Award Some teams really know how to have fun. This award goes to the team that most enthusiastically demonstrates a commitment to getting others to see how accessible, fun, and rewarding science and technology can be, especially when you are part of a great team. Judges Award During the course of competition the judges may encounter a team whose unique efforts, performance, or dynamics merit recognition. Some teams have a story that sets them apart in a unique way. Sometimes a team is so close to winning an award that the judges choose to give special recognition to the team. This award gives the judges the freedom to recognize the most remarkable teams for which a standard award does not exist. Judges should provide details on why they chose to recognize this team to be shared during the awards ceremony. Universal Design Award This award is given to the team that incorporates Universal Design principles into its project solution to benefit people of all ages and abilities. Core Principles of Universal Design For all, regardless of physical or mental limitations, the design is: simple, helpful, flexible, easy to use and understandable, minimizes errors and problems when used incorrectly. 14
FLL Advancement Policy Updated for 2017 HYDRO DYNAMICS Original Document provided by FIRST. Check FIRST website for updates to this policy. Advancement Percentages For regions with qualifying tournaments, FIRST LEGO League recommends: 10% is the minimum percentage of teams that should advance Around 30% is the recommended percentage of teams to advance 50% is the maximum percentage of teams that should advance After considering the advancement recommendations, the Partner is required to decide on a region-wide advancement percentage. The advancement percentage should be the same (or as close as possible) for all qualifying tournaments in the same level of events. To determine how many teams will advance from each event, tournaments multiply the regional advancement percentage by the total number of teams at the event. (Advancement %) * (total #of planned teams at event) = Approx. # of advancing teams Example: A qualifying tournament is planned for 12 teams. The advancement percentage rate for this region is 30%. Multiply the advancement percentage by the number of teams or.30 x 12 = 3.6. Rounding up, 4 teams will advance from this event. In a region with multiple qualifiers, it may sometimes be necessary to not follow standard rounding rules in this procedure. A region with 110 teams is planning for 36 teams to attend the Championship. 36 / 110 = 33% region-wide advancement percentage. They have 4 qualifiers: 20 teams, 24 teams, 32 teams, 34 teams. 20 teams x.33 = 6.6, 7 teams advance 24 teams x.33 = 7.92, 8 teams advance 32 teams x.33 = 10.56, 10 teams advance **Rounded down to create a total of 36 advancing teams** 34 teams x.33 = 11.22, 11 teams advance Team Eligibility for Advancement Teams are eligible for advancement if they meet the following criteria as required by the FIRST LEGO League Global Standards and the annual Challenge. Specifically, teams must: Follow the FIRST LEGO League Participation Rules Have no disqualifying (Red-level) Core Values behaviors Have a rank in the top 75% of Robot Game scores at the event Robot Performance Advancement Hurdle (Updated for 2017 HYDRO DYNAMICS) In order to ensure they meet a minimum level of achievement in Robot Performance, advancing teams must have a rank in the top 75% of robot game scores at the event. (In other words, the bottom quartile is not eligible to advance.) The number of teams eligible for advancement always applies to the planned number of teams. The planned number of teams is defined as the number reasonably expected to attend an event; the teams have registered, paid, and/or otherwise confirmed their intent to be at the event. This planned number must be announced to teams in advance of the event. The number of teams eligible for advancement must not be changed for no-shows after teams have been notified of planned number of teams expected at the event, including at any time during the tournament day. 15
Example: 12 teams are expected to attend an event, but one team does not arrive. Teams still must be ranked in the top 75% out of the planned 12 teams (75% * 12 teams = Top 9 teams are eligible to advance). The Robot Performance Hurdle for advancement does not impact the hurdle for Champion s Awards. Teams at all tournaments must be ranked in the top 40% of Robot Game scores to be considered for Champion s Awards. Advancing Team Selection Procedures All advancing teams are selected using the Champion s Award criteria; the highest achieving teams who meet the 75% minimum Robot Performance Hurdle will advance. FIRST LEGO League does not require a specific method of ranking teams within each judging room or within the overall judging area. Ranking teams through numeric compilation of rubrics is not recommended, but if done, it must be used as an addition to a discussion-based deliberations process and should be just one of many indicators of team performance. Advancing teams are selected using the golf score approach OR using official FIRST LEGO League Judging Lite or Advancement Calculator tools. The golf score method requires adding the team s area ranks to create an overall rank, then removing any teams below the hurdle. When selecting teams for advancement, Champion s Award winning teams are always chosen first. Example: 4 teams are advancing from a 12-team tournament and is awarding a 1st place Champion s Award. The Champion s Award winner must be ranked in the top 5 Robot Performance scores (40% hurdle) and the advancing teams must be ranked in the top 9 scores (75% hurdle). The Judges create a golf score rank by adding the ranks; the top five teams combined ranks are 4, 6, 7, 8, and 12. The team with a combined rank of 4 finished at 10th place in the Robot Game Scores. Since they are not ranked in the top 9 teams (75% hurdle) at the event eligible to advance, they are removed from the advancing teams. The first team chosen to advance (with combined rank of 6) is the 1st place Champion s Award winning team; they were required to be in the top 5 scores (40% hurdle). The remaining 3 teams advancing must be in the top 9 scores (75%) but are not required to be in the top 5 (40%). In addition to the Champion s Award winning team, the remaining 3 teams (with combined ranks of 7, 8, and 12) are selected to advance. 16