Evaluation of contextual queries Technical Report TR-FBK-DKM-2011-2 Loris Bozzato 1, Francesco Corcoglioniti 1,2, Mathew Joseph 1,2, and Luciano Serafini 1 1 Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy 2 DISI, University of Trento, Via Sommarive 14, 38123 Trento, Italy {bozzato,corcoglio,mathew,serafini@fbk.eu Abstract. This document evaluates the effectiveness of CKR queries through a benchmark of 32 queries on a FIFA World Cup knowledge base. A first set of queries has been chosen by the authors, with the goal of stressing the contexts structure and the context dependent knowledge, while the remaining queries have been taken from a generic website of quizzes about the FIFA World Cup. Queries are formulated against both the CKR and the flat OWL versions of the knowledge base. The document compares the different formulations and reports their corresponding query evaluation times. Table of Contents 1 Introduction..................................................................... 1 2 Test environment description....................................................... 2 3 Query formulation and execution................................................... 2 3.1 Queries whose scope belongs to one or few contexts............................... 3 3.2 Queries that scope over all/large number of contexts............................... 6 4 Analysis........................................................................ 13 4.1 Comparison of query performances for Flat vs CKR queries........................ 13 1 Introduction This paper reports an evaluation on the efficiency of query answering for the Contextualized Knowledge Repository framework CKR [2,3,4,5] with respect to the same task over a non contextualized RDF repository. The evaluation has been carried out as a comparative test over a common set of queries and a common dataset. In particular, the proposed queries were evaluated on the FIFA WC use case CKR and Flat models introduced in [1]. In order to verify the applicability of our framework a set of 32 query were given. Analyzed queries are reported in Table 1. Queries Q1 Q6 were proposed by the authors: their objective is to stress the contextual structure and intercontextual reasoning of both models. The remaining queries Q7 Q32 were taken from a generic website of quizzes about the FIFA world cup competition 3 : this choice has been taken in order to test behavior of both systems in over general questions. In the following, we will first report on the testing environment in which the comparison took place, We then proceed to present the formulation and execution times for all of the queries of our experiment. We conclude by comparing the overall results and briefly comment about their differences w.r.t. the two models. 3 http://www.funtrivia.com/quizzes/sports/fifa_world_cup/2010_south_africa.html
# Query Table 1. Selected queries. Q1 Which players played at least a match in FIFA WC 2010 but did not play in its knockout stage? Q2 Which teams participated to FIFA WC 2010? Q3 Which player has been substituted with whom and in which match? Q4 Which team(s) from group A reached the semi-finals of FIFA WC 2010? Q5 Which teams passed the quarter-finals but did not pass the semi-finals of FIFA WC 2010? Q6 Which players scored in a final of any FIFA WC edition? Q7 Which team won FIFA WC 2010? Which teams finished in 2nd and 3rd place? Q8 Which team won Group A? Q9 Which team received the most red/yellow cards during FIFA WC 2010? Q10 Which team scored the most goals during FIFA WC 2010? Q11 Which team suffered the most goals during FIFA WC 2010? Q12 Which teams were in Group A of FIFA World Cup 2010? Q13 Which teams passed the group stage with three wins? Q14 Which teams were eliminated from the group stage of FIFA WC 2010 without losing a match? Q15 Which teams passed the group stage of FIFA WC 2010 without suffering goals? Q16 Which teams from Africa advanced to FIFA WC 2010 knockout stage? Q17 Which team(s) won against the winner of FIFA WC 2010 during the competition? Q18 Which players are the top 5 scorers of FIFA WC 2010? Q19 Which player(s) were punished with a red card in FIFA WC 2010? Q20 Which player(s) substitute a substitute player in FIFA WC 2010? Q21 Which coach managed Greece in FIFA WC 2010? Q22 Which referee was in charge for the final match of FIFA WC 2010? Q23 Which match opened FIFA WC 2010? Q24 Which matches were decided by penalty shootout in FIFA WC 2010? Q25 Which matches of FIFA WC 2010 were won with at least two goals of difference? Q26 How many goals did Spain s player David Villa score in FIFA WC 2010? Q27 How many goals were scored in FIFA WC 2010? Q28 How many goals were scored in Group A of FIFA WC 2010? Q29 How many stadiums were used in the FIFA WC 2010? Q30 Which was the largest stadium used in FIFA WC 2010? Q31 What was the final score of Argentina vs Germany in FIFA WC 2010? Q32 What was the attendance of the FIFA WC 2010 final match? 2 Test environment description The set of experiments was carried out on a machine with a 2 x 2.16 GHz processor, 2.0 GB RAM with a Linux kernel. For both CKR and flat model, the Sesame 2.5.0 semantic repository 4 was used as the platform to which the closed datasets were loaded and queries were evaluated. The closure of data sets was computed with a sound and complete set of inference rules (see [3] for details about CKR closure). The size of the flat data set, after closure was computed (on which the flat queries were evaluated) was 426,622 triples. The size of the CKR data set after closure (on which the CKR queries were evaluated) was 1,119,699 triples. However, we remark that, since the dataset is fixed, the number of objects represented in both repositories is the same, with the measures reported in [1]. 3 Query formulation and execution In the following we present our formulation and result of executions for the queries we proposed in our evaluation. We divided the queries of Table 1 in two groups, on the base of their scope. The first group is composed by queries that are evaluated over a fixed and previously defined number of contexts, The 4 http://www.openrdf.org/ 2
second group is formed by queries evaluated over all or a large number of contexts. Inside each group we order the queries by ascending value of the difference between CKR and flat execution times. Note that, for the sake of clarity, prefixes are omitted in the SPARQL formulation of queries. Prefixes of flat queries are listed below, and must be prepended to queries in order for them to be executable: PREFIX top: <http://dkm.fbk.eu/fkb#> PREFIX sport: <http://dkm.fbk.eu/fkb/sport#> PREFIX fb: <http://dkm.fbk.eu/fkb/sport/football#> PREFIX wc: <http://dkm.fbk.eu/fkb/sport/football/fifa-wc#> PREFIX competitions: <http://dkm.fbk.eu/fkb/competitions#> PREFIX team: <http://dkm.fbk.eu/fkb/teams#> PREFIX person: <http://dkm.fbk.eu/fkb/persons#> PREFIX xsd: <http://www.w3.org/2001/xmlschema#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> The following prefixes are used by CKR queries: PREFIX xsd: <http://www.w3.org/2001/xmlschema#> PREFIX top: <http://dkm.fbk.eu/ckb#> PREFIX sport: <http://dkm.fbk.eu/ckb/sport#> PREFIX fb: <http://dkm.fbk.eu/ckb/sport/football#> PREFIX dim: <http://www.livememories.org/0.1/context-schema#> PREFIX topic: <http://www.livememories.org/0.1/topics-schema#> 3.1 Queries whose scope belongs to one or few contexts 1. QUERY Q4: Which team(s) from group A reached the semi-finals of FIFA WC 2010? Flat version (3.418 ms) [] a wc:semifinals ; top:hasparticipant?t ; top:hasdate?df. competitions:fifa-wc-2010- south-africa-group-a top:hasparticipant?t ; top:hasdate?dg. FILTER ( year(?df) = 2010 && year(?dg) = 2010 ) CKR version (0.57 ms) SELECT?x WHERE { CONTEXT?c1 {?x rdf:type top:team sports?c1 ; dim:topic topic:fifa-wc-group-a CONTEXT?c2{?x rdf:type top:team sports?c2 ; dim:topic topic:fifa-wc-semi-finals RESULTS: 1 team Uruguay. 2. QUERY Q5: Which teams passed the quarter-finals but did not pass the semi-finals of FIFA WC 2010? Flat version (1.409 ms)?s a wc:semifinals ; top:hasparticipant?t ; FILTER NOT EXISTS {?s sport:hasqualifiedparticipant?t CKR version (0.27 ms) SELECT?x WHERE { CONTEXT?c1 {?x rdf:type fb:passedteam?c1 dim:topic topic:fifa-wc-quarter-finals ; FILTER NOT EXISTS{ CONTEXT?c2 {?x rdf:type fb:passedteam?c2 dim:topic topic:fifa-wc-semi-finals ; RESULTS: 2 teams Uruguay and Germany. 3
3. QUERY Q6: Which players scored in a final of any FIFA WC edition? Flat version (1.186 ms) SELECT?p WHERE { [] a fb:goal ; fb:hasscorerplayer?p ; top:occoursin [ a wc:final ; top:hasdate?d CKR version (0.1592 ms) SELECT?p WHERE { [] a fb:goal ; fb:hasscorerplayer?p.?c dim:topic topic:fifa-wc-final-match. RESULTS: 1 player Spain s player Andres Iniesta Lujan. 4. QUERY Q7: Which team won FIFA WC 2010? Which were the teams finished in 2nd and 3rd place? Flat version (0.976 ms) SELECT?t1?t2?t3 WHERE { [] a wc:tournament ; sport:haswinner?t1 ; sport:hasrunnerup?t2 ; sport:hasthirdplace?t3 ; CKR version (0.156 ms) SELECT?t1?t2?t3 WHERE {?t1 a fb:winner.?t2 a fb:runnerup.?t3 a fb:thirdplace.?c dim:topic topic:fifa-wc-tournament ; RESULTS: Spain 1st, Netherlands 2nd, Germany 3rd. 5. QUERY Q8: Which team won Group A in FIFA WC 2010? Flat version (0.688 ms) competitions:fifa-wc-2010- south-africa-group-a sport:haswinner?t ; CKR version (0.1325 ms)?t a fb:winner?c dim:topic topic:fifa-wc-group-a ; RESULTS: Uruguay. 6. QUERY Q12: Which teams were in Group A of FIFA World Cup 2010? Flat version (0.626 ms) competitions:fifa-wc-2010- south-africa-group-a top:hasparticipant?t. FILTER (year(?d) = 2010 ) CKR version (0.195 ms) SELECT DISTINCT?x WHERE {?x a top:team sports?c dim:topic topic:fifa-wc-group-a ; RESULTS: 4 teams Uruguay, South Africa, France, Mexico. 4
7. QUERY Q16: Which teams from Africa advanced to FIFA WC 2010 knockout stage? Flat version (2.398 ms) [] a wc:knockoutstage ; top:hasparticipant?t ;?t fb:registeredwith fb:caf. CKR version (0.92 ms) SELECT DISTINCT?t WHERE { CONTEXT?c1 {?t a top:team sports?c1 dim:topic topic:fifa-wc-round-of-16 ; dim:starttime?d1 FILTER ( YEAR(?d1) = 2010 ) CONTEXT?c2 {?t fb:registeredwith fb:caf?c2 dim:topic topic:pro-football ; dim:starttime?d2. FILTER ( YEAR(?d2) = 2010 ) RESULTS: 1 team Ghana. 8. QUERY Q21: Which coach managed Greece in FIFA WC 2010? Flat version (5.242 ms) SELECT?x WHERE {?m sport:hascoach?x. team:greece sport:hasmember?m.?w a wc:tournament ; sport:hasparticipatingteammember?m. CKR version (0.28 ms) SELECT?x WHERE { team:greece sport:hascoach?x?c ; dim:topic topic:fifa-wc-tournament RESULTS: Otto Rehhagel. 9. QUERY Q22: Which referee was in charge for the final match of FIFA WC 2010? Flat version (0.972 ms) SELECT?r WHERE { [] a wc:match ; top:occoursin [ a wc:final ] ; fb:hasreferee?r. CKR version (0.085 ms) SELECT?r WHERE { [] fb:hasreferee?r?c dim:topic topic:fifa-wc-final-match RESULTS: Howard Webb. 10. QUERY Q32: What was the attendance of the FIFA WC 2010 final match? Flat version (1.111 ms) SELECT?n WHERE { top:occoursin [ a wc:final ] ; top:hasattendance?n. CKR version (0.094 ms) SELECT?n WHERE { [] top:hasattendance?n?c dim:topic topic:fifa-wc-final-match ; RESULTS: 84490 persons. 5
3.2 Queries that scope over all/large number of contexts 11. QUERY Q1: Which players played at least a match in FIFA WC 2010 but did not play in its knockout stage? Flat version (5286.44 ms) SELECT?p WHERE {?w a wc:tournament ; sport:hasparticipatingteammember [ sport:hasplayer?p FILTER EXISTS { [] a wc:match ; top:occoursin?w ; sport:hasparticipatingteammember [ sport:hasplayer?p FILTER NOT EXISTS { [] a wc:match ; top:occoursin [ a wc:knockoutstage ; top:occoursin?w ] ; sport:hasparticipatingteammember [ sport:hasplayer?p CKR version (61.87 ms) SELECT distinct?p WHERE { CONTEXT?c1 {?p fb:playsfor football [].?c1 dim:topic?topic;.?topic a topic:fifa-wc-match FILTER NOT EXISTS { CONTEXT?c2 {?p fb:playsfor football []?c2 dim:topic topic:fifa-wc-round-of-16;. RESULTS: an unsorted list of 368 players (omitted for the sake of brevity). 12. QUERY Q2: Which teams participated to FIFA WC 2010? Flat version (1.384 ms) [] a wc:tournament ; top:hasparticipant?t. CKR version (0.18 ms)?t a fb:team?c dim:topic topic:fifa-wc-tournament ; RESULTS: an unsorted list 32 teams, consisting of Algeria, Argentina, Australia, Brazil, Cameroon, Chile, Denmark, England, France, Germany, Ghana, Greece, Honduras, Italy, Ivory Coast, Japan, Korea DPR, Korea Republic, Mexico, Netherlands, New Zealand, Nigeria, Paraguay, Portugal, Serbia, Slovakia, Slovenia, South Africa, Spain, Switzerland, USA, Uruguay. 13. QUERY Q3: Which player has been substituted with whom and in which match? Flat version (17.912 ms) SELECT?m?pin?pout WHERE { [] a fb:substitution ; fb:hassubstituteout?pin ; fb:hassubstitutein?pout ; top:occoursin?m. CKR version (5.03 ms) SELECT?m?pin?pout WHERE { [] fb:hassubstitutein?pin ; fb:hassubstituteout?pout.?c dim:topic?m ;.?m a topic:fifa-wc-match RESULTS: 355 match, leaving player, entering player triples (omitted for the sake of brevity). 6
14. QUERY Q9: Which team received the most red/yellow cards during FIFA WC 2010? Flat version (10388.95 ms) SELECT?t (COUNT(?x) AS?n) WHERE {?w a wc:tournament ; sport:hasparticipatingteammember?m.?x a fb:refereecall; fb:hascalledplayer?p; top:occoursin?w.?t sport:hasmember?m.?m sport:hasplayer?p. GROUP BY?t ORDER BY DESC(?c) LIMIT 1 CKR version (13.10 ms) SELECT?t (COUNT(?x) AS?n) WHERE { {?x a fb:yellowcard ; fb:hascalledplayer [ fb:playsfor football?t ] UNION {?x a fb:redcard ; fb:hascalledplayer [ fb:playsfor football?t ]?c GROUP BY?t ORDER BY DESC(?n) LIMIT 1 RESULTS: Netherlands, with 23 red/yellow cards. 15. QUERY Q10: Which team scored the most goals during FIFA WC 2010? Flat version (13.987 ms) SELECT?t (COUNT(?g) AS?n) WHERE {?g a fb:goal ; fb:hasscorerteam?t ; top:occoursin [ a wc:tournament ; top:hasdate?d GROUP BY?t ORDER BY DESC(?g) LIMIT 1 CKR version (1.54 ms) SELECT?t (COUNT(?s) as?g) WHERE { graph?c {?s a fb:goal; fb:hasscorerteam?t?c ; dim:topic?top.?top a topic:fifa-wc-match GROUP BY?t ORDER BY DESC(?g) LIMIT 1 RESULTS: Germany, with 16 goals. 16. QUERY Q11: Which team suffered the most goals during FIFA WC 2010? Flat version (25.914 ms) SELECT?t (COUNT(?g) AS?n) WHERE { top:hasparticipant?t.?g a fb:goal ; top:occoursin?m ; fb:hasscorerteam?t1. FILTER ( year(?d) = 2010 &&?t!=?t1 ) GROUP BY?t ORDER BY DESC(?g) LIMIT 1 CKR version (8.288 ms) SELECT?t1 (COUNT(?s) as?g) WHERE {?s rdf:type fb:goal; fb:hasscorerteam?t. [] fb:hasparticipant?t1; FILTER (?t!=?t1)?c ; dim:topic?top.?top a topic:fifa-wc-match GROUP BY?t1 ORDER BY DESC(?g) LIMIT 1 RESULTS: Korea DPR, with 12 goals. 7
17. QUERY Q13: Which teams passed the group stage with three wins? Flat version (5.70 ms) SELECT?t (count(?m) AS?count) WHERE {?s a wc:firstgroupstage ; sport:hasqualifiedparticipant?t ; top:occoursin?s ; top:hasparticipant?t.?m sport:haswinner?t GROUP BY?t HAVING (?count=3) CKR version (1.06 ms) SELECT DISTINCT?team (COUNT(?t) AS?count) WHERE { CONTEXT?round16 {?team a top:team sports?round16 dim:topic topic:fifa-wc-round-of-16 ;. CONTEXT?groupmatch {?team a fb:winnerteam.?groupmatch dim:topic?t.?t a topic:fifa-wc-match GROUP BY?team HAVING (?count = 3 ) RESULTS: 2 teams Argentina and Netherlands. 18. QUERY Q29: How many stadiums were used in the FIFA WC 2010? Flat version (1.945 ms) SELECT (COUNT(DISTINCT?v) as?n) WHERE { [] a wc:match ; top:hasvenue?v. CKR version (1.11 ms) SELECT (COUNT(DISTINCT?v) as?c) WHERE {?x fb:hasvenue?v.?c ; dim:topic [a topic:fifa-wc-match] RESULTS: 10 stadiums. 19. QUERY Q15: Which teams passed the group stage of FIFA WC 2010 without suffering goals? Flat version (20.02 ms)?s a wc:firstgroupstage ; sport:hasqualifiedparticipant?t. FILTER NOT EXISTS { top:occoursin?s ; top:hasparticipant?t. [] a fb:goal ; top:occoursin?m ; fb:hasscorerteam?t1. FILTER (?t1!=?t ) CKR version (14.95 ms) SELECT DISTINCT?team WHERE { CONTEXT?c{?team a fb:team?c dim:topic topic:fifa-wc-tournament FILTER NOT EXISTS{ CONTEXT?groupmatch{ {?s rdf:type fb:goal.?s fb:hasscorerteam?team1. [] fb:hasparticipant?team; FILTER(?team1!=?team)?groupmatch dim:topic?t1.?t1 a topic:fifa-wc-match RESULTS: 2 teams Uruguay and Portugal. 8
20. QUERY Q17: Which team(s) won against the winner of FIFA WC 2010 during the competition? Flat version (0.706 ms)?w a wc:tournament ; sport:haswinner?t1.?match a wc:match ; top:occoursin?w ; sport:haswinner?t ; sport:hasrunnerup?t1. CKR version (0.22 ms) SELECT DISTINCT?winner WHERE { CONTEXT?groupmatch {?champion a fb:loserteam.?winner a fb:winnerteam.?groupmatch dim:topic?t ;.?t a topic:fifa-wc-match CONTEXT?tournament {?champion a fb:winner?tournament dim:topic topic:fifa-wc-tournament;. RESULTS: 1 team Switzerland. 21. QUERY Q18: Which players are the top 4 scorers of FIFA WC 2010? Flat version (13.266 ms) SELECT?p (COUNT(?g) AS?n) WHERE {?g a fb:goal ; fb:hasscorerplayer?p ; top:occoursin [ a wc:tournament ; top:hasdate?d GROUP BY?p ORDER BY DESC(?n) LIMIT 4 CKR version (1.34 ms) SELECT?p (COUNT(?g) as?n) WHERE {?g fb:hasscorerplayer?p.?c GROUP BY?p ORDER BY DESC(?n) LIMIT 4 RESULTS: W. Sneijder (Netherlands), D. Villa (Spain), D. Forlan (Uruguay), T. Muller (Germany). 22. QUERY Q19: Which player(s) were punished with a red card in FIFA WC 2010? Flat version (1.128 ms) SELECT DISTINCT?p WHERE { [] a fb:redcard ; fb:hascalledplayer?p ; top:occoursin [ a wc:tournament ; top:hasdate?d ] CKR version (1.28 ms) SELECT DISTINCT?p WHERE { [] a fb:redcard ; fb:hascalledplayer?p.?c RESULTS: 9 players F. Melo (Brazil), V. Behrami (Switzerland), R. Costa (Portugal), L. Suarez (Uruguay), Y. Gourcuff (France), S. Kaita (Nigeria), H. Kewell (Australia), I. Khune (South Africa), T. Cahill (Australia). 9
23. QUERY Q20: Which player(s) substitute a substitute player in FIFA WC 2010? Flat version (13.80 ms) SELECT DISTINCT?p WHERE {?s1 top:occoursin?m ; fb:hassubstitutein?p ; fb:hassubstituteout?p1.?s2 top:occoursin?m ; fb:hassubstitutein?p1. FILTER (?s1!=?s2 && year(?d) = 2010 ) CKR version (11.57 ms) SELECT DISTINCT?playerIn1 WHERE { CONTEXT?match { [] a fb:substitution ; fb:hassubstituteout?player ; fb:hassubstitutein?playerin. [] a fb:substitution ; fb:hassubstituteout?playerin ; fb:hassubstitutein?playerin1.?match RESULTS: 2 players A. Komac (Slovenia), R. Afolabi (Nigeria). 24. QUERY Q23: Which match opened FIFA WC 2010? Flat version (2.251 ms) SELECT?t1?t2?d WHERE { fb:hashometeam?t1 ; fb:hashostteam?t2 ; ORDER BY?d LIMIT 1 CKR version (3.51 ms) SELECT DISTINCT?t1?t2?d WHERE { [] fb:hasdate?d.?t1 a fb:hometeam.?t2 a fb:hostteam.?c ; dim:topic?x.?x a topic:fifa-wc-match ORDER BY ASC(?d) LIMIT 1 RESULTS: South Africa vs Mexico. 25. QUERY Q24: Which matches were decided by penalty shootout in FIFA WC 2010? Flat version (0.757 ms) SELECT?m WHERE { [] a fb:penaltyshootout ; top:occoursin?m. CKR version (1.82 ms) SELECT DISTINCT?date?p1?p2 WHERE {?y fb:hasperiod [ a fb:penaltyshootout]; fb:hasdate?date.?p1 a fb:hometeam.?p2 a fb:hostteam FILTER(?p1!=?p2)?c ; dim:topic [a topic:fifa-wc-match] RESULTS: 2 matches - Paraguay vs Japan, Uruguay vs Ghana. 10
26. QUERY Q28: How many goals were scored in Group A of FIFA WC 2010? Flat version (1.893 ms) SELECT (COUNT(?g) AS?n) WHERE { competitions:fifa-wc-2010- south-africa-group-a?g a fb:goal ; top:occoursin competitions:fifa-wc-2010- south-africa-group-a. FILTER (year(?d) = 2010 ) CKR version (0.43 ms) SELECT (Count(?x) as?c) WHERE { CONTEXT?groupA {?x a fb:goal?groupa ; dim:topic [ topic:iscoveredby topic:fifa-wc-group-a; a topic:fifa-wc-match ] RESULTS: 11 goals. 27. QUERY Q26: How many goals did Spain s player David Villa score in FIFA WC 2010? Flat version (14.011 ms) SELECT (count(?g) AS?n) WHERE {?g a fb:goal ; fb:hasscorerplayer person:david-villa-sanchez-338 ; top:occoursin [ a wc:tournament ; top:hasdate?d CKR version (7.91 ms) SELECT (count(?g) AS?n) WHERE { CONTEXT?c1 {?g a fb:goal ; fb:hasscorerplayer person:david-villa-sanchez-338?c1 ; dim:topic [a topic:fifa-wc-match] RESULTS: 5 goals. 28. QUERY Q27: How many goals were scored in FIFA WC 2010? Flat version (10.896 ms) SELECT (COUNT(?g) AS?n) WHERE {?g a fb:goal ; top:occoursin [ a wc:tournament ; top:hasdate?d CKR version (3.43 ms) SELECT (COUNT(?g) AS?n) WHERE {?g a fb:goal?c ; dim:topic [ a topic:fifa-wc-match]. RESULTS: 145 goals 29. QUERY Q25: Which matches of FIFA WC 2010 were won with at least two goals of difference? Flat version (4.86 ms) SELECT?m?s1?s2 WHERE { fb:hasscorehome?s1 ; fb:hasscorehost?s2. FILTER ( year(?d) = 2010 && abs(?s1 -?s2) >= 2 ) CKR version (11.178 ms) SELECT DISTINCT?t?s1?s2 WHERE { CONTEXT?match { [] a fb:match; fb:hasscorehome?s1; fb:hasscorehost?s2. FILTER ( abs(xsd:integer(?s1) - xsd:integer(?s2)) >= 2 )?match dim:starttime?time ; dim:topic?t. FILTER ( year(?time) = 2010 ) RESULTS: 17 matches (omitted for the sake of brevity). 11
30. QUERY Q14: Which teams were eliminated from the group stage of FIFA WC 2010 without losing a match? Flat version (1.66 ms)?s a wc:firstgroupstage ; top:hasparticipant?t. FILTER NOT EXISTS {?s sport:hasqualifiedparticipant?t FILTER NOT EXISTS { [] a wc:match ; top:occoursin?s ; sport:hasrunnerup?t. CKR version (20.91 ms) SELECT DISTINCT?team WHERE { CONTEXT?c{?team a fb:team?c dim:topic topic:fifa-wc-tournament; FILTER NOT EXISTS{ CONTEXT?round16 {?team a top:team sports?round16 dim:topic topic:fifa-wc-round-of-16 FILTER NOT EXISTS{ CONTEXT?groupmatch {?team a fb:loserteam.?groupmatch dim:topic?t.?t a topic:fifa-wc-match RESULTS: 1 team New Zealand. 31. QUERY Q30: Which was the largest stadium used in FIFA WC 2010? Flat version (3.301 ms) SELECT?v?s WHERE { [] a wc:match ; top:hasvenue?v.?v top:haspeoplecapacity?s. ORDER BY DESC(?c) LIMIT 1 CKR version (45.149 ms) SELECT DISTINCT?v?s WHERE { CONTEXT?c1 { [] fb:hasvenue?v?c ; dim:topic [ a topic:fifa-wc-match] CONTEXT?c2 {?v sport:haspeoplecapacity?s?c2 dim:topic topic:sports ORDER BY DESC(?s) LIMIT 1 RESULTS: the Johannesburg stadium. 32. QUERY Q31: What was the final score of Germany vs Argentina in FIFA WC 2010? Flat version (0.8132 ms) SELECT?s1?s2 WHERE { top:hasparticipant team:germany, team:argentina; fb:hasscorehome?s1 ; fb:hasscorehost?s2. CKR version (2.19 ms) SELECT?s1?s2 WHERE { team:argentina a fb:hometeam. team:germany a fb:hostteam. [] fb:hasscorehome?s1 ; fb:hasscorehost?s2; a fb:match.?c RESULTS: Argentina 0 - Germany 4. 12
4 Analysis In the following we propose a graphical representation of the comparison between the results of query execution times over the CKR and flat models. We conclude with a brief comment on the differences in these results. 4.1 Comparison of query performances for Flat vs CKR queries In Figure 1 we report a graphical comparison of execution times for fixed-context queries (queries whose scope belongs to one or few contexts), while in Figure 2 we depict the comparison with respect to multicontext queries (queries whose scope to all or many contexts). Flat model CKR model 6 5 4 3 2 1 0 q6 q9 q4 q10 q2 q3 q1 q5 q7 q8 Fig. 1. Queries whose scope belongs to one or few contexts 100000 Flat model CKR model 10000 1000 100 10 1 0.1 q14 q11 q16 q19 q13 q27 q15 q23 q21 q28 q17 q29 q31 q24 q18 q26 q30 q12 q22 q32 q25 q20 Fig. 2. Queries whose scope belongs to all or many contexts 13
From the results shown by the two evaluation graphs we can verify that, while in all of the fixedcontext queries CKR queries outperform their flat counterpart, in the multi-context queries which ask for a search over a large number of contexts (as in the case of queries 30 32), contextualized queries may have sensibly worst execution times. In fact, in a contextualized knowledge base, queries that target only one or few contexts may be evaluated more efficiently than in a global model, since only part of the information stored in the system must be accessed and that information is already available because precomputed at CKR closure time. On the other hand, queries spanning multiple contexts have to constantly change their scope in order to be answered. For example, let us consider query Q30 (one of the most demanding queries for the CKR): since in CKR model the relation of a match and its venue is stored in the context of type Match and the relation between each venue and its capacity are listed in the general context of sports, there needs to be a search that spans two context to obtain the match - venue - capacity relation of matches and then one needs to sort on descending order and extract the top value. On the other hand, in the flat approach this initial join that is required is not cross-contextual, thus the execution time is lower. To this respect, contexts are similar to views in a database system, with CKR closure corresponding to view materialization. As in databases, this advantage does not come for free: the price to pay is the redundancy and computational cost introduced by CKR closure, but it may be more than justified by the resulting run-time savings, depending on the particular application. References 1. Loris Bozzato, Francesco Corcoglioniti, Martin Homola, and Luciano Serafini. Evaluation of Contextual Modelling in the FIFA World Cup Use Case. Technical Report TR-FBK-DKM-2011-2, Fondazione Bruno Kessler, Trento, Italy, 2011. https://dkm.fbk.eu/index.php/resources. 2. Martin Homola, Luciano Serafini, and Andrei Tamilin. Modeling contextualized knowledge. In Proc. of International Workshop on Context, Information And Ontologies (CIAO10), Portugal, 2010. 3. Mathew Joseph and Luciano Serafini. Simple Reasoning for Contextualized RDF Knowledge. In WoMo 2011, 2011. 4. Luciano Serafini and Martin Homola. Contextual representation and reasoning with description logics. In DL 2011, volume 745 of CEUR-WS, 2011. 5. Luciano Serafini and Martin Homola. Contextualized knowledge repositories for the semantic web. Journal of Web Semantics, Special Issue: Reasoning with context in the Semantic Web, 2012. To appear, preprint available: https: //dkm.fbk.eu/images/1/1b/jws-serafini-homola-ckr-2012.pdf. 14