Communication Amid Uncertainty

Transcription

1 Communication Amid Uncertainty Madhu Sudan Harvard University Based on joint works with Brendan Juba, Oded Goldreich, Adam Kalai, Sanjeev Khanna, Elad Haramaty, Jacob Leshno, Clement Canonne, Venkatesan Guruswami, Badih Ghazi, Pritish Kamath, Ilan Komargodski and Pravesh Kothari. May 10, 2016 MIT TOC: Communication Amid Uncertainty 1 of 28

2 Obligatory Sales Pitch Most of communication theory [a la Shannon, Hamming]: Built around sender and receiver perfectly synchronized. Most Human communication does not assume perfect synchronization. And increasingly device-device communication can also not rely on this. Can we build a mathematical theory of imperfectly synchronized communication? What are questions/answers? May 10, 2016 MIT TOC: Communication Amid Uncertainty 2 of 28

3 Context in Communication In most forms of communication: Sender + Receiver share (huuuge) context In human comm: Language, news, Social In computer comm: Protocols, Codes, Distributions Helps compress communication Perfectly shared Can be abstracted away. Imperfectly shared What is the cost? How to study? May 10, 2016 MIT TOC: Communication Amid Uncertainty 3 of 28

4 Communication Complexity The model xx (with shared randomness) ff: xx, yy Σ RR = $$$ yy Alice Usually studied for lower bounds. This talk: CC as +ve model. Bob CCCC ff = # bits exchanged by best protocol ff(xx, yy) w.p. 2/3 May 10, 2016 MIT TOC: Communication Amid Uncertainty 4 of 28

5 Aside: Easy CC Problems [Ghazi,Kamath,S 15] Problems with large inputs and small communication? Equality testing: Protocol: EEEE xx, yy = 1 xx = yy; CCCC EEEE Fix = OO ECC 1 EE: 0,1 nn 0,1 NN ; Shared randomness: ii NN ; xx = (xx HH kk xx, yy = 1 Δ xx, yy kk; CCCC HH kk Exchange pppppppp = OO(kk kk Protocol: 1,, xx nn ) logeekk) xx[huang ii yy, EE= yyyy ii ; etal.] 1,, yy nn Accept Use common iff EE xx randomness ii = EE yy ii. kk xx, yy = 1 wt xx, wt yy kk & to ii hash ss. tt. xx nn ii = yy [kk xx, ii = 2 1. ] yy xx ii yy ii Hamming distance: Small set intersection: CCCC kk = OO(kk) [Håstad Wigderson] Gap (Real) Inner Product: Main Insight: Unstated xx, yy Rphilosophical nn ; xx 2, yy 2 = 1; contribution If GGof CC NN a 0,1la nn Yao:, then Communication GGGGPP cc,ss xx, yy = 1 with if xx, a yy focus cc; = ( only 0 if xx, yy ss; EEneed GG, xx to GG, determine yy = xx, yy ff xx, yy ) can be more effective 1 (shorter than xx, HH xx, HH yy, II(xx; yy) ) CCCC GGGGPP cc,ss = OO 2 ; [Alon, Matias, Szegedy] cc ss ii May 10, 2016 MIT TOC: Communication Amid Uncertainty 5 of 28

6 Modelling Shared Context + Imperfection Many possibilities. Ongoing effort. Alice+Bob may have estimates of xx and yy. More generally: xx, yy correlated. Knowledge of ff function Bob wants to compute may not be exactly known to Alice! Shared randomness Alice + Bob may not have identical copies May 10, 2016 MIT TOC: Communication Amid Uncertainty 6 of 28

7 Part 1: Uncertain Compression May 10, 2016 MIT TOC: Communication Amid Uncertainty 7 of 28

8 Classical (One-Shot) Compression Sender and Receiver have distribution PP [NN] Sender/Receiver agree on Encoder/Decoder EE/DD Sender gets XX [NN] ; Sends EE XX Receiver gets YY = EE(XX) ; Decodes XX = DD YY Requirement: XX = XX (always) Performance: EE XX PP EE XX Trivial Solution: EE XX PP EE XX = log NN Huffman Coding: Achieves EE XX PP EE XX HH PP + 1 May 10, 2016 MIT TOC: Communication Amid Uncertainty 8 of 28

9 The (Uncertain Compression) problem Design encoding/decoding schemes (EE/DD) s.t.: Sender has distribution PP NN Receiver has distribution QQ [NN] Sender gets XX [NN] ; Sends EE(PP, XX) to receiver. Receiver gets YY = EE(PP, XX); Decodes XX = DD(QQ, YY) Want: XX = XX (provided PP, QQ close), While minimizing EEPP XX PP xx [ EE PP, XX ] Δ PP, QQ Δ if log Δ for all xx QQ xx [Juba,Kalai,Khanna,S. 11] Motivation: Models natural communication? May 10, 2016 MIT TOC: Communication Amid Uncertainty 9 of 28

10 Solution (variant of Arith. Coding) Uses shared randomness: Sender+Receiver rr 0,1 Use rr to define sequences dictionary rr 1 1, rr 1 2, rr 1 3, rr 2 1, rr 2 2, rr 2 3, rr NN 1, rr NN 2, rr NN 3, Analysis: EE rr LL = 2Δ + log 1 PP(xx) EE xx,rr LL = 2Δ + HH(PP) Sender sends prefix of rr xx [1 LL] as encoding of xx Receiver outputs argmax zz QQ zz rr zz 1 LL = rr xx [1 LL] Want: LL rr zz 1 LL = rr xx 1 LL QQ zz < QQ xx ; (QQ zz > QQ xx rr zz 1 LL rr xx [1 LL]) (PP zz > 4 Δ PP xx rr zz 1 LL rr xx [1 LL]) May 10, 2016 MIT TOC: Communication Amid Uncertainty 10 of 28

11 Implications Coding scheme reflects the nature of human communication (extend messages till they feel unambiguous). Reflects tension between ambiguity resolution and compression. Larger the ((estimated) gap in context), larger the encoding length. Entropy is still a valid measure! The shared randomness assumption A convenient starting point for discussion But is dictionary independent of context? This is problematic. May 10, 2016 MIT TOC: Communication Amid Uncertainty 11 of 28

12 Deterministic Compression: Challenge Say Alice and Bob have rankings of N movies. Rankings = bijections ππ, σσ NN NN ππ ii = rank of i th movie in Alice s ranking. Further suppose they know rankings are close. ii NN : ππ ii σσ ii 2. Bob wants to know: Is ππ 1 1 = σσ 1 1 How many bits does Alice need to send (noninteractively). With shared randomness OO(1) Deterministically? With Elad Haramaty: OO(log nn) OO 1? OO(log NN)? OO(log log log NN)? May 10, 2016 MIT TOC: Communication Amid Uncertainty 12 of 28

13 Open Questions (Compression) Best Deterministic Uncertain Compression? Best known: O HH PP + log log NN Dependence on NN? Leading constant? Does Private Randomness help? Can we do OO HH PP + log log log NN? Movie ranking problem: Dependence on NN necessary? Compression length w. Det/Priv. Randomness grows with NN May 10, 2016 MIT TOC: Communication Amid Uncertainty 13 of 28

14 Part 2: Imperfectly Shared Randomness May 10, 2016 MIT TOC: Communication Amid Uncertainty 14 of 28

15 Model: Imperfectly Shared Randomness Alice rr ; and Bob ss where rr, ss = i.i.d. sequence of correlated pairs rr ii, ss ii ii ; rr ii, ss ii { 1, +1}; EE rr ii = EE ss ii = 0; EE rr ii ss ii = ρρ 0. Notation: iiiirr ρρ (ff) = cc of ff with ρρ-correlated bits. cccc(ff): Perfectly Shared Randomness cc. pppppppp ff : cc with PRIVate randomness Starting point: for Boolean functions ff cccc ff iiiirr ρρ ff pppppppp ff cccc ff + log nn What if cccc ff log nn? E.g. cccc ff = OO(1) = iiiirr 1 ff = iiiirr 0 ff ρρ ττ iiiirr ρρ ff iiiirr ττ ff May 10, 2016 MIT TOC: Communication Amid Uncertainty 15 of 28

16 Imperfectly Shared Randomness: Results Model first studied by [Bavarian,Gavinsky,Ito 14] ( Independently and earlier ). Their focus: Simultaneous Communication; general models of correlation. They show iiiiii Equality = OO 1 (among other things) Our Results:[Canonne,Guruswami,Meka,S 15] Generally: cccc ff kk iiiiii ff 2 kk Converse: ff with cccc ff kk & iiiiii ff 2 kk May 10, 2016 MIT TOC: Communication Amid Uncertainty 16 of 28

17 Equality Testing (our proof) Key idea: Think inner products. Encode xx XX = EE(xx);yy YY = EE yy ;XX, YY 1, +1 NN xx = yy XX, YY = NN xx yy XX, YY NN/2 Estimating inner products: Building on sketching protocols Alice: Picks Gaussians GG 1, GG tt R NN, Sends ii tt maximizing GG ii, XX to Bob. Bob: Accepts iff GG ii, YY 0 Analysis: OO ρρ (1) bits suffice if GG ρρ GG Gaussian Protocol May 10, 2016 MIT TOC: Communication Amid Uncertainty 17 of 28

18 General One-Way Communication Idea: All communication Inner Products (For now: Assume one-way-cc ff kk) For each random string RR Alice s message = ii RR 2 kk Bob s output = ff RR (ii RR ) where ff RR : 2 kk 0,1 W.p. 2 3 over RR, ff RR ii RR is the right answer. May 10, 2016 MIT TOC: Communication Amid Uncertainty 18 of 28

19 General One-Way Communication For each random string RR Alice s message = ii RR 2 kk Bob s output = ff RR (ii RR ) where ff RR : 2 kk 0,1 W.p. 2 3, ff RR ii RR is the right answer. Vector representation: ii RR xx RR 0,1 2kk (unit coordinate vector) ff RR yy RR 0,1 2kk (truth table of ff RR ). ff RR ii RR = xx RR, yy RR ; Acc. Prob. XX, YY ; XX = xx RR RR ; YY = yy RR RR Gaussian protocol estimates inner products of unit vectors to within ±εε with OO ρρ 1 εε 2 communication. May 10, 2016 MIT TOC: Communication Amid Uncertainty 19 of 28

20 Two-way communication Still decided by inner products. Simple lemma: KK AA kk, KK BB kk R 2kk convex, that describe private coin k-bit comm. strategies for Alice, Bob s.t. accept prob. of ππ AA KK AA kk, ππ BB KK BB kk equals ππ AA, ππ BB Putting things together: Theorem: cccc ff kk iiiiii ff OO ρρ (2 kk ) May 10, 2016 MIT TOC: Communication Amid Uncertainty 20 of 28

21 The Tightness Example Sparse Gap Inner Product: Alice xx 0,1 nn ; wt xx 2 kk nn (Sparse) Bob yy 1,1 nn ; Decide: xx, yy.9 2 kk nn or xx, yy 0 Shared randomness protocol: Alice communicates random bit ii s. t. xx ii = 1 Bob outputs yy ii Lower bound: Invariance principle, Gap Hamming Distance May 10, 2016 MIT TOC: Communication Amid Uncertainty 21 of 28

22 Open Questions (I.S.R.) Exponential gap for total function? ff: 0,1 nn 0,1 nn 0,1, with cccc ff kk, & iiiiii cccc ff 2 kk Level of correlation? ff: 0,1 nn 0,1 nn 0,1,?, with iiiirr.9 ff kk, & iiiirr.1 ff 2 kk Does interaction help if randomness is not perfectly shared? May 10, 2016 MIT TOC: Communication Amid Uncertainty 22 of 28

23 Part 3: Uncertain Functionality May 10, 2016 MIT TOC: Communication Amid Uncertainty 23 of 28

24 Model Bob wishes to compute ff(xx, yy); Alice knows gg ff; Alice, Bob given gg, ff explicitly. (Input size ~ 2 nn ) Modelling Questions: What is? Is it reasonable to expect to compute ff xx, yy? Answers: E.g., ff xx, yy = ff xx? Can t compute ff xx, yy without communicating xx Assume xx, yy 0,1 nn 0,1 nn uniformly. ff δδ gg if δδ ff, gg δδ. Suffices to compute h xx, yy for h εε ff May 10, 2016 MIT TOC: Communication Amid Uncertainty 24 of 28

25 Results - 1 Thm [Ghazi,Komargodski,Kothari,S.]: ff, gg, μμ s.t. cc 1wwwwww μμ,.1 ff, cc 1wwwwww μμ,.1 gg = 1 and δδ μμ ff, gg = oo(1); but uncertain communication = Ω( nn); Thm [GKKS]: But not if xx yy (in 1-way setting). (2-way, even 2-round, open!) Main Idea: Canonical 1-way protocol for ff: Alice + Bob share random yy 1, yy mm 0,1 nn. Alice sends ff xx, yy 1,, ff xx, yy mm to Bob. Protocol used previously but not as canonical. Canonical protocol robust when ff gg. May 10, 2016 MIT TOC: Communication Amid Uncertainty 25 of 28

26 Open Questions (Uncertain Functionality) What happens when xx, yy correlated? [Ghazi-S.] functions where cc grows with II(xx; yy) Exact dependence? Is II xx; yy right measure? What happens to communication with multiple rounds? Two rounds? What is the right task that captures uncertainty in natural communication? May 10, 2016 MIT TOC: Communication Amid Uncertainty 26 of 28

27 Conclusions Positive view of communication complexity: Communication with a focus can be effective! Context Important: New layer of uncertainty. New notion of scale (context LARGE) Importance of oo(log nn) additive factors. Many uncertain problems can be solved without resolving the uncertainty (which is a good thing) Many open directions+questions May 10, 2016 MIT TOC: Communication Amid Uncertainty 27 of 28

28 Thank You! May 10, 2016 MIT TOC: Communication Amid Uncertainty 28 of 28