Some applications of relative entropy in additive combinatorics

doi:10.1017/9781108332699.010

9 - Some applications of relative entropy in additive combinatorics

Published online by Cambridge University Press: 21 July 2017

Edited by

David Manlove and

Julia Wolf: Affiliation:
University of Bristol
Anders Claesson: Affiliation:
University of Iceland, Reykjavik
Mark Dukes: Affiliation:
University College Dublin
Sergey Kitaev: Affiliation:
University of Strathclyde
David Manlove: Affiliation:
University of Glasgow
Kitty Meeks: Affiliation:
University of Glasgow

Book contents

Get access

Summary

Abstract

This survey looks at some recent applications of relative entropy in additive combinatorics. Specifically, we examine to what extent entropy-increment arguments can replace or even outperform more traditional energy-increment strategies or alternative approximation arguments based on the Hahn-Banach theorem.

Introduction

Entropy has a long history as a tool in combinatorics. Starting with a well-known estimate for the sum of the first few binomial coefficients, some of the classical applications include Spencer's theorem that six standard deviations suffice, which states that given n finite sets, there exists a two-colouring of the elements such that all sets have discrepancy at most; a proof of the Loomis-Whitney inequality, which gives an upper bound on the volume of an n-dimensional body in Euclidean space in terms of its (n - 1)-dimensional projections; or Radhakrishnans proof [33] of Bregman's theorem on the maximum permanent of a 0/1 matrix with given row sums. For a beautiful introduction to these fascinating applications, as well as an extensive annotated bibliography, see [10].

There are other more recent results in additive combinatorics in particular where the concept of entropy has played a crucial role. Notable examples include Fox's improvement [7] of the bounds in the graph removal lemma (see also [30], which appeared in the proof-reading stages of this article); Szegedy's information-theoretic approach [41] to Sidorenko's conjecture (see also the blog post [14] by Gowers); Tao's solution [45] to the Erds discrepancy problem (see the discussion [44] on Tao's blog).

Since the above developments appear to be well captured by discussions online, we shall not cover them in any detail here. Instead we shall focus on a particular strand of recent results in additive combinatorics that could all be described as “approximation theorems” of a certain kind.

The text naturally splits into five parts. To start with we give a very brief introduction to the concept of entropy and its variants, in particular relative entropy (also known as Kullback-Leibler divergence). In Section 3, we state and prove a rather general sparse approximation theorem due to Lee [28].

Information

Type: Chapter
Information: Surveys in Combinatorics 2017 , pp. 409 - 442
DOI: https://doi.org/10.1017/9781108332699.010 [Opens in a new window]
Publisher: Cambridge University Press
Print publication year: 2017

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

[1] Boaz, Barak, Moritz, Hardt, and Satyen, Kale. The uniform hardcore lemma via approximate Bregman projections. In Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1193–1200. SIAM, Philadelphia, PA, 2009.

[2] Thomas, Bloom. A quantitative improvement for Roth's theorem on arithmetic progressions. J. Lond. Math. Soc. (2), 93(3):643–663, 2016.Google Scholar

[3] Mei-Chu, Chang. A polynomial bound in Freiman's theorem. Duke Math. J., 113(3):399–419, 2002.Google Scholar

[4] David, Conlon, Jacob, Fox, and Yufei, Zhao. The Green-Tao theorem: an exposition. EMS Surv. Math. Sci., 1(2):249–282, 2014.Google Scholar

[5] David, Conlon, Jacob, Fox, and Yufei, Zhao. A relative Szemerédi theorem. Geom. Funct. Anal., 25(3):733–762, 2015.Google Scholar

[6] Thomas, Cover and Joy, Thomas. Elements of information theory. Wiley Series in Telecommunications. John Wiley & Sons, Inc., New York, New York, USA, 1991.

[7] Jacob, Fox. A new proof of the graph removal lemma. Ann. of Math. (2), 174(1):561–579, 2011.Google Scholar

[8] Ehud, Friedgut. An information-theoretic proof of a hypercontractive inequality. arXiv, 1504.01506, April 2015.

[9] Alan, Frieze and Ravi, Kannan. Quick approximation to matrices and applications. Combinatorica, 19(2):175–220, 1999.Google Scholar

[10] David, Galvin. Three tutorial lectures on entropy and counting. arXiv, 1406.7872, June 2014.

[11] Timothy, Gowers. A new proof of Szemerédi's theorem for arithmetic progressions of length four. Geom. Funct. Anal., 8(3):529–551, 1998.Google Scholar

[12] Timothy, Gowers. A new proof of Szemerédi's theorem. Geom. Funct. Anal., 11(3):465–588, 2001.Google Scholar

[13] Timothy, Gowers. Decompositions, approximate structure, transference, and the Hahn-Banach theorem. Bull. Lond. Math. Soc., 42(4):573–606, 2010.Google Scholar

[14] Timothy, Gowers. Entropy and Sidorenko's conjecture — after Szegedy. Personal blog, https://gowers.wordpress.com/2015/11/18/entropy-and-sidorenkosconjecture-after-szegedy/, November 2015.

[15] Timothy, Gowers and Julia, Wolf. Linear forms and quadratic uniformity for functions on Fnp. Mathematika, 57(2):215–237, 2012.Google Scholar

[16] Ben, Green. Some constructions in the inverse spectral theory of cyclic groups. Combin. Probab. Comput., 12(2):127–138, 2003.Google Scholar

[17] Ben, Green. Spectral structure of sets of integers. In Fourier analysis and convexity, pages 83–96. Birkhäuser Boston, Boston, MA, 2004.

[18] Ben, Green. Finite field models in additive combinatorics. In Bridget S Webb, editor, Surveys in combinatorics 2005, pages 1–27. Cambridge Univ. Press, Cambridge, Cambridge, 2005.

[19] Ben, Green. Roth's theorem in the primes. Ann. of Math. (2), 161(3):1609–1636, 2005.Google Scholar

[20] Ben, Green. Montréal notes on quadratic Fourier analysis. In Additive combinatorics, pages 69–102. Amer. Math. Soc., Providence, RI, 2007.

[21] Ben, Green and Tom, Sanders. Boolean functions with small spectral norm. Geom. Funct. Anal., 18(1):144–162, 2008.Google Scholar

[22] Ben, Green and Terence, Tao. An inverse theorem for the Gowers U3(G) norm. Proc. Edinb. Math. Soc. (2), 51(1):73–153, 2008.Google Scholar

[23] Ben, Green and Terence, Tao. The primes contain arbitrarily long arithmetic progressions. Ann. of Math. (2), 167(2):481–547, 2008.Google Scholar

[24] Ben, Green and Terence, Tao. Linear equations in primes. Ann. of Math. (2), 171(3):1753–1850, 2010.Google Scholar

[25] Russell, Impagliazzo. Hard-core distributions for somewhat hard problems. In FOCS ‘07. 48th Annual IEEE Symposium on Foundations of Computer Science, 2007., pages 538–545. IEEE, 1995.

[26] Russell, Impagliazzo, Cristopher, Moore, and Alexander, Russell. An entropic proof of Chang's inequality. SIAM J. Discrete Math., 28(1):173–176, 2014.Google Scholar

[27] Solomon, Kullback and Richard, Leibler. On information and sufficiency. Ann. Math. Stat., 22(1):79–86, 1951.Google Scholar

[28] James, Lee. Covering the large spectrum and generalized Riesz products. arXiv, 1508.07109, August 2015.

[29] Chi-Jen, Lu, Shi-Chun, Tsai, and Hsin-Lung, Wu. Complexity of hardcore set proofs. Comput. Complexity, 20(1):145–171, 2011.Google Scholar

[30] Guy, Moshkovitz and Asaf, Shapira. A sparse regular approximation lemma. arXiv, 1610.02676, October 2016.

[31] Ryan, O'Donnell. Lecture 16: The hypercontractivity theorem. Personal website, https://www.cs.cmu.edu/õdonnell/booleananalysis/lecture16.pdf.

[32] Sean, Prendiville. Four variants of the Fourier-analytic transference principle. arXiv, 1509.09200, September 2015.

[33] Jaikumar, Radhakrishnan. An entropy proof of Bregman's theorem. J. Combin. Theory Ser. A, 77(1):161–164, 1997.Google Scholar

[34] Omer, Reingold, Luca, Trevisan, Madhur, Tulsiani, and Salil, Vadhan. Dense subsets of pseudorandom sets. Electronic Colloquium on Computational Complexity, 45:1–33, 2008.Google Scholar

[35] Omer, Reingold, Luca, Trevisan, Madhur, Tulsiani, and Salil, Vadhan. New proofs of the Green-Tao-Ziegler dense model theorem: an exposition. arXiv, 0806.0381, June 2008.

[36] Sheldon, Ross. A first course in probability. Macmillan Publishing Co., Inc., New York; Collier Macmillan Publishers, London, 1976.

[37] Alex, Samorodnitsky. Low-degree tests at large distances. In STOC'07—Proceedings of the 39th Annual ACM Symposium on Theory of Computing, pages 506–515. ACM, New York, New York, New York, USA, 2007.

[38] Tom, Sanders. On Roth's theorem on progressions. Ann. of Math. (2), 174(1):619–636, 2011.Google Scholar

[39] Tom, Sanders. On the Bogolyubov–Ruzsa lemma. Anal. PDE, 5(3):627–655, 2012.Google Scholar

[40] Ilya, Shkredov. On sets of large exponential sums. Dokl. Math., 74(3):860–864, 2006.Google Scholar

[41] Balazs, Szegedy. An information theoretic approach to Sidorenko's conjecture. arXiv, 1406.6738, June 2014.

[42] Terence, Tao. Moser's entropy compression argument. Personal blog, https://terrytao.wordpress.com/2009/08/05/mosers-entropycompression-argument/, August 2009.

[43] Terence, Tao. Higher order Fourier analysis, volume 142 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2012.

[44] Terence, Tao. Entropy and rare events. Personal blog, https://terrytao.wordpress.com/2015/09/20/entropy-and-rareevents/, September 2015.

[45] Terence, Tao. The logarithmically averaged Chowla and Elliott conjectures for two-point correlations. Forum Math. Pi, 4:e8–36, 2016.Google Scholar

[46] Terence, Tao and Tamar, Ziegler. The primes contain arbitrarily long polynomial progressions. Acta Math., 201(2):213–305, 2008.Google Scholar

[47] Madhur, Tulsiani and Julia, Wolf. Quadratic Goldreich-Levin theorems. SIAM J. Comput., 43(2):730–766, 2014.Google Scholar

[48] Salil, Vadhan and Jia, Zheng. A uniform min-max theorem with applications in cryptography. In Advances in Cryptology – CRYPTO 2013, pages 93–110. Springer Berlin Heidelberg, Berlin, Heidelberg, 2013.

[49] Julia, Wolf. Finite field models in arithmetic combinatorics – ten years on. Finite Fields Appl., 32:233–274, 2015.Google Scholar

[50] Jiapeng, Zhang. On the query complexity for Showing Dense Model. Electronic Colloquium on Computational Complexity, 38, 2011.Google Scholar

[51] Yufei, Zhao. An arithmetic transference proof of a relative Szemerédi theorem. arXiv, 1307.4959, July 2013.

[52] Jia, Zheng. A uniform min-max theorem and characterizations of computational randomness. PhD thesis, 2014.

Accessibility standard: Unknown

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the PDF of this chapter is currently unknown and may be updated in the future.