We prove the existence of infinitely many distinct phases along the entire boundary of the Razborov triangle. The proofs involve bootstraps, showing that an optimal graphon near the boundary (which always exists by compactness) has to be approximated by a particular graphon using increasingly tight norms, eventually showing that the optimal graphon is unique with a simple structure called multipodal, defined by parameters having a specified asymptotic behavior as we approach the boundary. The "worth" functional, developed in Chapter 2, is a key tool. We then develop order parameters to distinguish these phases. We also relate uniform random graph models to exponential random graph models (ERGMs), showing that due to lack of appropriate convexity almost the entire Razborov triangle is "ERGM-invisible", meaning that one cannot describe graphs with those edge and triangle values using an ERGM. This extends similar results of Chatterjee and Diaconis.

This chapter provides an introduction to key concepts in optimization with motivating applications. We begin with a categorization of different optimization problems, based on the objective function, to be maximized or minimized, and constraint equations that limit the set of feasible solutions. Next we introduce two major concepts in optimization: convexity and the gradient. Convex problems are essentially solved, at least numerically. Linear programming and quadratic programming are both convex. Nonconvex problems are more challenging. However, there is a shift in optimization, where it is increasingly possible to solve nonconvex problems. The gradient is another fundamental concept, providing information about the geometry of the optimization landscape, and which direction locally improves the solution. We then show how Lagrange multipliers are an intuitive way of handling constraints. We conclude with three major applications: inverse problems, control theory, and machine learning. Throughout, we give examples to illustrate how to formulate an optimization problem and solve it algorithmically.

This chapter focuses on the core concepts of optimization theory and its application in data science and AI. It begins with a review of differentiable functions of several variables, including the gradient and Hessian matrices, and key results like the Chain Rule and the Mean Value Theorem. The chapter then introduces optimality conditions for unconstrained optimization, explaining first-order and second-order conditions, and the role of convexity in ensuring global optimality. A detailed discussion of the gradient descent algorithm is provided, including its convergence analysis under different assumptions. The chapter concludes with an application to logistic regression, demonstrating how gradient descent is used to optimize the cross-entropy loss function in a supervised learning context. Practical Python examples are integrated throughout to illustrate the theoretical concepts.

This chapter introduces the numerical convex optimization problem that minimize a certain objective function subject to some constraints. We also introduce an efficient algorithm for solving such problems.

Chapter 10 delves into the quantification of quantum resources, an essential aspect of quantum resource theories that determines the value of quantum states for specific applications. It begins by defining resource measures and investigating their fundamental properties such as monotonicity under free operations and convexity. The chapter discusses distance-based resource measures, which quantify how far a given quantum state is from the set of free states. Such measures often utilize divergences and metrics explored in earlier chapters. Techniques to compute the relative entropy of a resource are also covered.

To refine resource measures, the chapter introduces the concept of smoothing, which considers small deviations from the ideal state to make the measures more robust against perturbations. This approach is crucial in single-shot scenarios where finite resources are available. Furthermore, the chapter examines resource monotones and support functions, offering a comprehensive framework for the theoretical and practical assessment of quantum resources.

A mini course on the functional analytic approach to convexity-based optimization theory. Familiarity with this appendix is a prerequisite for the main text.

This chapter introduces the important idea of a vector through the example of bundles of goods. The dot product of two vectors is defined and it is shown how a budget constraint can be expressed in terms of dot product. It is explained how, in order to rank bundles according to a particular consumer's preference, we can use a utility function. Indifference curves are defined as the contours of the utility function. Linear and convex combinations and the concept of a convex set are explained. The utility maximisation problem -- to maximise utility subject to a budget constraint -- is explored and the relevance of convexity is emphasised.

The structure of equilibrium thermodynamics, in harmony with statistical mechanics, does not contain a time asymmetry, except when it is trivially supplemented with one.

This appendix collects a review of the calculus and analysis in one and several variables that the reader should be familiar with. Notions of convergence, continuity, differentiability and integrability are recalled here.

This chapter serves two purposes: it introduces several essential concepts of linear and nonlinear functional analysis that will be used in subsequent chapters and, as an illustration of them, studies the problem of unconstrained minimization of a convex functional. All the necessary notions of existence, uniqueness, and optimality conditions are presented and analyzed. Preconditioned gradient descent methods for strongly convex, locally Lipschitz smooth objectives in infinite dimensions are then presented and analyzed. A general framework to show linear convergence in this setting is then presented. The preconditioned steepest descent with exact and approximate line searches are then analyzed using the same framework. Finally, the application of Newton’s method to the Euler equations is discussed. The local convergence is shown, and how to achieve global convergence is briefly discussed.