Book contents
- Frontmatter
- Contents
- 1 Introduction and Overview
- 2 Preface to the Planning Chapters
- 3 Deterministic Decision Processes
- 4 Markov Chains
- 5 Markov Decision Processes and Finite Horizon Dynamic Programming
- 6 Discounted Markov Decision Processes
- 7 Episodic Markov Decision Processes
- 8 Linear Programming Solutions
- 9 Preface to the Learning Chapters
- 10 Reinforcement Learning: Model Based
- 11 Reinforcement Learning: Model Free
- 12 Large State Spaces: Value Function Approximation
- 13 Large State Space: Policy Gradient Methods
- 14 Regret Minimization
- Appendix A Dynamic Programming
- Appendix B Ordinary Differential Equations
- Reference
- Index
8 - Linear Programming Solutions
Published online by Cambridge University Press: 31 July 2026
Book contents
- Frontmatter
- Contents
- 1 Introduction and Overview
- 2 Preface to the Planning Chapters
- 3 Deterministic Decision Processes
- 4 Markov Chains
- 5 Markov Decision Processes and Finite Horizon Dynamic Programming
- 6 Discounted Markov Decision Processes
- 7 Episodic Markov Decision Processes
- 8 Linear Programming Solutions
- 9 Preface to the Learning Chapters
- 10 Reinforcement Learning: Model Based
- 11 Reinforcement Learning: Model Free
- 12 Large State Spaces: Value Function Approximation
- 13 Large State Space: Policy Gradient Methods
- 14 Regret Minimization
- Appendix A Dynamic Programming
- Appendix B Ordinary Differential Equations
- Reference
- Index
Summary
This chapter presents an alternative optimization-based approach to solving MDPs through linear programming (LP) formulations. The chapter develops LP formulations for both finite-horizon and discounted infinite-horizon MDPs. The key innovation is representing policies through occupancy measures. The primal LP maximizes return subject to flow constraints, while the dual LP has an elegant interpretation in terms of value functions. The LP approach offers theoretical insights through duality.
Keywords
Information
- Type
- Chapter
- Information
- Reinforcement Learning Foundations , pp. 83 - 90Publisher: Cambridge University PressPrint publication year: 2026
Accessibility standard: WCAG 2.2 A
Why this information is here
This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.Accessibility Information
The PDF of this chapter
complies with version 2.2 of the
Web Content Accessibility Guidelines
(WCAG), offering more comprehensive accessibility measures for a broad range
of users and meets the basic
(A)
level of WCAG compliance, addressing essential accessibility barriers.
Content Navigation
Table of contents navigation
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Allows you to navigate directly to chapters, sections, or non‐text items through a linked table of contents, reducing the need for extensive scrolling.
Index navigation
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Provides an interactive index, letting you go straight to where a term or subject appears in the text without manual searching.
Reading Order & Textual Equivalents
Single logical reading order
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
You will encounter all content (including footnotes, captions, etc.) in a clear, sequential flow, making it easier to follow with assistive tools like screen readers.
Short alternative textual descriptions
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.
You get concise descriptions (for images, charts, or media clips), ensuring you do not miss crucial information when visual or audio elements are not accessible.