This chapter covers infinite horizon MDPs under the long-run average reward criterion. The average reward criterion applies to non-terminating systems in which decisions are made frequently and in perpetuity. These models pose numerous analytical challenges because the existence and structure of the average reward depend on the limiting properties of underlying Markov chains. Markov chain structure and its implications for the average reward model are discussed. The concepts of bias and gain are formalized, and connections to the discounted and expected total reward models are drawn. The Bellman equation and its properties are presented. Value iteration, policy iteration, and linear programming approaches are studied. The concept of relative value iteration is introduced. A brief discussion of structured policies is included. The chapter concludes with a case study of queuing service rate control, comparing solution algorithms.