This chapter studies infinite horizon MDPs under the expected total reward criterion. Without discounting, care must be taken to ensure that the relevant quantities, such as the policy value function, are well defined and finite. Four model classes are distinguished, namely,: transient, stochastic shortest path, positive, and negative models. The Bellman equation and its role in characterizing optimal values are presented. Value iteration, policy iteration, modified policy iteration, and linear programming approaches to solving the model are described. A brief discussion on structured policies is included. The chapter concludes with two case studies: Gridworld navigation and optimal stopping.