This chapter begins with a description of the basic MDP components, namely, the planning horizon, decision epochs, states, actions, transition probabilities, and rewards. Next, it describes derived objects, including decision rules, policies, derived stochastic processes, and reward processes. Formal definitions of the expected total reward, expected discounted reward, and long-run average reward criteria are presented for finite and infinite horizon models. A one-period model is formulated, which is used to introduce the key concepts of value functions, state-action value functions, optimal values, and optimal policies. Deterministic and randomized policies are distinguished, and a one-period problem illustrates the fundamental result that deterministic policies typically suffice. Finally, a two-state MDP model is presented and analyzed in depth. This important example appears throughout the book to illustrate new concepts as they are presented.