This chapter studies a class of widely applicable models referred to as partially observable Markov decision processes (POMDPs). In a POMDP, the decision maker cannot observe the system state but instead receives a signal that provides noisy information about the hidden state. The key idea is to leverage Bayes’ theorem to update a prior probability distribution over the hidden states using the action taken and signal observed to obtain a posterior distribution. These probability distributions are referred to as belief states, which become states in an equivalent fully observable MDP, thus transforming a finite-state POMDP into an uncountable-state MDP. The finite horizon POMDP value function is shown to be piecewise linear convex, which leads to an exact but complex solution algorithm. Approximate solution methods are also provided. A brief discussion of infinite horizon POMDPs is included. Formulations of the following applications are presented: preventive maintenance and inspection, Bayesian decision problems, multi-armed bandits, breast cancer screening, and robotic control.