This chapter discusses machine learning methods (like supervised and generative models) along with a variety of other artificial intelligence approaches (like graph clustering and logic reasoning) that have provided a path towards modeling two important types of contexts for situated human-agent interactions. One type of context is the interpersonal and social context of the interactions, which includes factors about the individual interactants, factors that describe interpersonal relationships, and whole group constructs. The other type of context is the normative context, which includes human norms that govern behavior and the perception of behavior during interactions. These two types of contexts interweave the where, what, when, who and why of situated human-agent interactions. The chapter discusses progress in modeling all these contextual factors, as well as open challenges and paths forward to advance the state of the art in context modeling. Importantly, solving many open challenges requires interdisciplinary perspectives that combine user-centered design, system development and evaluation with technical advances across disciplines, like machine learning, affective computing and robot ethics.