The built environment is increasingly recognized as being associated with health outcomes among older adults. This study examines how multiple housing features collectively are associated with self-rated health in the United States. Using data from Round 12 of the National Health and Aging Trends Study (n = 930), multiple correspondence analysis combined housing type, residential setting and building floors into a composite measure of residential complexity. Ordinal logistic regression models, adjusted for age, sex, education, race and functional capacity, assessed associations with self-rated health, including age and sex interactions. Higher residential complexity was linked to poorer self-rated health (β = –0.249, p < 0.001), but this relationship became non-significant after adjusting for socio-economic and health factors (β = –0.065, p = 0.148). A significant interaction with age revealed a clear gradient (p < 0.001). The adverse association was negligible in those aged 65–69 (β = −0.02, p = 0.812) but substantially stronger in the oldest-old: for example, in 85+ it was (β = −0.31, p < 0.001). There was no significant interaction with sex (p = 0.552). Grounded in person–environment fit theory, findings underscore the need for housing policies that integrate structural accessibility with socio-economic supports.