I argue that Wittgenstein was no lifelong musical formalist. I further contend that the attribution of musical formalism obscures, while the break with it I propose explains, the role that music played in the development of his philosophy of language. What is more, I sketch a perspective on the later Wittgenstein’s remarks on the music and musical understanding that supports my claims. Throughout my discussion, rather than assimilating Hanslick’s and Wittgenstein’s views on music, I point to similarities and differences between them, suggesting that taking snapshots and putting them side by side sheds more light on how they are related to each other.