For a positive integer $n$ let $\mathcal{X}_n$ be either the algebra $M_n$ of $n \times n$ complex matrices, the set $N_n$ of all $n \times n$ normal matrices, or any of the matrix Lie groups $\mathrm{GL}(n)$, $\mathrm{SL}(n)$ and $\mathrm{U}(n)$. We first give a short and elementary argument that for two positive integers $m$ and $n$ there exists a continuous spectrum-shrinking map $\phi : \mathcal{X}_n \to M_m$ (i.e., $\mathrm{sp}(\phi(X))\subseteq \mathrm{sp}(X)$ for all $X \in \mathcal{X}_n$) if and only if $n$ divides $m$. Moreover, in that case we have the equality of characteristic polynomials $k_{\phi(X)}(\cdot) = k_{X}(\cdot)^\frac{m}{n}$ for all $X \in \mathcal{X}_n$, which in particular shows that $\phi$ preserves spectra. Using this, we show that whenever $n \geq 3$, any continuous commutativity-preserving and spectrum-shrinking map $\phi : \mathcal{X}_n \to M_n$ is of the form $\phi(\cdot)=T(\cdot)T^{-1}$ or $\phi(\cdot)=T(\cdot)^tT^{-1}$, for some $T\in \mathrm{GL}(n)$. The analogous results fail for the special unitary group $\mathrm{SU}(n)$ but hold for the sets of semisimple matrices in either $\mathrm{GL}(n)$ or $\mathrm{SL}(n)$. As a consequence, we also recover (a strengthened version of) Šemrl’s influential characterization of Jordan automorphisms of $M_n$ via preserving properties.