In this paper, we study the irregular multifractal spectrum of the Gauss map, by investigating the upper and lower Birkhoff spectra. For potentials satisfying a weakly tempered distortion property, we establish that the Hausdorff dimensions of the classical, upper, and lower Birkhoff level sets are identical. Furthermore, we derive an exact formula for the dimension of the intersection of the upper and lower Birkhoff level sets, proving that this dimension equals the minimum of the two individual dimensions. These findings broaden the multifractal theory previously developed for conformal repellers and finite branch expanding systems to the infinite branch context of the Gauss map. As applications, we furnish complete multifractal descriptions of the upper and lower Lyapunov spectra of the Gauss map, as well as the upper and lower arithmetic-mean spectra of continued fractions.