We develop an asymptotic theory for the first two sample moments of a stationary multivariate long memory process under fairly general conditions. In this theory the convergence rates and the limits (the fractional Brownian motion, the Rosenblatt process, etc.) all depend intrinsically on the degree of long memory in the process. The theory of the sample moments is then applied to the multiple linear regression model. An interesting finding is that, even though all the regressors and the disturbance are stationary and ergodic, the joint long memory in one single regressor and in the disturbance can invalidate the usual asymptotic theory for the ordinary least squares (OLS) estimation. Specifically, the convergence rates of the OLS estimators become slower, the limits are not normal, and the standard t- and F-tests all collapse.