Sample preparation and measurement

Samples from RC1 to RC9 were available based on the participation of Stephen Hillier in all previous Reynolds Cup contests. For RC1 and RC2, samples were spiked with a known weight percentage of an internal standard (~20% corundum), whereas for RC3 through to RC9, the sample-preparation protocol was changed and all samples were prepared without addition of an internal standard.

Each of the 27 RC samples was prepared for XRPD as received by McCrone milling 3 g of sample for 12 min in ethanol and spray drying the resulting slurry to obtain a random powder specimen as described by Hillier (Reference Hillier1999) and demonstrated by Kleeberg et al. (Reference Kleeberg, Monecke and Hillier2008). This preparation was done at the time of each of the respective Reynolds Cups, so over a 16 year period (2002–2018). To enable further the detection of trace-mineral phases, very high quality diffraction data were recorded by scanning over the range 4−70°2θ on a Bruker D8 using Ni-filtered Cu Kα radiation, fixed divergence slits, and a Lynxeye XE detector, with counts recorded for 16 s per 0.0195°2θ step yielding scan times of 16 h. These scans were already available for RC7 to RC9, but for RC1 to RC6 the spray-dried specimens were retrieved from their storage (8−18 y) in capped glass vials and re-run on the D8 diffractometer, which was not available in the authors’ laboratory prior to RC7 (2014).

Reference library preparation

A reference library of standard XRPD patterns of pure minerals has been compiled in Stephen Hillier’s laboratory over a period of time from specimens of pure minerals obtained from various mineral collections or purchased, such as from the Source Clays Repository of The Clay Minerals Society (Costanzo and Guggenheim Reference Costanzo and Guggenheim2001). The purity of the minerals was assessed mainly by XRPD data, and for many samples – especially the clay minerals – the best purity was obtained by picking or by size-fractionation procedures. Inevitably, small impurities remained in many samples, e.g. quartz is a ubiquitous contaminant of most clays, even very fine-size clay fractions. Where required, remaining impurities were, therefore, removed electronically by subtraction of the whole pattern of the pure phase impurity. All such treated patterns were scaled to a maximum intensity of 10,000 counts prior to determination of a full-pattern RIR from a mixture of the pure mineral (plus any impurities) with corundum as an added internal standard, for which the weight fractions were known. Further details of this procedure will be presented elsewhere. All standards were run under the same diffractometer conditions as the unknowns, except that the recording time per 0.0195°2θ step for library standards was just 2 s. All backgrounds of the standards and samples were retained throughout.

The full reference library available for this investigation included 201 diffractograms of pure standards designed to cover most components associated with geologic, soil, and sediment samples. Of these, 76 were clay mineral/phyllosilicate reference patterns, 116 were non-clay, and nine were amorphous, nanocrystalline or paracrystalline (allophane, ferrihydrite, glass, obsidian, opal-CT, opal-A, aluminosilicate gel, organic matter, and graphite). Many of the library entries are for the same mineral, e.g. the library as used contains patterns for seven different specimens of kaolinite. Since RC5 was organized by Hillier, this library contains patterns for exactly the same mineral specimens that were used for the preparation of the RC5 samples. Given this, the current testing of the automated algorithm for RC5 samples represents a ‘best case scenario.’ In all other cases, the minerals in the library are not necessarily from the same source as the minerals in the unknown Reynolds Cup samples, though some may be when RC organizers have used widely available materials such as those from the Source Clays Repository of The Clay Minerals Society.

Mineral standards used to create this reference library were also associated with the top-three place finishes in all RC contests except for RC5 (organized by Hillier). The key component tested here was, therefore, the ability of the present algorithm to pre-select the appropriate phases from a large and comprehensive reference library for subsequent automatic quantification.

Step 1: Sample alignment

Previous in-house experience with full-pattern summation has highlighted the importance of aligning the sample diffractogram along the 2θ axis to that of a calibrated pattern in order to correct for common experimental aberrations associated with the collection of XRPD data (Butler et al. Reference Butler, Sila, Shepherd, Nyambura, Gilmore, Kourkoumelis and Hillier2019). The discrete nature of XRPD peaks means that seemingly small misalignments can have particularly detrimental effects on data analysis along with the accuracy of phase identification and quantification.

The automated alignment of a sample via the afps algorithm used here requires selection of a phase present within it to use as an internal standard (std argument; Table 1). For RC1 and RC2, samples were prepared with corundum as the internal standard which was, therefore, used for this alignment. For RC3–RC9, samples were prepared for XRPD analysis without an internal standard, and hence the ‘internal standard’ for alignment was chosen simply as a component of the mineral mixture with sharp, well characterized diffraction features for use as internal d-spacing standard. The designated ‘internal standard’ for each sample was then used by afps to align the diffractogram along the 2θ axis and hence correct for common experimental aberrations such as sample displacement.

Alignment of the sample to the chosen standard is achieved by maximizing the Pearson correlation via one-dimensional optimization (Brent Reference Brent1971) within a fixed limit of positive and negative 2θ shifts defined by the align argument (Table 1).

With respect to many geologic and environmental samples, the omnipresent mineral quartz can act as a suitable internal standard for the large majority of cases. In the absence of quartz, any well characterized non-clay mineral with few overlapping peaks may be suitable (e.g. dolomite, calcite, anhydrite) providing it is present within the sample(s) being considered and that any solid solutions are represented appropriately by a standard in the reference library. For the present dataset, visual inspection of each of the samples identified that quartz reference patterns would be suitable internal standards in 18 cases. In the remaining three cases where a suitable quartz signal was not observable, internal standards of fluorite (RC5-2), anhydrite (RC7-1), and dolomite (RC9-3) were selected. For all samples presented here, the align argument was set to 0.1°2θ.

Step 2: Phase selection with non-negative least squares

For high-throughput datasets that may display substantial mineralogical variation, a comprehensive reference library is necessary that can cover most, if not all, of the non-clay, clay, and amorphous phases that may exist within the sample set. In such cases it is reasonable to expect that reference libraries containing >100 reference patterns would be required, in which case it becomes impractical to optimize so many variables at once – both in terms of accuracy and time. For this reason, phase selection on a sample-by-sample basis is a key component of automated full-pattern summation.

The afps algorithm applied here uses non-negative least squares (NNLS) to identify quickly the phases that can be removed from the reference library. Functionality for NNLS in R is provided by the NNLS package (Mullen and van Stokkum Reference Mullen and van Stokkum2012), which is based on the FORTRAN code of Lawson and Hanson (Reference Lawson and Hanson1995). Application of NNLS facilitates rapid identification of phases in the reference library that probably exhibit no contribution to the observed pattern via derivation of coefficients equal to zero, which are thus omitted from the process (Fig. 1).

Step 3: Minimization of an objective function

As outlined by Chipera and Bish (Reference Chipera and Bish2002), a range of functions can be minimized for full-pattern summation. Choosing an appropriate function for minimization is key to accurate quantitative analysis via this approach. For mixtures containing clay minerals, non-clay minerals, and amorphous phases, past experience at the James Hutton Institute has shown that the minimization of R _wp (Bish and Post Reference Bish and Post1989), defined as:

(1) $R_{\mathrm{wp}}={\left(\frac{\sum \left(I_m^{-1}\times {\left(I_m-I_c\right)}^2\right)}{\sum \left(I_m^{-1}\times I_m^2\right)}\right)}^{\frac{1}{2}}$

often results in the most accurate quantitative results (I _m and I _c are vectors of measured and calculated intensities, respectively). Indeed, the R _wp statistic is noted typically as one of several performance parameters in Rietveld refinements (Toby Reference Toby2006), and by weighting the count intensities via the $I_m^{-1}$ terms, results in calculated patterns that prioritize the fitting of regions near to the tails of peaks (Bish and Post Reference Bish and Post1989). This attribute has beneficial effects when handling the diffuse diffraction signal of poorly ordered and amorphous phases that are encountered commonly in RC samples. Hence, the R _wp was used as the objective function for all RC samples presented here, and was minimized using the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm (Broyden Reference Broyden1970; Fletcher Reference Fletcher1970; Goldfarb Reference Goldfarb1970; Shanno Reference Shanno1970).

The initial optimization of R _wp in the afps algorithm is applied across all reference patterns that remain after NNLS (Fig. 1). The BFGS optimization routine does not constrain coefficients to positive values; thus any reference patterns that have negative coefficients after optimization are removed from the process, and R _wp re-optimized until no negative values remain (Fig. 1).

Step 4: Shifting of reference library patterns

Further to linear alignment of the sample pattern to a reference pattern (Step 1), small additional 2θ shifts applied to each reference pattern in the fitting process of the XRPDBULK program (Hillier Reference Hillier2015, Reference Hillier2018) have been found to yield more accurate results. Hence, a second alignment step implemented in the afps algorithm seeks to apply small 2θ corrections to the reference patterns relative to the sample pattern to account for additional small variations such as uncorrected sample displacement errors that may be present in the reference library. More specifically, after an initial optimization of the scaling coefficients, the objective function (Eqn 1) is again minimized by optimizing a shifting coefficient for each remaining reference pattern, which specifies its positive or negative adjustment along the 2θ axis. During the optimization of the shifting coefficients, the scaling coefficients are fixed and reference patterns are maintained on the same 2θ axis interval using cubic spline interpolation. If the absolute value of any optimized shifting coefficient exceeds the value specified in the shift argument, its shifting coefficient is reset to zero. Reference patterns are then shifted by the derived shifting coefficients, with cubic spline interpolation again used to ensure that they all remain on the original 2θ axis interval, and the scaling coefficients re-optimized (Step 3). In all cases presented here, the shift argument was set to 0.5°2θ.

Step 5: Quantification and limit of detection estimation

At this stage, appropriate phases were assumed to have been selected from the reference library and the fitted pattern was assumed to be reasonable, from which a reasonably accurate estimation of phase concentrations can be obtained using the RIRs. Because all RC samples presented here were prepared without an internal standard, all detectable phases within the mixture were assumed to be identifiable and that their concentrations summed to 100 wt.%. As such, phase concentrations (X) were computed by:

(2) $X=\frac{s/\mathrm{RIR}}{\sum \left(s/\mathrm{RIR}\right)}\times 100$

where s and RIR denote vectors of the scaling coefficients (i.e. the parameters derived from NNLS and optimization of R _wp) and RIRs covering all remaining phases, respectively.

At this point in the process, some phases, estimated to have very small concentrations, may, inevitably, have been selected in error. Whilst all phases below a defined limit (e.g. 0.1%) could be simply excluded, such an approach would not account for the way in which different phases diffract X-rays with different power (reflected in the RIRs) and, hence, have different limits of detection (LOD) (Hillier Reference Hillier2003). For example, a strong diffractor such as quartz, with a RIR (relative to corundum) of ~5.7, would have a smaller LOD than a weak diffractor such as muscovite (RIR ≈ 0.5). Thus the afps algorithm uses the RIRs to derive sensible estimations of LODs for all remaining phases via:

(3) $\mathrm{LOD}={\mathrm{LOD}}_{\mathrm{std}}\times \frac{{\mathrm{RIR}}_{\mathrm{std}}}{\mathrm{RIR}}$

where LOD_std is the LOD of the internal standard (defined by the lod and std arguments of the afps algorithm; Table 1), RIR_std is the RIR of the internal standard, and RIR is a vector of RIRs for all remaining phases. Upon calculating the LODs, all clay and non-clay phases below their respective LOD are removed from the process. For all RC samples presented here, the lod argument was based on the assumption that the LOD of quartz (RIR = 5.68) was 0.15%, from which the LODs of other internal standards used (fluorite, RIR = 4.9; anhydrite, RIR = 3.0; and dolomite, RIR = 2.3) were estimated via Eq 3. Note that the actual value for the LOD of any phase used as reference can be calculated if a sample is spiked with a known weight fraction of another phase as outlined by Hillier (Reference Hillier2003), but the simplified approach presented here is based on an arbitrary but realistic LOD for quartz.

Amorphous phases need to be treated slightly differently in the afps algorithm to account for the way in which their diffusely scattered signal can be difficult to detect in XRPD data, deeming the approach of Eq 3 as inappropriate. For this reason the amorphous phases defined by the amorphous argument are retained unless their estimated concentrations are lower than the value specified in the amorphous_lod argument (Table 1). For all RC samples presented here, nine phases in the reference library were defined in the amorphous argument (allophane, ferrihydrite, glass, obsidian, opal-CT, opal-A, aluminosilicate gel, organic matter, and graphite), and the amorphous_lod argument set to 2%.

After omission of phases based on LODs, a final re-optimization of R _wp was applied until no negative parameters remained (Fig. 1). At this point the fitting process was considered complete, and final concentrations computed via Eq 2 in units of wt.%.