I. INTRODUCTION
Chemical Analysis by X-ray Diffraction is the title of a famous 1938 publication (Hanawalt et al., Reference Hanawalt, Rinn and Frevel1938; Reference Hanawalt, Rinn and Frevel1986) that describes a method of identification of crystalline materials using a database of powder diffraction patterns. The publication included 1000 patterns and was the genesis of today's Powder Diffraction File (PDF), which from the very beginning, was not just a database but a method for chemical analysis. The method, which was manual, included search indexes, sorted and edited data, and a written procedure for the analysis of multiphase unknowns. It was the standardized data, indexes, and procedures that were the innovation required for material identification by powder diffraction methods.
As we celebrate the 75th anniversary of the Powder Diffraction File, the PDF is still an important method for chemical and material analyses. Through continuous input of the global scientific membership of the International Centre for Diffraction Data (ICDD®), the producers of the PDF, the PDF has continued to evolve and grow to address the expanding needs of the scientific community. The PDF, as embodied in the PDF-4 family of databases, is no longer limited to crystalline materials or diffraction analysis by X-rays. A continuous focus on improvements in editing, standardizing, and correcting published and contracted diffraction data ensures that the correct results are obtained with a wide range of materials. Expert input and editorial review on an extensive subfile system helps users target the appropriate chemistry and obtain more accurate results.
Powder diffraction analysis is often referred to as a fingerprinting method. Like fingerprints, unknowns are identified by comparison of their diffraction data to a series of standardized reference data. In the case of fingerprints, reference fingerprints are usually collected from people when they are born, are a suspect in a crime, enter government service, or are required for international travel security. For powder diffraction analyses, reference data were initially collected by individual scientists working for various government, industry, and academic groups whom voluntarily shared their data to produce the PDF (Jenkins and Smith, Reference Jenkins and Smith1996; Messick, Reference Messick2012).
The PDF was first produced in 1941. From the very beginning, the PDF was a materials analysis system that used indexes to compare unknown materials to reference data for the purposes of phase identification. The data files themselves consisted of a series of d-spacings and intensities that were sorted and indexed for rapid identification using manual search methods. D-spacings and intensities are defined by the type and arrangement of atoms in a crystalline material and are characteristic of that material. The creation of several types of indexes enabled the data to be searched rapidly (Hanawalt, Reference Hanawalt1986). This was important in years past when data were stored on cards. The physical size of the card and book collections became very large when the number of entries exceeded 10 000 materials (Figure 1). Today, the PDF contains references for hundreds of thousands of materials and millions of data points. Indexes coupled with modern algorithms are still used to rapidly search through this immensity of information. There are over 100 million d-spacings tabulated in the modern PDF. Since X-rays were discovered in 1895 and the fundamental physics of powder diffraction over the succeeding two decades, it was the method of analysis that distinguished the PDF from several earlier database efforts. The data in the database itself were organized and edited in a manner to facilitate the search and identification process.
(Color online) Growth of the Powder Diffraction File, PDF-4+ (red) and PDF-4/Organics (green)
Now 75 years since its origin, the PDF continues to be designed to facilitate materials analysis. The materials are organized in a manner to facilitate data mining for the purpose of identification and analysis. The original methods have been expanded and enhanced. Indexes are still used, but they are coupled to sophisticated algorithms to speed the search process. The identification processes originally designed to identify crystalline materials can now be used to analyze a variety of solid-state materials, including amorphous and nanomaterials. The original PDF was designed for X-ray diffraction and now it encompasses electron, neutron and synchrotron diffraction. Physical property data have been added or calculated to enhance the identification process through multi-variant analyses. Fundamentally the PDF has changed and adapted to the ever changing needs of the materials analysis community. In this paper, we will discuss the form, format, focus, and function of the 2016 PDF.
II. RESULTS AND DISCUSSION
A. Content and function
In physical size, the PDF database is larger than any comparable crystallographic database. This is simply because the PDF contains more information. The database contains information needed to identify and characterize materials, which includes chemical and physical properties, diffraction, and crystallographic data. A material does not have to be crystalline to be included in the database. The database includes solid-state diffraction patterns of any material that exhibits chemical stability with a reproducible diffraction pattern. This includes semi-crystalline layered clays, mixed crystallinity polymers, waxes, gels, amorphous solids, and nanomaterials. These inclusions allow database users to study crystallite size, crystallinity, and both the coherent and incoherent scatter produced by a solid-state material. The critical criteria for the inclusion of non-crystalline materials are the evidence of chemical purity and chemical stability provided by the original authors.
A major difference between the PDF produced in 2016 and those produced in early decades is the expansion in scope and function of the database. This was a fundamental change driven by the global scientific membership of the ICDD, the non-profit scientific organization that produces the PDF. Member scientists either individually, or as a member of a task group or committee of the ICDD, regularly suggest changes in both scope and function to address their own needs in materials analysis. Suggestions come not only from annual meetings of the ICDD, but also from workshops and events conducted around the globe. The members elect a board of directors composed of scientists working in the materials analysis community and various boards have promoted substantial changes. Of particular note in the past 30 years were the board decisions to be an early adopter of CD-ROM technologies, the decision to form strategic database alliances to combine powder and single-crystal data sources, and the decision to direct the organization toward total pattern analysis. This latter decision involved the addition of non-crystalline materials, the development of relational databases and the ability to simulate patterns using a variety of instrumental and specimen profiles, energy sources, and detector configurations. These changes were driven by Boards of Directors under the direction of Deane K. Smith (1986–1990), Robert L. Snyder (1996–2000), and Camden R. Hubbard (2000–2004). The vision was also supported by many notable ICDD board members who were also pioneers in computational chemistry and crystallography. In addition to the chairmen mentioned above, this group included among others, James Kaduk, Daniel Louër, Gerry Johnson, Paolo Scardi, Brian O'Connor, Ray Young, Jan Visser, Ron Jenkins, Herbert Göbel, and Raymond Goehner. These changes were articulated at the ICDD workshop entitled “ICDD Reinvented” presented at the EPDIC-7 conference in Barcelona, Spain (Hubbard, Reference Hubbard2000) and subsequently implemented (Faber and Fawcett, Reference Faber and Fawcett2002). This process is a continuous evolution, directed by ICDD members, with new features and capabilities added to each annual release of the database.
As shown in Table I, the database function has been broadened to include several types of analysis that are common with modern diffractometers. The broadening of the design function meant that additional types of data needed to be added to the database. For non-crystalline materials, this meant capturing full digital patterns of the coherent and incoherent scatter. These digital patterns can be used by pattern fitting methods to determine non-crystalline phases and measure sample crystallinity. For multi-dimensional modulated structures, the ICDD had to display modulation vectors (Petricek et al., Reference Petricek, Dusek and Palatinus2014), superspace groups, with up to six-dimensional indexing and atomic displacement tables with new functions describing the modulation. For quantitative analysis, atomic coordinates were added and now Release 2016 PDF-4+ has both the world's largest collection and chemically most diverse collection of atomic parameters for inorganic materials. This leads to an unprecedented ability to analyze very complex mixtures and quantitate the results by Rietveld refinement and Whole Pattern Fitting methods (Smith et al., Reference Smith, Johnson, Scheible, Wims, Johnson and Ullman1987). For these total pattern analysis methods, the analysis range had to be extended for structure refinement, lower symmetry organic molecules, and modulated structures. For low symmetry inorganics, hundreds of low-intensity peaks were added, greatly enlarging the size of all PDF databases. Figure 2 shows the increase in database size for the last 10 years of PDF-4+ and PDF-4/Organics.
(Color online) The growth of the Powder Diffraction File 2004–2016, both in database size (GBytes, orange and dark blue lines), content and capability (searches in green, display fields in purple).
Comparison of general features in the 1941, 1983, and 2016 PDF databases.
The need for different types of information and comprehensive materials coverage drove the ICDD to develop a series of strategic partnerships with other crystallographic database organizations (Table I). This first occurred with a collaboration with the National Bureau of Standards in 1953 that lasted for more than 30 years (Wong-Ng et al., Reference Wong-Ng, McMurdie, Hubbard and Mighell2002). There were sudden increases in data content in 1998 with the addition of ICSD data (Inorganic Crystal Structure Data, FIZ Karlsruhe), in 2001 with the addition of NIST data (National Institute of Standards and Technology), in 2002 with the addition of CSD data (Cambridge Structural Data, Cambridge Crystallographic Data Centre), and in 2005 with the addition of LPF data (Linus Pauling File) from a collaboration with MPDS (Material Phases Data System). The additions of data from NIST and CSD are primarily historic in nature. ICDD, FIZ, and MPDS are active partners today with each organization having their own bibliographic and editorial staffs that add new annual content to the PDF.
In Rietveld structural refinement and quantitation, the identification of the appropriate structure set to use as a starting set is critical (O'Connor and Li, Reference O'Connor and Li1998, Reference O'Connor and Li2000; Scarlett and Madsen, Reference Scarlett and Madsen2001; Scarlett et al., Reference Scarlett, Madsen, Cranswick, Lwin, Grouleau, Stephenson, Aylmore and Agron-Olshina2002; Fawcett et al., Reference Fawcett, Needham, Faber and Crowder2010). To help scientists identify isostructural and isotypical compounds, the database contains several types of structural classification, scientific nomenclature, and functional group indexing.
B. Format and capability
The addition of data does not equate to an analysis capability. The PDF uses a relational database format where the data are stored in large cross-linked tables (Kabekkodu et al., Reference Kabekkodu, Faber and Fawcett2002). This has been found to have two major advantages over earlier database formats: (1) the acquisition of new data only requires new tables and/or new fields, and (2) the format facilitates data mining. Once the data are tabulated, there is an additional requirement for writing query software, so that fields can be easily and rapidly searched. In the PDF database, Java is used for facile interfaces. This is why you will find that PDF databases offer more searches and more display fields than any other competitive database. The PDF has more data and it is organized efficiently so there are more data mining possibilities and increased analysis capability. The database is loaded with Boolean search operators, so that queries can be added, subtracted, and combined. Since Release 2012, all ICDD database products have included embedded data mining software. This leads to improved phase identification and quantitation, an ability to identify trace phases and perform improved Rietveld analyses (Kaduk, Reference Kaduk2009; Fawcett et al., Reference Fawcett, Needham, Faber and Crowder2010, Reference Fawcett, Crowder, Kabekkodu and Kaduk2011). It has also led to an ability to measure thermal expansion coefficients (ICDD, 2015) for over 10 000 material systems, the ability to structurally classify complex systems, the ability to identify pharmaceutical polymorphs (Fawcett et al., Reference Fawcett, Faber, Needham, Kabekkodu, Hubbard and Kaduk2006), and the ability to identify instrument performance and calibration.
C. Focus on materials
A strategic element in the design of the PDF has been the development of subfiles. Subfiles at their best allow the user to focus on the specific chemistry or application of their specimen, eliminating entries that are inappropriate, which fundamentally produces more accurate results and less false positive results. The subfile can only be effective if it is edited by experts in the field and has comprehensive content coverage. The ICDD utilizes its international membership to be field experts for the database. These experts are supported by a professional editorial staff to classify, standardize and edit the materials included in a subfile. Some of the ICDD's editorial subfile task teams have been established for decades with several generations of scientists that have served as volunteer editors. The very first products in 1941 contained inorganic and organic subfiles. In subsequent decades, minerals, metals and alloys, and forensics subfiles were added to the database (Table I). The PDF subfile system currently contains 52 subfiles and subclasses. New subfiles are added with the discovery and growth of new materials on a worldwide basis. In the last decade, the ICDD has added subfiles for modulated structures, battery, thermoelectric, and hydrogen storage materials. Each subfile is reviewed and changed on an annual basis as both new materials and higher quality determinations are added to the database.
A key criterion for an effective subfile system is to have comprehensive coverage. The ICDD has multiple programs that target commercial materials for inclusion in the database. Most published crystal structures for both inorganic and organic materials do not have commercial applications and are unlikely be found in an unknown material identification by materials analysis laboratories. Therefore it is important to have comprehensive coverage of targeted materials used in commercial applications (Table II), which includes all minerals, metals, and alloys. ICDD has had grant programs since 1953, which are designed to help scientists collect data on new materials with commercial application. Decades of grants have resulted in materials coverage that is not matched by other databases. In recent years, the ICDD has also entered into specific material grants targeted at pharmaceuticals, modulated structures, and polymer additions. These programs have resulted in the world's largest database of properly defined modulated structures (for example, see Figure 3) and the first structural analysis of some of the highest sales volume pharmaceuticals (Kaduk et al., Reference Kaduk, Crowder, Zhong, Fawcett and Suchomel2014). This pharmaceutical reference is one of 27 co-authored by J. A. Kaduk et al., on top selling pharmaceuticals published in Powder Diffraction between 2014 and 2016.
(Color online) Six dimensional indexing and simulated powder pattern of bismuth chromium oxide (PDF 05-001-0336), a material containing both atomic displacement and occupancy modulations.
Select subfile populations in PDF products. The subfiles outlined in a box are the largest known crystallographic compilations of these materials.
D. Focus on quality
Not all reference data are equal, as both powder and single-crystal determinations can be plagued by a number of well-documented problems (Klug and Alexander, Reference Klug and Alexander1974; Calvert et al., Reference Calvert, Flippen-Anderson, Hubbard, Johnson, Lenhert, Nichols, Parrish, Smith, Smith, Synder and Young1980; Jenkins and Snyder, Reference Jenkins and Snyder1996) that degrade the overall reliability and quality of a reference powder pattern (Fawcett et al., Reference Fawcett, Kabbekodu, Faber, Needham and McClune2004; Kabekkodu and Fawcett, Reference Kabekkodu and Fawcett2015). In experimental determinations, there are a number of common instrumental and specimen errors and there can also be errors associated with the data processing such as peak cluster deconvolution, finding atoms, indexing the unit cell with the proper space group, and dealing with structural disorders. There is also a lack of precision and accuracy associated with certain types of measurements and instrumental configurations, both now and in the past. Alternately monochromatic sources and energy selective detectors may produce superior quality data, but only if coupled with good specimen preparation and data processing. This produces a wide range in the quality of diffraction data and crystal structure solutions in the world's published literature. While several commercial database and scientific publishers perform quality checks, the ICDD is the only database organization to have its quality processes ISO certified with published quality marks and editorial comments to let users of the PDF evaluate the quality of each reference. ICDD's quality review for each entry is a combination of automated analyses with editorial review by field experts, both volunteer, and paid staff. This quality review and grant programs (http://www.icdd.com/grants/index.htm) for data acquisition are the main contributors to the cost of producing the database, but they also contribute value in obtaining accurate results. This is because automated identification processes use residual analyses for phase identification in multiphase samples. If a reference diffraction pattern is of poor quality, it typically will not pass the statistical analysis checks used by most search/match algorithms. If the reference pattern does pass the statistical analysis, but has errors, then the calculated residual pattern will be incorrect. This results in the identification being stopped, or more dangerously, a false positive analysis based on the incorrect residual. This phenomenon is easily seen in side-by-side comparisons of edited and unedited databases using multiphase samples.
The ICDD has continuously improved its quality control review processes. Initially, data were manually reviewed by ICDD staff editors and member task groups. However, an automated review program, NBS*Aids80 (Hubbard et al., Reference Hubbard, Stalick and Mighell1980), was developed in the 1980s and applied to all ICDD data (Wong-Ng et al., Reference Wong-Ng, Hubbard, Stalick and Evans1988). This program has been continuously modified and updated over the last 35 years of continuous use by ICDD editors. Field experts used the automated review programs to further edit and classify data. At the turn of the millennia the ICDD was adding a substantial amount of single-crystal data and a quality review system was specifically developed for single-crystal analysis. In the last 5 years, new review systems have been developed for both non-crystalline materials and multi-dimensional (4–6 dimensions) modulated and composite structures (ICDD, 2017). This creates a grid of quality review marks that vary with types of data input as shown below in Table III.
Quality marks and symbols used by the PDF to evaluate and classify different types of data.
Hypothetical and prototype structures are typically not published in the PDF databases except for the existence of strong supporting data that the phase exists. For example, some hypothetical structures of fibrous materials are included if there is a fiber pattern, solid-state NMR, or other analytical data that are input to constrain the model and verify results. In many prototypes, there is experimental evidence that the prototype belongs to an isotypical series. ICDD editors frequently process and review prototype and hypothetical structures to gather quality review data before deciding whether to publish. For example, the authors may publish a unit cell that can be used to calculate density, which can be checked relative to the stated chemistry (Kabekkodu and Fawcett, Reference Kabekkodu and Fawcett2015).
A major objective of ICDD's quality program is to correct any errors in the reference data prior to publication. Unit cells are recalculated to verify the author's results, powder patterns are recalculated as necessary, and typographical errors are automatically analyzed, identified, and corrected. With single-crystal data, the data may be reviewed several times, first by a refereed journal, then by an ICDD collaborating database organization (FIZ, CCDC, NIST, MPDS), and finally by the ICDD. Sometimes suspected errors are not corrected because of insufficient information as provided by the author or author's experimental procedure. In these cases, use of the author's original data is used by default. Frequent errors of this type might be a disordered single-crystal structure or specimen displacement in experimental powder diffraction data. If errors are severe, then the entry is not published. Approximately 1.3 million reference publications have been reviewed by ICDD with final publication of >950 000 entries by the end of the 2016.
E. Methods development
Methods of analysis have greatly expanded since the original development of the PDF. Today, ~15 different commercial software analysis packages interface to the PDF, including the software of the world's global diffractometer manufacturers. This provides users with a tremendously wide range of analysis capabilities. The ICDD works with many software developers and distributors by sharing file formats and accessing tables for seamless transitions between the database and software. We also have common import and export formats available in the database for common freeware programs such as GSAS and FullProf.
Staying true to the original 1938 concept of chemical analysis by diffraction, the ICDD has incorporated many fields into each data entry that enables more accurate analyses by defined chemistry. These fields are grouped into large categories such as subfiles, periodic table search, elemental composition, nomenclature, structure, and physical properties. The use of standard nomenclature (IUPAC, IZA, or IMA for example) in current scientific literature is abysmal and even worse in historical publications. ICDD editors estimate that less than 50% of current journals enforce standard chemical nomenclature. ICDD editors standardize nomenclature based on IUPAC, IZA, and IMA guidelines, often providing nomenclature not found in the original publication, with minor modifications to provide consistency among international groups and conventions. The ICDD also adds trade names and common names whenever possible. Once the entry is standardized, nomenclature becomes an accurate source for functional group and elemental analysis searches. The ICDD also added weight % and atom % searches in addition to the commonplace periodic table search so that complimentary elementary analysis techniques such as X-ray fluorescence (weight %) and Energy Dispersive Spectroscopy (atomic %) can be used to search and identify materials. Physical properties such as color, density, and melting point can also be used to identify materials. The ability to search by chemical formula and alphabetic name was a feature in the first PDF in 1941, the Matthews Index (1965) and EISI index (1989) included searches on elemental composition. Current PDF-4 databases use 63 searches and 115 display fields that can be used in any combination producing almost unlimited data mining.
However, as applied to the PDF database for the identification of unknowns, the selection of 3–5 characteristic material quantitates (d-spacing, composition, properties, etc.) will usually define a unique result as shown by descriptive statistics (Fawcett et al., Reference Fawcett, Crowder, Kabekkodu and Kaduk2011). The structure and format of the database work in harmony with the identification process to produce the most accurate results.
As an organization, the ICDD continues to look at new methods of analysis. This has led to the incorporation of crystallite size simulations (Scardi et al., Reference Scardi, Leoni, Lamas and Cabanillas2005), the development of search methods for electron diffraction (Reid et al., Reference Reid, Crane, Blanton, Crowder, Kabekkodu and Fawcett2011) as shown in Figure 4, and neutron diffraction (Faber, Reference Faber, Crowder, Blanton, Kabekkodu, Gourdon, Blanton and Fawcett2015) as shown in Figure 5, and the development of similarity indexes for the analysis of amorphous and non-crystalline materials (Faber and Blanton, Reference Faber and Blanton2008). These capabilities are all embedded with the PDF-4 databases. In each case, the ICDD worked with field specialists from its international membership who work in international research facilities, and guided the ICDD in product development. Many of these methods have been combined, leading to a series of tools that can be used in the analysis of nanomaterials, where crystalline, nanocrystalline, and amorphous domains intersect (Fawcett et al., Reference Fawcett, Kabekkodu, Blanton, Crowder and Blanton2015).
(Color online) Electron diffraction analysis module contained in the PDF-4 databases. Top figure demonstrates from the left; nanomaterial, ring, and spot pattern reference simulations. The bottom figure shows experimental data that is overlaid with a reference simulation to identify the material and its crystallographic orientation.
(Color online) Identification of materials from a constant wavelength neutron diffraction experiment (top) and a synchrotron diffraction experiment (bottom). Each module has an ability to import experimental data and match it to reference data that has been modified for instrument type, detector (1D or 2D), wavelength, and radiation type.
ICDD continues to do fundamental development of the original and well-proven search/match algorithms that include the Hanawalt and Fink methods. This work started with Faber et al. (Reference Faber, Weth and Bridge2004), expanding the permuted indexes to eight lines and developing new match evaluation criteria.
In recent years, data processing has been added, as well as new software modules for better residual analysis, pattern matching, weighting algorithms, and semi-quantitative analysis methods.
III. CONCLUSIONS
The PDF is all about results. The PDF started with the publication, Chemical Analysis by X-ray Diffraction (Hanawalt et al., Reference Hanawalt, Rinn and Frevel1938). The database is now sold in over 120 countries and used by the best materials research laboratories in the world, including most national laboratories. The last 75 years has seen dramatic changes in the size, quality, and efficacy of the PDF database and a move from phase identification into a broad material characterization tool. The ICDD has always relied on its member scientists to recommend changes and develop improvements to the database. In recent years, these teams have enhanced the ability to do neutron and synchrotron diffraction analyses (Faber et al., Reference Faber, Crowder, Blanton, Kabekkodu, Gourdon, Blanton and Fawcett2015), study polymers (Fawcett et al., Reference Fawcett, Crowder, Kabekkodu, Needham, Kaduk, Blanton, Petkov, Bucher and Shpanchenko2013; Gates et al., Reference Gates, Blanton and Fawcett2014) and pharmaceuticals (Fawcett et al., Reference Fawcett, Faber, Needham, Kabekkodu, Hubbard and Kaduk2006; Kaduk et al., Reference Kaduk, Crowder, Zhong, Fawcett and Suchomel2014), analyze modulated structures, and develop new tools for analyzing nanomaterials (Scardi et al., Reference Scardi, Leoni and Faber2006; Fawcett et al, Reference Fawcett, Kabekkodu, Blanton, Crowder and Blanton2015). In addition, a series of technical bulletins with case studies have been published by the ICDD (ICDD, 2014a, 2014b, 2015, 2017). The publications cited above give numerous examples of the analysis capabilities of PDF-4 products in phase identification, quantitation, and crystallite size and crystallinity measurements. These analyses were performed on some of the world's top selling drugs, the world's most common biomaterial – cellulose, mining samples, and new energy materials establishing a direct link between data quality, targeted design, and better results.
