In the spring of 2014, art historians Louisa Wood Ruby and Samantha Deutch approached Claudio Silva and Juliana Freire of the Tandon School of Engineering to see if they would be interested in collaborating to build a research tool that would assist the daily work of art historians. Wood Ruby and Deutch offered NYU two ideas, one which they hoped would connect information held in separate repositories within the Frick Art Reference Library: the Photoarchive and the Archives Directory for the History of Collecting in America.Footnote ² Linking the data in these repositories had the potential to reveal undiscovered insights, and strengthen the library's catalogue entries. The second idea, and what ultimately led to the creation of ARIES, was a digital tool that would mimic and enhance methods employed by art historians when working with images in the physical world: organizing and reorganizing images (photographs, slides, transparencies, xeroxes), making connections between them, finding similarities and differences, and charting stylistic changes over time or geography using visualizations. Observations made by art historians using analogue methodologies often lead to further research. With the advent of digital images, art historians lacked the technology capable of replacing what they had been able to accomplish in the analogue world.

Wood Ruby and Deutch also realized that art historians, having spent a great deal of their education learning their craft—the art historical canon, subject specialties, theories, and methodologies—needed an out-of-the-box solution that would not require them to learn the trade of yet another discipline—that of computer science and engineering. Their work necessitated a simple technological answer that would offer hesitant scholars a taste for what emerging technologies might provide them and supply a shortcut for those who already had them in their methodological toolbox. Thanks to a successful collaboration, participants' willingness to learn from each other, and a $200,000 gift from an anonymous donor, ARIES is now available in BETA form at www.artimageexplorationspace.com.

The following will explore a series of examples drawn from test images and data to demonstrate how ARIES's features might benefit art historians. The first example examines how ARIES might be used when investigating an artist's workshop practices or analysing possible stylistic changes over time. The Flemish painter Sir Anthony van Dyck (Flemish, 1599–1641), is considered to have used repeated formulas when painting portraits.Footnote ³ This practice most likely contributed to his prolific output as a portraitist. To intently scrutinize this theory, we used the Overlay, Relative Size, Rect (rectangle), and Lens tools in ARIES to examine five different portraits of women by Van Dyck from The Frick Collection (figure 1).Footnote ⁴ After bringing the images into the Image Menu and the workspace, selecting Relative Size, and scaling up for close analysis, we stacked the images on top of one another and carefully lined up the features; in this case, the hairline, eyebrows, and nose of each of the sitters. The comparison is stunning; not only do these features line up nicely for each sitter, but their shoulders and clothing do as well. The features line up whether or not the sitters are wearing a collar, are painted in full or three-quarter-length, are sitting or standing (figure 2).

Fig. 1.

Images of oil paintings by Anthony van Dyck in ARIES and shown in relative size to one another, images from left to right, and top to bottom (title, date, and accession number): Genoese Noblewoman, ca. 1625–27, (1914.1.43); Lady Anne Carey, Later Viscountess Claneboye and Countess of Clanbrassil, ca. 1636 (1917.1.37); James Stanley, Lord Strange, Later Seventh Earl of Derby, with His Wife, Charlotte, and Their Daughter, ca. 1636 (1913.1.40); Margareta de Vos, ca. 1620 (1909.1.42); Marchesa Giovanna Cattaneo, ca. 1622–27 (1907.1.41). New York, The Frick Collection. Note that the family portrait is shown inversed so that female figure is facing the same direction as the sitters in the other paintings.

Fig. 2.

Overlay and Relative Size: Same five portraits by Van Dyck featured in figure 1, shown in relatives, features aligned, and stacked one on top of the other.

By utilizing ARIES's Rect or Lens applications, we could conduct an even closer examination of the five paintings by Van Dyck, which span a little over a decade, ca. 1621 through ca. 1636. For each feature, we looked at two portraits of very different sitters: Marchesa Giovanna Cattaneo, 1622–27, and Margareta de Vos, ca. 1620. After using the Relative Size function and overlaying the works of art as above, we used the Rect application and defined an area we wished to magnify. ARIES then showed the images on either side of a generated heatmap. The heatmap normalizes the pictures, distilling the features, thus allowing us to see similarities and differences more clearly (figure 3). When used on the Van Dyck pictures, the heatmap emphasizes how the lips, eyebrows, hairline, and shoulders line up; even the mid-seams of the dresses and collars shadow each other.

Fig. 3.

Rect and Match Point tools: comparison of two portraits by Van Dyck the Marchesa Giovanna Cattaneo, and Margareta de Vos.

We then wished to examine congruent areas within each painting more closely. Using the Lens function and defining corresponding elements in each canvas with the Match Point tool, we eliminate extra space between the images and the selected points of reference, allowing us to move through the works of art with more precision (figure 4). In the example provided, we chose the outside corner of the sitters' left eye as the comparison point. When moving through the canvases, we discover that the garments' fabric—at the left shoulders, the collar's peak, and mid-seams of the dresses—all aligned with one another. Moreover, the outlines of the sitters’ foreheads, placement of their hairlines, and other features all hold similar positions within each of the compositions. As observed when utilizing the Overlay function in a previous example, each of the paintings adheres to a similar pattern: it is Van Dyck's placement of the sitter's arms, hand gestures, costume choice, and other added details that individualizes each of the portraits. These few examples underscore Van Dyck's repetitious use of formulas in portraiture, and perhaps this practice is more pervasive in his oeuvre than has been previously speculated.

Fig. 4.

Lens: comparing Van Dyck's Marchesa Giovanna Cattaneo, and Margareta de Vos.

Following the close examinations in earlier examples, we wanted to see if we could identify possible trends or stylistic changes in Van Dyck's portraits using ARIES's Timeline feature. We augmented our original group of images by adding paintings from the National Gallery of Art, Washington, D.C., allowing for a more robust side by side comparison in order of creation date (figure 5). One can imagine how useful ARIES would be when exploring a painting alongside its infrared radiograph, preparatory drawings, or portraits authored by contemporaries.

Fig. 5.

Timeline: thirteen portraits of women by Van Dyck from the Frick Collection and the National Gallery of Art.

The examples above have illustrated how ARIES allows for multiple ways to compare images, from using dynamic overlays analogous, to a physical lightbox, to advanced image analysis and feature–matching functions available only through computational image processing. ARIES also offers users another out-of-the-box visualization, the Geochart. Creating a Geochart or map pinpointing where Van Dyck's portraits, used in the examples mentioned above, are currently located would not be very interesting. The paintings are only from two venues: The Frick Collection and The National Gallery of Art. However, using the map feature in a very different project, one illustrating where all portraits by Anthony van Dyck are currently located, could be intriguing in studying the history of collecting and art markets. Similarly, ARIES could be extremely helpful when tracing an individual's now-dispersed art collection (figure 6).

Fig. 6.

Relative Size and Group: works of art, in relative size to one another, grouped tighter as where they were displayed in Mary J. Morgan's Picture Gallery in 1884.

In figure 6, you see a project featuring paintings once owned by the art collector Mary J. Morgan (1823–1885), all of which hung in her picture gallery on the fourth floor of her New York City brownstone.Footnote ⁵ Morgan died intestate, and her collection was dispersed at an auction in 1886 which set a record for a private art collection, $1.2 million. The record would not be broken for another twenty-four years, when in 1910, the Chicago financier Charles Yerkes's (1837–1905) paintings sold for roughly $1.7 million.Footnote ⁶ The buyers at Morgan's sale included such familiar collecting figures as Baltimore's William T. Walters (1820–1894), New York's Henry G. Marquand (1819–1902), and William Rockefeller (1841–1922).Footnote ⁷ In ARIES, we can easily visualize the current or last known locations for each of Morgan's pictures (figure 7).Footnote ⁸

Fig. 7.

Geochart: illustrating current, or last known locations of paintings once owned by Mary J. Morgan.

Unfortunately, there are no existing photographs of Morgan's home's interior, and her brownstone was demolished at the turn of the twentieth century to make way for Madison Squares’ Brunswick Building.Footnote ⁹ However, an 1884 publication, Mrs. Morgan's Collection of Paintings, details where each picture was installed in her home at that time.Footnote ¹⁰ Entering this information into an ARIES project allows us to recreate Morgan's aesthetic and observe pictures alongside others once installed in one room or area of her house (see figure 6).Footnote ¹¹ By considering these works of art together, we might discern trends in how Morgan may have grouped works of art: school, colour, genre, creation, or purchase date. Using the Wall feature, we may reasonably hypothesize how selected pictures once hung together on a single wall (figure 8).Footnote ¹²

Fig. 8.

Wall: demonstrating how six painting once in Mary J. Morgan's collection might have been exhibited together in her Picture Gallery.

In addition to creating visualizations in ARIES, users can sort the images in the Image Menu by year created, artist, dimensions, and techniques (in ascending or descending order). Similarly, users can search the photos in their projects using the metadata headings such as artist, title, creation date, technique, provenance, and annotations. They can edit their images by applying filters such as contrast, greyscale, and brightness. They can crop their pictures, and when trimming a portion of a work of art, the resulting image retains its dimensions based on those entered for the original image.Footnote ¹³ Further, the system encourages collaborations, allowing for projects to be shared with other ARIES users. Finally, users may import and export data in and out of ARIES; thus, the data is not permanently deposited in the software program and may be used elsewhere.

From the examples shown above, it is evident that ARIES presents a comprehensive toolset for supporting researchers in the exploration of images while, at the same time, offering them visualization applications such as Timelines and Geocharts that do not require them to have coding skills. ARIES is in BETA, the code is continuously being improved, and for now, only available via the internet. It has always been our wish that it would have a computer application component, allowing scholars to work with images in the field without the requirement of a reliable internet connection. We have workshopped many more features, including working with images within a user-defined three-dimensional space, but this will require more development and funding. In the meantime, we continue to work with our partners at New York University's Tandon School of Engineering and Brazil's Universidade Federal Fluminense with an eye on the future: enhancements and added functionalities that can answer the ever-changing research demands of the art library patron.