The Work of Archaeology in the Age of Digital Surrogacy

This is the tenth in a series of posts exploring 3D modeling in Mediterranean and European archaeology. For more in this series click here. We hope these papers will start a discussion either in the comments of the blog or on Twitter using the #3DMedArch hashtag.

Adam Rabinowitz, The University of Texas at Austin

I’m very glad that Bill has run this series of 3D Thursday blog posts, because they have demonstrated with particular clarity that field archaeology is at a turning-point in its engagement with three-dimensional visualization. A decade ago, a series of posts on 3D technologies in archaeology would have been concerned mainly with computer-aided virtual reconstructions and immersive environments or with the use of laser scanners. This series, however, has highlighted an emerging common interest in the use of computational photography to create photorealistic 3D representations of archaeological material. 

3DThrusday

Previous posts have emphasized the way new software and methods have lowered the bar for the creation of high-quality 3D or 2.5D models of physical objects. James Newhard suggests that these tools are becoming wrenches in the standard archaeologist’s toolkit, and I think this trend is likely to intensify over the next decade, especially as drone-based photography becomes a matter of course. Such tools offer two enormous advantages: the enrichment of data collection in the field, which in turn enhances the archaeologist’s interpretive process (a point made by Brandon Olson, Rachel Opitz, and Eric Poehler); and the ability to make distant objects available for scholarly autopsy, as Dimitri Nakassis argues in his post on Reflectance Transformation Imaging. As an added benefit, quick, cheap 3D representations created through computational photography provide a new way for mass audiences to engage with the physicality and materiality of objects, both in an academic publishing environment, as Andrew Reinhard enthusiastically affirms, and, as Sebastian Heath demonstrates, in connection with active excavations and museum collections as well. That last point was driven home just yesterday by the public launch of the Smithsonian X 3D project, which presents models of objects in the institute’s collections, complete with downloads suitable for 3D printing (though it must be noted that most of this data was captured by more traditional laser scanning). 

These contributions also highlight the extraordinary extent to which this shift has been driven by just two recent developments: the popularization of the Polynomial Texture Mapping algorithm developed by Tom Malzbender (Malzbender, Gelb, and Wolters 2001), in large part through the efforts of Cultural Heritage Imaging; and the rapid improvement of the algorithms that produce 3D models using structure-from-motion (sometimes casually referred to as “photogrammetry”), represented particularly by Agisoft’s Photoscan software and Autodesk’s 123DCatch app. Many of us had carried out experiments with computational photography and structure-from-motion in the early 2000s (e.g. Tschauner and Siveroni Salinas 2007; Rabinowitz et al. 2007), but the new tools have transformed a laborious manual process involving a certain amount of technical expertise in both image capture and transformation into a fully automated workflow that even corrects for the defects of the photographer (see now De Reu et al. 2013 and Olson et al. 2013).

Pmp CH06SR 84 m

3D model of bedrock features uncovered at Chersonesos in the 2006 field season, created from photos with PhotoModeler Pro. The model was georeferenced in ESRI’s ArcScene, and then exported to a 3D PDF using Adobe Acrobat 9 Professional Extended (sadly no longer available).
Click to download the 3D Pdf 

The attraction is powerful: unlike conjectural 3D reconstructions or the pure geometry offered by laser-scan point clouds, computational photography seems to promise unmediated access to the physical reality of existing material remains. As Heath points out, and as can be seen even more dramatically in the recent use of Google Glass to capture a 3D digital model of a head of Marcus Aurelius in the Walters Art Gallery, the technology also offers significant possibilities for open access and democratization. Anyone with a smartphone with a camera and a few minutes can create a passable 3D model of an archaeological object or work of art and post it online. The examples below give a sense of the variability in the effort required and the quality of the results. Above is a model of a cast of the Belvedere Torso in the Blanton Museum of Art, created in the space of about five minutes with 16 hasty photographs taken with an iPhone and processed with AutoDesk’s 123DCatch app. Below is a model of an inscription found at Troy and now held by the UT Department of Classics — this more careful representation involved 37 photographs and a light source, and took about 20 minutes to create, again with 123DCatch.

Belvedere Torso

Belvedere Torso (Click to view in 3D)

BPTVp

Photogrammetrical model of Troy inscription (Click to view in 3D)

I’ve called these 3D digital objects “models” and “representations”, but they are perhaps more accurately described as “digital surrogates”. “Digital surrogate” is a term of art used in the libraries and archives to refer to any digital representation of a work that exists in the physical world (a thumbnail, a metadata record, a digital image). More commonly, however, the term indicates a faithful digital copy that seeks to represent an analogue original as accurately and in as much detail as possible: “By definition, a surrogate can be used in place of the original. If a surrogate is electronic, the same files can be used both internally (to protect the original when the surrogate is of sufficient quality and accuracy to stand in place of the original), and externally (to provide wider access for those who might otherwise be unable to view or study an original)” (Grycz 2006, 34). 

Not all surrogates are “of sufficient quality” to serve as substitutes for originals, of course, and there is still a lot of discussion about the extent to which even the highest-resolution scan can replace contact with an original document (the term is almost always used to talk about two-dimensional objects like manuscripts or photographs). Nevertheless, the notion of the “digital surrogate” reflects an underlying assumption that a digital reproduction ought to be able to stand in for the real thing — and therefore it is particularly appropriate for three-dimensional digital objects that seek to reproduce the visual and spatial characteristics of objects in the real world. A good surrogate isn’t merely a copy: it is supposed to provide, in some sense, access to the original, now made ubiquitous and opened for inspection on a level of detail that the original itself might not allow. Brandon Olson makes just such a point in his reflections on the use of 3D models in the first post of this series.

Popular accounts of the rise of computational photography already treat the surrogate as if it provided access to the reality of the physical original. In the most recent issue of Archaeology, for example, a brief article on the use of drone photography in Peru concludes with the excavator’s somewhat breathless claim that “[y]ou can model every single stone” of a site (Swaminathan 2013). And even sober NPR correspondent Robert Siegel, covering the Smithsonian 3D project, is compelled to ask whether digital reproduction techniques will become so good that they will allow the creation of perfect “forgeries” that are indistinguishable from the originals. But when we look at these surrogates, are we really being afforded closer contact with reality? Or do these exciting, rapid, “disruptive” (to use a word very much of the moment) changes mask some underlying epistemological and methodological problems? I think it’s worth attempting to establish a theoretical framework to help us understand not only the benefits conferred by these technological advances, but also what is really happening as we leap from original to digital surrogate.

A starting point for this discussion is offered by Walter Benjamin’s oft-cited essay “The Work of Art in the Age of Mechanical Reproduction”, in which the cultural critic reacts to the impact of new technologies on the social role of Art (with a capital “A”). These new technologies made possible the large-scale dissemination of faithful representations of unique artworks and the exploration of visual phenomena that could not be captured through ordinary perception. Benjamin was concerned that such reproductions would destroy what he called the “aura” of original, “authentic” works of art: that is, the artwork’s “presence in time and space, its unique existence at the place where it happens to be” (Benjamin 1968, 220). The availability of copies, mass-produced for mass consumption, led to “the desire of contemporary masses to bring things ‘closer’ spatially and humanly, which is just as ardent as their bent toward overcoming the uniqueness of every reality by accepting its reproduction.” “Every day,” he continues, “the urge grows stronger to get hold of an object at very close range by way of its likeness, its reproduction” (223).

The essay was published in 1936, and the new technologies that inspired Benjamin’s concerns were  photography and its offshoot, cinema. But he might as well be speaking of the three-dimensional representations of ancient “originals” that many of the posts in this series focus on. The mechanical reproduction has given way to the digital surrogate — a representation of an analog original in the form of the ones and zeroes of binary code — but less has changed in the last 80 years than one might expect, especially since these digital surrogates continue to be generated with the same old new technology of photography. 

The effects of digital surrogates mirror those Benjamin ascribes to photography and film: they distance us from the unique physical thing-ness of that which they represent while allowing us to manipulate reality in ways that the original would not permit. Compare, for example, his claim that the “enlargement of a snapshot does not simply render more precise what in any case was visible, though unclear: it reveals entirely new structural formations of the subject” (236) with Eric Poehler’s statement that 3D surrogates of Pompeian architecture allow views that would not be physically possible for an observer present in person. Or compare the 3D isolation of stratigraphic sequences that Rachel Opitz describes, free from extraneous layers, excavators, plants, tools, etc., with Benjamin’s comment on the invisibility in movies of the cameras, lights, and personnel needed to make them: “[t]he equipment-free aspect of reality here has become the height of artifice” (233).

Two fundamental points emerge from Benjamin’s critique. One is that a surrogate is not the original, nor does it represent reality: it is the product of “artifice”, of techniques and processes that are themselves not visible in the end product. The other is that photography is fundamentally different from earlier two-dimensional copying techniques: “for the first time in the process of pictorial reproduction, photography freed the hand of the most important artistic functions which henceforth devolved only upon the eye looking into a lens” (219). He has in mind techniques like woodcuts and engravings, which produced multiple copies of a single original. The engraved original itself might be a reproduction of an existing work of art — but in that case, too, the engraving was also an artistic interpretation, not a straightforward reproduction, and the product of the creativity and skill of the engraver (Fyfe 2004).  

Image

Engraving depicting an intaglio gem from von Stosch’s collection, from Winckelmann’s 1760 publication. This “digital surrogate” of the printed illustration is housed in the Arachne database of the German Archaeological Institute, where it is also linked to records associated with the publication, the original gem, and casts of gem impressions.

The tension between original and reproduction, art and artifice, access and authenticity, is not a new one in Mediterranean archaeology. In fact, long before the discovery of photography, the birth of ancient art history was entwined with the creation of 3D surrogates through increasingly mechanical means. In the Renaissance and the early Baroque period, sculptors like Bernini prided themselves on their ability to imitate or add to ancient sculptures, and gem-cutters produced new intaglio gemstones based on Greek and Roman originals. By the 17th century, however, broader interest in Classical Antiquity and its iconography led to a market for casts of impressions from the ancient gems themselves. These casts were collected in dactyliothecae that served as iconographic encyclopedias (Knüppel 2009). Dactyliothecae functioned in the same way as the digital 3D surrogates  we’ve been discussing: they allowed the close study of 1:1 representations of absent objects to extract “authentic” visual information, and they offered mass access to originals that were scattered among different collections across Europe. At the same time, large-scale ancient sculpture was also being cast in plaster, again to allow the experience of the “real” (or hyper-real!) form and volume of absent originals and to permit the centralized collection of works otherwise dispersed in geographic space (Borbein 2000). 

The mechanical reproduction processes involved meant that casts of gem impressions and sculptures were usually at a scale of more or less 1:1, which added to the sense of access to an original. That impression was reinforced by artifice as well: casts of sculpture could be acquired with various finishes meant to evoke marble patinas, different stones, even metals. Here, however, we move from authenticity to verisimilitude — that is, the finishes didn’t necessary reproduce the appearance of the original, but evoked the way an object like this was supposed to look. The distinction between truth and truthiness extended to the form of sculptural casts as well. Because so many of the sculptures reproduced were already Roman “reproductions” of Greek statues, they existed in multiple exemplars, each of which might have better preserved components (one might have a head but no arms, another arms but no head, etc.). Some casts were thus actually amalgams of the best-preserved parts of different originals. In other cases, minor additions or changes were made to better suit the appearance of the reproduction to the tastes or expectations of the consumer. The introduction of these modifications during the technical process that created the cast is invisible to the viewer. Instead, changes made to make a cast look more “Classical” shape in turn our notion of what the “Classical” is supposed to look like. For all the use of mechanical reproduction, then, these apparently straightforward surrogates have a problematic relationship with their originals.

If casts offer a Victorian analogy for the 3D digital surrogates created with structure-from-motion algorithms, the venerable paper-pulp epigraphic squeeze is the analog ancestor of the reflectance transformation image. Again, the crucial quality of the squeeze is its mechanical reproduction, at a 1:1 scale, of the physical surface features of the original, without the interpretive intervention involved in the publication of measurements and transcriptions. Squeezes were generally produced by and for scholars, and were not subject to the sorts of interventions that casts are. On the other hand, the quality of a given squeeze depends on the technical abilities and equipment of the squeeze-maker. Like casts, then, squeezes are not simply “physical surrogates” for originals, but objects of artifice derived from originals through specific processes mediated by their creators and often conditioned by preconceived ideas about how the final product should look. 

I have spent this time on physical 3D surrogates for two reasons. First, they highlight the importance of scale and measurement for the usefulness of a surrogate. Squeezes and casts are valuable precisely because they are 1:1 in scale, and can thus allow measurement of the original by proxy. Second, as Benjamin argues for mechanical reproductions in general, they have a problematic relationship with their originals. On one hand, some surrogates have preserved information about originals that are now lost or damaged — early examples of the LOCKSS (Lots of Copies Keeps Stuff Safe) principle. And squeezes continue to be used by epigraphers for off-site study of inscriptions, even when the original still exists. On the other hand, plaster casts have had a generally unsuccessful run. After being introduced as better representations of the reality of the originals than the originals themselves in the 18th century, they had fallen out of favor by the middle of the 20th, perhaps in part as a result of the concern of critics like Benjamin with authenticity and “aura”. Many collections were destroyed; others persisted in a sort of half-life as curiosities, but not as resources for scholarly inquiry. 

Fundamentally, casts have not been able to maintain their initial value as “cultural capital”, in sociologist Pierre Bourdieu’s sense. This is in part due to their diffusion, which makes it difficult to control their use as symbols of distinction: the cultural capital provided by a full-scale cast of the Farnese Hercules is somewhat diminished by the existence of the SkyMall version. But it is also due to the opacity of the process by which they were created and the lack of information about the context of that creation. This is true of all varieties of physical 3D surrogate. A cast made for the tastes of the commercial market may not be trustworthy for academic research, and a squeeze made by an untrained beginner may be a poor representation of the original.

The history of these physical surrogates and Benjamin’s critique of mechanical reproduction offer us a foundation for a more considered approach to digital 3D surrogates. I’d like to conclude with two starting points for further theorization and discussion.

First, I think it will be of fundamental importance to remember that the digital 3D model is not a true surrogate for the original, even when derived from photographs. This is particularly true for models of archaeological remains in the process of excavation, which will never again be available for first-hand autopsy. I would argue, therefore, that it will be critical not to throw out the handicraft in archaeological documentation — the measured hand-drawn plans and interpretive sketches that, like engravings of artworks, admit that they are representations that seek to highlight the choices and ideas of the creator, and do not claim to be mechanical reproductions of objective reality. Rachel Optiz makes the same observation in her post, and I think it’s supported by a growing body of research on the special ways in which our brains interact with drawing surfaces and writing instruments (we need more studies on haptics and embodied cognition in archaeology!). It would be a grave mistake to lose the skills necessary to create interpretive drawings in our rush to adopt quick, easy, and powerful computational-photographic methods.

Second, I would like to propose a set of four basic principles for publication and archiving to ensure the future scholarly usefulness of the 3D digital surrogates derived from computational photography. Some of these are already in place, at least in some forward-looking projects; others will require the development of new tools. All of them, I think, will be extremely important as computational-photographic methods become more powerful, more democratic, and more black-boxy for most users. 

1) Measurement. To ensure that scholars can reuse the 3D data you’re generating, the models have to include some user-accessible information about scale and units, ideally in the form of both in-interface measurement tools and a platform-independent marker (like a clearly-marked meter stick included in the image). This seems like an easy one, but it’s trickier than it looks. Andrew Reinhard noted that keeping scale constant would be hard across different publication formats, and the models of the portico and the marble feet in Sebastian Heath’s post highlight the effects of lack of scale. When we consult 2D documentation, we frequently make our own measurements using either the scale statements or the scale bars provided in the illustration. In order to be truly useful, 3D model-delivery platforms must offer the same opportunities (Adobe’s 3D PDFs include a measurement tool, but the basic Unity interface and p3d.in do not).  

2) Raw data. I feel that we are ethically bound to provide not only models, but our original raw data for reuse wherever possible. The importance of this — and the problems with access that turns out to be open in name only — has been reinforced by a recent case involving CyArk and proprietary control of laser-scanning data (for details, see the original post here, with additional discussion here). These digital objects are at least one step closer to the reality of the original, and they also make it possible to reprocess the data as more powerful tools and algorithms come online.

3) Metadata. The raw data, of course, are of little use without comprehensive metadata that not only describe file formats, creation dates, etc., but also indicate what those raw data represent. This is going to be especially important for the enormous batches of photographs generated for the purposes of computational photography. It’s great if the final model has metadata that tell us it’s a Pompeiian building — but I would argue that we need to ensure that every photograph in the sequence has metadata that describes at least the technical details of the photographic file AND the basic identifying information for the object or monument it represents AND the context of its creation, including date, actors, project, etc.

4) Process history. A 3D digital surrogate isn’t the same as the physical original not just because of its format, but because it is the product of the computational manipulation of a series of intervening digital surrogates, whether photographs or laser scans. Our ability to trust and use the model depends on our ability to trace — and ideally to walk back — the processes by which it was generated, including parameters, assumptions, and fudges. Systems must be in place to capture and store this information so that users in the future will understand how a model was produced and therefore how it can or cannot be used. Cultural Heritage Imaging has made a lot of progress toward this goal with their work on empirical provenience and the “digital lab notebook”, and I would like to see similar practices adopted by all archaeological and heritage projects that use computational photography.

These principles come with a handy acronym: MRMPH, recognizable as the noise you make when you’re asked a question while taking a really large bite of something or while just waking up (I’ve never been good at generating slick acronyms). In all seriousness, though, the application of these principles to the publication and archiving of 3D digital surrogates will mark a watershed in archaeology: it will be the first time since the birth of our discipline that the archaeological record will grow richer, rather than poorer, with age, since new algorithms and software will permit the ever-more precise and accurate reprocessing of digital photographs for the extraction of 3D information. As 3D printing takes off, we will also have a new opportunity to recreate the archaeological record through widely available physical surrogates, which offer their own advantages for accessibility and interpretation. All this will be possible, however, only if we recognize with Walter Benjamin that reproductions cannot be stand-ins for originals, and acknowledge that digital surrogates in particular have their own independent reality as objects or works requiring their own documentation and explanation (Manovich 2001; Cameron 2007). 

Author biography: Adam Rabinowitz is an Assistant Professor in the Department of Classics at The University of Texas at Austin and the Assistant Director of UT’s Institute of Classical Archaeology.

Works cited

Benjamin, W. 1968 [1936]. “The Work of Art in the Age of Mechanical Reproduction.” Reprinted in H. Arendt, ed., Illuminations. Essays and Reflections, translated by H. Zohn. New York: Schocken Books, 217-251.

Borbein, A. 2000. “Zur Geschichte der Wertschätzung und Verwendung von Gipsabgüssen antiker Skulpturen (insbesondere in Deutschland und in Berlin).” In H. Lavagne and F. Queyrel, eds., Les moulages de sculptures antiques et l’histoire de l’archéologie. Actes du colloque international Paris, 24 octobre 1997. Geneva, 29-43. Text translated by B. Frischer, available at: http://www.digitalsculpture.org/casts/borbein/.

Cameron, F. 2007. “Beyond the Cult of the Replicant–Museums and Historical Digital Objects: Traditional Concerns, New Discourses.” In F. Cameron and S. Kenderdine, eds., Theorizing Digital Cultural Heritage. A Critical Discourse. Cambridge and London: The MIT Press, 49-76.

de Reu, J., G. Plets, G. Verhoeven, P. De Smedt, M. Bats, B. Cherrett, W. De Maeyer, J. Deconynck, D. Herremans, P. Laloo, M. Van Meirvenne, and W. De Clercq. 2013. “Towards a Three-Dimensional Cost-Effective Registration of the Archaeological Heritage.” Journal of Archaeological Science 40: 1108-1121.

Fyfe, G. 2004. “Reproductions, Cultural Capital, and Museums: Aspects of the Culture of Copies.” museum and society 2, 1: 47-67.

Grycz, C. J. 2006. “Digitising Rare Books and Manuscripts.” In L. W. MacDonald, ed., Digital Heritage. London: Elsevier, 33-68.

Knüppel, H. C. 2009. Daktyliotheken: Konzepte einer historischen Publikationsform. Rutzen.

Malzbender, T., D. Gelb, and H. Wolters. 2001. “Polynomial Texture Maps.” In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 519-528. Available at http://www.hpl.hp.com/research/ptm/papers/ptm.pdf.

Manovich, L. 2001. The Language of New Media. Cambridge: The MIT Press.

Olson, B. R., R. A. Placchetti, J. Quartermaine, and A. E. Killebrew. 2013. “The Tel Akko Total Archaeology Project (Akko, Israel): Assessing the Suitability of Multi-Scale 3D Field Recording in Archaeology.” Journal of Field Archaeology 38: 244-262.

Rabinowitz, A., S. Eve, and J. Trelogan. 2007. “Precision, accuracy, and the fate of the data: experiments in site recording at Chersonesos, Ukraine.” In J. Clark and E. Hagemeister, eds., Digital Discovery: Exploring New Frontiers in Human Heritage, CAA 2006. Budapest: Archeolingua, 243-256.

Swaminathan, N. 2013. “Drones Enter the Archaeologist’s Toolkit.” Archaeology 66, 6: 22.

Tschauner, H., and S. Siveroni Salinas. 2007. “Stratigraphic modeling and 3D spatial analysis using photogrammetry and octree spatial decomposition.” In J. Clark and E. Hagemeister, eds., Digital Discovery: Exploring New Frontiers in Human Heritage, CAA 2006. Budapest: Archeolingua, 256-270.

 

6 Comments

  1. Adam, this is a wonderful post. It’s a great idea to publish our raw data except that our raw data are enormous. This summer we were producing 40 GB of photographs in a single day. That’s not including all the raw data from the 3D scanner. So we’re talking about nearly a TB in photographs alone in a single season. Extrapolate that out over the entire project, and you’re talking about 4-5 TB in photographic raw data plus the 3D scanner raw data, for one artifact class at one site. That boggles the mind.

    Reply

    1. Dimitri, you’re absolutely right. This is why we need better archiving strategies and infrastructure for archaeological data. I don’t think this is going to be a question of the expense of storage media, given Moore’s law, but I do think it will be an issue of having repositories willing to take this on (and serve the data publicly) at prices most projects can afford. To put your dataset in perspective, at at tDAR’s non-discounted rate, you’d have to pay $75K to deposit it all with them (according to the pricing calculator at http://www.tdar.org/about/pricing/). We have about 5 terabytes of born-digital and digitized data for Chersonesos and Metaponto already. So what are we going to do about this? I think we owe it to future archaeologists to figure it out, or they’re going to have a lot of dead portable hard drives to deal with.

      Reply

      1. Sorry, messed up my terabyte to megabyte conversion — 5TB, without a discount, would actually cost $2.6 MILLION dollars to deposit with tDAR, according to their pricing calculator.

      2. Adam and Dimitri,

        There are indeed challenges here, yet the requirement of allowing access to your original data for verification, as well as reuse is so critical.

        I also think we can be smarter about this in terms of what gets saved in what formats, etc. For example, I might shoot 80 images of an RTI subject, because it was difficult to work with in some way, such as a really shiny surface, and I wanted to over sample. Once I go through all those images and determine a reasonable set that has good coverage of the subject in terms of light positions, do I really need to save all 80 images? I realize these are tough choices, maybe future software will deal better with specular surfaces, and so those additional images could be used. However, at what point is there a diminishing return?

        Also, I think that we need some work to determine “good practice” for the number of images to take for different techniques and situations. Often the number of images isn’t the determining factor (or only factor) for a high quality result. You have to take the proper sequence, with the right camera settings, good exposure and getting things in focus. You can have a large number of images that give a poorer result than a much smaller set that were properly captured.

        I realize that what I’m saying here doesn’t even begin to solve the problem, and there are many more issues with the longevity of this data. Still, I agree with Adam, that we have to find a way to save original data and make it accessible or we aren’t doing scientific imaging.

        Carla

  2. Reblogged this on AIA Geospatial Interest Group and commented:
    3D Thursday!

    Reply

  3. Great post Adam! You know we have ben thinking about these issues for a long time. I really love your examples that show these issues aren’t new, though of course the scale with which we can create surrogates (real facsimiles as well as digital surrogates) has changed and is changing dramatically.

    Thank you for keeping this conversation going!

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s