Augmented Vision(s): Visual Rhetoric in the Augmented Reality Apparatus

Jason Crider, University of Florida

---

Abstract

This article argues that the increasing ubiquity of augmented reality (AR) technologies demands not only new critical approaches and methods, but a whole new poetics. In doing so, I outline some of the current applications of AR, both by corporate entities and by humanities and activist practitioners, in order to demonstrate the unique affordances granted by AR, as well as some of the emerging ethical considerations this new technology beckons. Ultimately, this article draws on rhetorical theories of visuality to outline a new poetics for augmented reality technology
that allows us to renegotiate our understanding of and relationship to our discursive technologies.

---

Essay

Introduction

“No one knows how to create words and pictures that are meant to be consumed out there in the world.”
–Alexis Madrigal, “The World is Not Enough,” Atlantic

“One cannot simply write about prosthesis when one is automatically, just by virtue of writing, writing prosthesis, entering into prosthetic relations, being prosthetic.”
–David Wills, Prosthesis

In 2017, over two-thirds of the United States’ internet traffic took place through mobile smartphone devices (Molla). As mobile computing practices continue to play such a ubiquitous role in our daily lives, scholars of new media and digital rhetoric should adopt new practices for negotiating not only our communicative practices, but the way we conceptualize and embody space. One of the major affordances mobile computing offers as a writing technology is a means of interacting with “real world” objects and environments through the use of augmented reality (AR) interfaces. As opposed to virtual reality (VR), where a mediating device eclipses a user’s sensory input, AR operates alongside, and interacts with, physical, non-digital environments, and allows users to compose on top of shared spaces, what John Tinnell refers to as information’s ability to be integrated into its most “kairotic environments” (Actionable Media 17). In particular, AR offers scholars, students, artists, and activists new means for cultural reflection and critique that require not only new methodologies, but a new poetics. As Tinnell warns, this new form of digital communication “harbors a formidable set of problems, rhetorical-aesthetic challenges, and ethical considerations” (xix). The paradigm shift signaled by the proliferation of AR interfaces necessitates a restructuring of our civic practices that account for the increasing enmeshment of our informational practices with the visual rhetoric of physical objects and locations. In this article, I argue that the augmented reality apparatus demands new strategies that account for the use of digital media as an embodied practice, and in turn offer a multidisciplinary approach to thinking critically about and alongside AR technologies.

Some work towards an augmented reality poetics has already begun. For example, communication scholar Adriana de Souza e Silva examines the ways in which location-based texts and compositional tools create new affordances and theoretical frameworks for urban and public spaces (“Mobile Narratives”). Maria Engberg and Jay David Bolter discuss the extent to which AR exists as a medium and aesthetic in their introduction to their 2014 special issue of Convergence on AR and mixed reality tech, arguing that AR operates according to what Engberg calls polyaesthetics, and thus requires new analyses that account for its multisensory and proprioceptive qualities (3-6). And avant-garde poet Amaranth Borsuk’s collaboration with programmer Brad Bouse, the hybrid ergodic work Between Page and Screen, uses AR via webcam to “unlock” poems from coded symbols in the book, requiring users to read the poems in the augmented space “projected” from the page. These important works of scholarship demonstrate the wide range of AR’s critical potential and the need for dynamic, multidisciplinary critical approaches to AR as it becomes a more widely adopted cultural practice. As such, I build off of these critiques and uses of AR to argue for new ways of considering its mechanisms, and the potential impact these mechanisms can have not only on our ability to augment our external “reality,” but on our ability to augment (and in turn be augmented by) a new post-human potential via this emergent tech.

As Gregory L. Ulmer argues about the shift from literacy to what he terms “electracy,” new developments in technology and digital media demand entirely new strategies that deal with the ubiquity of the digitally remixed image. Rather than adapting the practices of literacy to digital technologies, Ulmer instead argues that the goal should be “to discover and create an institution and its practices capable of supporting the full potential of the new technology” (Internet Invention 29). In other words, traditional literate practices do not adequately prepare for the digital, image-based logics of the Internet. This move towards a rhetoric of images rings increasingly true with the rise of augmented reality devices and other mixed reality technologies that shift from the strictly virtual into a visual culture where entanglements of digital and analog spaces are progressively more common. When cartoon Pokémon invade your living space, and digital makeup can be applied to a physical face, the hologrammatic workings of AR are no longer confined to the prophetic visions of science fiction. In other words, what augmented reality signals is a possibility for a democratic control of the image through ubiquitous computing to make visual rhetoric externalized and readily demonstrable. As John Tinnell argues,

Given the rapid increase in access to location-based, augmented reality technologies conditioned by the spread of smartphones, as well as the relative ease with which anyone possessing basic computer skills can now contribute content to world browsing platforms, mobile world browsers hold the potential to become a go-to medium for subaltern counterpublics looking to contest popular, corporate, state, or otherwise dominant discourses (All the World’s a Link).

Building off of this, it is crucial to examine the ways in which augmented reality’s unique power as a discursive practice lies in its status as a cognitive prosthetic, one that externalizes and often literalizes invisible processes of imagination and ideology.^[1]I realize that “prosthetic” may be an uncomfortable term as it often has associations with disability studies, but my hope is that this term is not insensitive. Instead I use the term in … Continue reading Donna Haraway argues that it has become impossible to separate the body from technology, and AR expands on this as a biotechnology that allows the posthuman body to be seen as the media and technology that it is.^[2]Although the term “posthuman” has many contested meanings, I will be using it as it problematizes traditional liberal humanist conceptions of the “essential” human. In the … Continue reading My methodology for examining AR blends new media studies with that of the digital humanities in an attempt to develop a hybrid methodology to account for AR as both digital technology and medium. That is, while I offer a new critical vocabulary to discuss augmented reality’s reorientation of our physical experience of the world—a poetics or toolkit that aids us in the same way that the development of literate practices did with alphabetic writing and linear print, and situates AR as a writing technology within a matrix of institution, practice, and identity formation—I also approach AR from a tool-building perspective, and demonstrate how an account of the former can inform the latter. I begin this article by analyzing how AR emerges within a shift towards more digital, mobile, and networked modalities of constructing meaning, and how the digital-as-physical metaphor has facilitated this shift. Ultimately, I will argue that the emergent technology and medium of AR is of critical significance as a catalyst for looking at networked being with a new set of eyes (figuratively as well as literally), where augmented reality acts as what we might term a “cognitive prosthesis.” The prosthesis, or apparatus, of AR opens up cogent new avenues for understanding and articulating the ways the posthuman uses, and is used by, digital modalities. In doing so, AR technologies offer the user potential freedom from believing they are seeing an “unfiltered” gaze of reality, allowing for a bolstered rhetorical agency via a visual ideological awareness.

Augmented Poetics, Cognitive Prosthetics

With the publication of information no longer limited to a book, computer, or device, everything in the “analog” world becomes a potential location for discourse, even if that information does require a mediating device. This new potential for public discourse offers new possibilities for critical and political engagement with physical locations. But, with regards to AR, Sean Morey and John Tinnell ask in Augmented Reality: Innovative Perspectives Across Art, Industry and Humanities, “at what point does this contestation and dialog become commercialized and reterritorialized by the dominant paradigm? Can an AR of the public sphere by the public provide a space for critique?” (14). Writing and rhetoric have served as democratizing disciplines because they allow all literate individuals to participate and communicate in their society, but that literacy is already difficult enough to learn for many. What is emerging, specifically with AR, is a widening gap between those who are able to create content and those who are untrained in or unaware of digitally constructed methods of discourse. Physical spaces might be ripe for public discourse, but only if the public is knowledgeable about how to use the tools of such discourse. Digital humanists debate as to whether or not the ability to code should be a requirement for entry into the sub-discipline^[3] Matthew Gold’s edited collection Debates in the Digital Humanities serves as a great starting point for this., but knowledge and skill at computer coding may soon become a requirement for this emerging type of literacy in general. ^[4]This is, of course, a much larger debate, and corporate interests are backing coding literacy initiatives arguably even more so than educators. See Apple’s Tim Cook’s June 2017 statement to … Continue reading The question AR seems to pose is what does critical, (inter)active agency actually consist of in the politics of 21st Century discourse? In other words, could AR be used by a digital citizenry toward democratic purposes, not just those of military, commerce, or entertainment?

In The Poetics of Augmented Space, Lev Manovich calls for “reconceptualiz[ing] augmentation as an idea and cultural and aesthetic practice rather than as a technology” (220). Manovich makes comparisons between augmented space and architectural practices, arguing that “the design of electronically augmented space can be approached as an architectural problem” and that now “invisible space” needs “a structure, a politics, and a poetics” (237). The question however remains: what does that structure look like? As it stands right now, the effectiveness of public AR texts and installations relies on the specific applications or devices being used. Users are able to choose what mixed reality they subscribe to, like flipping channels on a television, which has the potential to be either a tool of empowerment, or a further system of isolation and estrangement among the populations of the first world.

This idea of subscription is crucial, it seems, to the workings and understandings of augmented reality going forward, as it makes visible the various media ecologies in play at any given time. While AR is most easily conceptualized as a technology, perhaps it is more prudent to configure as a practice, as Manovich states, or, as Vladimir Geroimenko argues, “a novel creative medium [that] is bound to become an organic part of the emerging hybrid world” (313). The significance of this medium, then, is that it has the ability to not only comment on reality, but the ability to visually and aurally alter it. Of course, all commentary in some way alters the perception of reality for the viewer, but augmented reality’s effectiveness lies in its ability to display new realities that are immediately available, and that bypass the need to imagine the interaction between world and commentary, or as Morey and Tinnell call it, “a transparent relationship between word and thing that becomes layered and conflated in real time” (25). Augmented reality’s effectiveness, then, is a result of its ability to actualize the imagination on behalf of the user, making it unnecessary for users to do so themselves.

As a contemporary example, L’Oréal’s Makeup Genius application for smart phones enacts such a performativity of imagination for the viewer. To use the app, users hold their phone up in front of their face, as they would a small mirror, and select from a wide variety of lipsticks, eyeshadows, and other makeups in order to see what they would look like wearing them in real time. If users are present at the physical store, they are even able to scan products on the shelves and receive an immediate overlay that shows them what they would look like without ever trying on or even touching a product. This use of augmented reality as a marketing strategy gives consumers a fun, hands-on experience, but also subtly takes away a step in the decision-making process. Instead of imagining owning a product and then imagining the subsequent use of it, the consumer instead has their imagination literalized for them and their agency in some ways invaded, even if that agency was an illusion to begin with. Or as one could argue from Marshall McLuhan’s work in Understanding Media: The Extensions of Man, when the ability to actualize makeup on oneself in this way is achieved, there is a tradeoff where the imagination is made numb. Media scholars such as Guy Debord and Neil Postman critique an image culture for its potential negative effects on a participant democracy, which begins to show how uses of AR that actualize imagination and create interactive imaging might also, in this way, numb the cognitive interactivity of democratic citizenry, exacerbating the problems already present in an image-based culture.^[5] See Society of the Spectacle and Amusing Ourselves to Death: Public Discourse in the Age of Show Business, respectively.

Even though Makeup Genius may seem like a trivial example, it shows what is at stake when it comes to the ways in which AR mediates imagination and choice. Like most digital tools, AR is marketed as invisible and immediate, even though it is neither, but rather serves as a kind of imagination prosthesis that works outside of the mind or actualizes the imagination. In many instances, the use of these kinds of technologies numbs the imagination, where, in the example of Makeup Genius, the user is still limited to the various constraints put in place by the designers of the application. The makeup is limited in how it is applied and does not, for example, allow the user to draw on their eyes with lipstick, or to smear mascara onto the wall behind them. However, while the application does not grant the same kind of visceral and immediately lived experience for the user as a traditional set of makeup would, it does serve as a prosthetic of the imagination, one that limits decisions while simultaneously expanding choice. For example, dozens of makeup combinations can be sorted through in seconds, a process that would involve hours of “manual” application and reapplication of makeup, so in that sense the “digital” application of the makeup (via the smartphone application) is in some sense actually faster than immediate, even with a tradeoff of dampened freedom and creativity.^[6] The distinction here between application as a verb (to apply) and application as a noun (as in a computer program or smartphone “app”) is, I think, an interesting one. But this is exactly where AR serves as prosthesis—the Makeup Genius app fills an otherwise invisible cognitive role for the user. When applying makeup, as when selecting from any consumer product, oftentimes the actual application of said colors comes after some sort of mental selection process, where one pictures various colors on themselves before beginning the process of putting them on. Once the makeup is fully realized it may of course be changed after seeing what it actually looks like, but the imagination is a crucial cognitive step that precedes the physical process. In tandem, the physical process often feeds back into the imaginative process. In terms of this application, the essential role that the “eye” of the smartphone camera plays consists primarily in its ability to serve as a literalization of the “mind’s eye.” It can “see” better than a human’s imagination can. Neither imaginative nor physical application is needed.

Makeup Genius is not the only AR application operating as an imagination prosthetic. The IKEA Catalog app lets users select and arrange digital furniture to be viewed within the context of their living room. LEGO stores have AR kiosks that digitally construct the contents of a set on top of its respective box. Converse’s The Sampler allows users to “try on” pairs of sneakers through their phone’s camera, and apps like Dulux and TapPaint let users “paint” the walls of their homes. While a phone application’s application of virtual paint might seem like nothing more than benign marketing gimmicks now, the rhetorical strategy of visually bypassing imagination and invention has clear ominous implications for an AR-saturated future. By constructing interactive narratives about products and using the lens of the various devices to make the consumer a part of the advertisement, these apps are building off of a customizable and interactive narrative tradition reminiscent of the customizable children’s storybooks popular in the 80’s. And just as those books were manufactured by various companies before being sent to their respective consumers, these prepackaged AR apps rely on a certain amount of identity appropriation in order to create their superficial narratives.

It is this particular quality of AR that makes it such a critical object of study. As digital technologies become ubiquitous components of our lived reality, it becomes increasingly critical to examine the ways that our physical experience and perception of reality are reoriented and articulated in nonverbal ways. ^[7] Of course this only applies to the privileged, industrialized world’s experience of tech. This aspect of the technology is certainly easy to harness and abuse by those in power, but the use of AR tech as a discursive medium also offers great potential as an agent of both democratic and poetic change. As Morey and Tinnell state, “the paradigm shift that augmented reality beckons is as much about everyday ethics as it is technological utopias,” (7) or as Alan B. Craig argues, “the biggest limitation right now is our imagination for the possibilities, combined with a lack of widely available, easy-to-use development tools” (Understanding Augmented Reality 265). While so far I have examined existing applications and tech, I will now examine a few of the ways AR as a writing technology has been imagined in popular culture, and the contemporary implications of its role as discursive medium. Through this analysis, I contend that AR can be useful as a metaphor for how ideology is constructed and remediated in contemporary culture, and the implications that has for a conception of posthuman writing and cognition.

Ideology Sunglasses, Posthuman Vision

In his early work on user interfaces, Andries van Dam argues, “The best interface is no interface—’I think, therefore the computer gives me what I thought about (and what I should think about)'” (64). While the general consensus here that the ideal interface should be invisible and intuitive, it is interesting to note the Cartesian undertones painted onto this claim. Van Dam echoes a long history of critics here: from Heidegger’s das Zeug, the equipment, tool, or “readiness-to-hand” whereby technology conceals itself through habitual use, to Mark Weiser’s opening claim in “The Computer for the 21st Century,” that “the most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it” (1). While it is difficult to argue that good interface design does tend to favor a naturalizing to the human body and sensory apparatus, van Dam’s claim should also take into account the other invisible underpinnings of digital technology, namely the effects that algorithmic processes have on our cognitive, psychological, and ideological formations. When it comes to both interface and apparatus, the true power of both the tech and medium of augmented reality is its ability to provide seamless, “interfaceless” digital-to-physical interaction through a readily apparent apparatus. Consider for example the yellow first down line in a football game, arguably the most-encountered pop culture use of AR—while it has the potential as a constant reminder of an authoritative figure getting final call, without a designated, interrogative apparatus, it has instead become normalized and made invisible through the mediating television screen.

Augmented reality technologies offer us new lenses through which we can view digital materiality by framing the AR apparatus as a literalization of what Slavoj Žižek calls ideology spectacles, where sunglasses can be read as a metaphor for both the physical AR apparatus and its affective properties. In fiction, these glasses often provide a mode of AR that exposes hidden truths or enlightens users to invisible and insidious ideologies, whereas a much more likely permutation of AR seems to be one that remediates our physiological reaction to (or experience of) the physical world. This does not mean, however, that these fictional analogies are not worth exploring, just that they must be explored with caution. For example, in The Wonderful Wizard of Oz, the inhabitants of the Emerald City are indoctrinated into a collective reality through the wearing of emerald-tinted glasses that force them to see the physical city in a way that is “untrue.” Alternatively, perhaps the perfect chiasmus to Oz is John Carpenter’s 1988 cult classic film They Live, where protagonist John Nada discovers a pair of sunglasses that give him the power to see subliminal messages displayed throughout the cityscape. When wearing the sunglasses, a billboard featuring a swimsuit model instead bears the message “MARRY AND REPRODUCE” and the money in Nada’s hand simply says “THIS IS YOUR GOD.” Eventually he discovers what Žižek refers to in the introduction to The Pervert’s Guide to Ideology as a “classic Hollywood topic of the invasion of the body-snatchers,” where aliens have hidden among society and are controlling it by enforcing consumer culture. Unlike in the Emerald City, where the city is changed at the mediating point of the physical apparatus of the sunglasses, the city of Los Angeles in They Live is unchanged—what the glasses display is reality, or at least reality as it existed prior to the aliens’ arrival. The body-snatchers and their brainwashing scheme are real, but completely invisible unless subscribed to the correct channel. By using his “de-actualizing” glasses, Nada is able to find the source of the aliens’ signal and destroy it, making the aliens’ fictional reality disappear for everyone without the glasses. As Žižek explains:

There is a further feature which makes this scene with “ideologico-critical spectacles” contemporary: in it, the ideological injunction is hidden, so that it can only be directly seen through the glasses. Such a relationship between visible and invisible is predominant in contemporary “consumerist” societies, in which we, subjects, are no longer interpellated [sic] on behalf of some big ideological identity, but directly as subjects of pleasures, so that the implied ideological identity is invisible (Denial: The Liberal Utopia).

These “ideology sunglasses” can be a useful framework for thinking about the rise of digitally altered spaces, particularly through the use of augmented reality interfaces and technologies; it’s easy to see the parallels—virtual, or otherwise invisible, information being overlaid atop physical locations. Unfortunately, in many cases of augmented reality, the “sunglasses” of our digital devices work in almost exactly the opposite way. For starters, the use of the device is not hidden by the marketing company, but is rather encouraged. In this way, more often than not, augmented reality applications are used to reinforce the typical ideologies of consumer culture. The body-snatchers hide in plain sight, and they freely give their sunglasses—their AR apps—to anyone willing to play along, whether conscious of it or not, in the spectacle of their mixed reality ad campaign. The bigger the company backing the project, the flashier and more attractive the reality they are able to offer, and thus the more effectively they are able to attract the attention of the consumer. Through his analysis of Prada’s conscious efforts to produce such an AR campaign, Manovich explains that we are reaching a point where “[all physical spaces] now have to compete.” (235)

This competition of public space already manifests itself through augmented mediums, with some noncommercial applications being developed to compete with corporate uses. One of the more recent examples is the augmented reality goggles called Brand Killer, developed by a group of students at the University of Pennsylvania’s PennApps hackathon. Brand Killer, described as “AdBlock for Real Life,” is a headset that allows for brand name logos to be blurred out or pixilated from the viewer’s perspective in real time, allowing an “opt out” option for everyday life (Vanhemert). A similar example is artist Mark Skwarek’s logo-hacking application called “The Leak in Your Hometown.” This application augments the oil and gas company BP’s logo to depict a broken gas pipe leaking unending gasoline upward in reference to the 2010 BP Deepwater Horizon oil spill. Similar to the glasses in They Live, this application literalizes an invisible truth and turns it into something that must be confronted. While these examples are by no means the most exciting or impressive, they are still emblematic of a deep desire to take back public space as a place of democracy and discourse.

This method of taking back public space through the integration of digital data, as well as the collapse of the imagination into our devices, is made possible from the collapse of the traditional polarity of digital and physical domains. As Steven Jones states throughout The Emergence of the Digital Humanities, the digital is, and always has been, physical. Digital capable devices allow us to access the network, but that network exists in physical locations as data that can be rewritten and manipulated within servers and other hardware. Jones argues that now “the network is no longer normally imagined as a place you jack into in order to upload your disembodied consciousness, a place you ‘visit’ as if it were another planet. It’s right here all around us, the water in which we swim” (20). For Jones, digital information “everts;” it integrates itself in the physical environment outside of the bounds of traditional computational interfaces such as the desktop. He argues that the common metaphor surrounding the “digital realm” comes to us from speculative fiction, particularly William Gibson’s Neuromancer, which features virtual environments that participants enter into, traditionally identified as “cyberspace,” termed by Gibson in his short story “Burning Chrome.” Eversion, on the other hand, appears in Gibson’s more recent novel Spook Country and represents the current scenario where the digital is stepping out into the everyday physical environment. Digital information is making itself more apparent as a physical entity, or as Jones puts it, “the augmented has displaced the virtual” (14).

In addition to data, the systems that support data storage and exchange have also become more integrated with, and impactful upon, external environments. George Yúdice argues for problematizing the “Cloud” metaphor of the Internet, the often-advertised model of an ethereal, ever-present and non-physical system. In reality cloud storage facilities can be the size of small cities, and in a time where California is struggling through historically severe droughts, cloud facilities in Silicon Valley use enough water in their cooling systems to provide for several small towns (“All the World’s a Cloud”). This “grounding” of the cloud is a shift away from what Nathan Jurgenson calls “digital dualism,” a belief “that the digital world is ‘virtual’ and the physical world ‘real.'” Jurgenson goes on to say that he thinks “digital dualism is a fallacy,” a statement that spawned a string of contentious dialogue between him and technology writer Nicholas Carr (“Digital Dualism vs. Augmented Reality”). ^[8]This back-and-forth between Jurgenson and Carr does a surprisingly good job of summing up what seems to be the two primary camps of thought when it comes to discussions of the “digital … Continue reading

The argument against digital dualism is perhaps best fleshed out as what N. Katherine Hayles and many “thing theory” practitioners might refer to as the coevolutionary spiral of humans and technology, or an object-oriented ontology. In her book How We Think: Digital Media and Contemporary Technogenesis, Hayles gives a useful example of this spiral through her analysis of Toc: A New Media Novel: “humans construct time through measuring devices, but these measuring devices also construct humans through their regulation of temporal processes,” which of course easily extends to AR and any other tech or tool (115). Hayles further argues for thinking as a process undergone by human and non-human actors, where humans are more like units in an evolutionary progression than they are fully autonomous agents demonstrating mastery over nature. When nature is so often lazily defaulted to as an Other in this way, it is easy to understand the dominance of a digital dualism ideology. Consider how Carr handles his initial rebuttal against Jurgenson:

We should celebrate the fact that nature and wilderness have continued to exist, in our minds and in actuality, even as they have been overrun by technology and society. There’s no reason to believe that grappling with the online and the offline, and their effects on lived experience and the formation of the self, won’t also produce important thinking and art. (“Digital Dualism Denialism”)

When Carr brings up the “self” throughout this discussion, he aligns himself with an outdated Cartesian model of the human, one that Walter Ong and Eric Havelock have shown has its roots in alphabetic and print literacy. Carr demonstrates how directly this thinking constructs a view of nature that is at once exotic and remote, as if human involvement would irrevocably ruin it, or that it should not be “ruined” in the first place. Nature of course relates to natural, but the implication Carr is making is that there is nothing natural about the human or technology, and certainly not about the digital. So while the beaver dam counts as natural, the iPhone does not, despite both being tools that were created by (and that in turn help create) agents in evolutionary discourse. Hayles and others suggest something more fluid, where thinking is more like a multitude of agents getting better at reacting to one another, and less like some divine, exclusive gift reserved for humans.

The construction of tools—just like the construction of a dam, iPhone, or anything that augments “nature” or environment by its presence and application—needs to be considered a part of how we approach augmented reality. From a tool-building, digital humanities perspective, it seems that it is no longer enough to ask questions of culture through preconfigured tools designed for industry or military use, but rather a time for building tools that address the needs of humanistic inquiry. At the same time, such tools must also be put through cultural critique, as Liu has argued. Or in other words, perhaps it is now the duty of those in the humanities to become practitioners of a reactionary discourse. As Hayles argues (though Hutchins) in How We Became Posthuman: Virtual Bodies in Cyberspace, “modern humans are capable of more sophisticated cognition than cavemen not because moderns are smarter…but because they have constructed smarter environments in which to work” (289). In Hayles’ How We Think coevolutionary spiral, it is this very act of mediating sensory experience through augmentation that is the most important tool at our disposal in creating these “smarter environments in which to work.” In this sense AR might be compared to the “book wheel” used by medieval scholars, which was in itself a cognitive prosthetic for quickly and efficiently working with multiple texts at once. The true power then of AR, both the literal tech and medium as well as what it represents in the digital-as-physical conversation, is not only in its potential as a literalized form of cognizance of this coevolutionary spiral, but also as a process of destabilization. What it demands is a reconfiguration in the way we think about AR as both physical and cognitive apparatus—if imagination ever really did take place within the mind, now these processes of creativity are beginning to be done for us; they have become everted and externalized. To think about these questions means not only interrogating the technology, but also using the technology as an interrogative device.

Towards an Electrate Camera

The sunglasses metaphor works to demonstrate the AR apparatus/prosthetic as electrate cultural interface, but also mimics Walter Benjamin’s work on the effects of the camera as a technocultural interface. Much like the ideology glasses, the camera works to expose naturalized or otherwise hidden elements of reality:

Evidently a different nature opens itself to the camera than opens to the naked eye – if only because an unconsciously penetrated space is substituted for a space consciously explored by man. Even if one has a general knowledge of the way people walk, one knows nothing of a person’s posture during the fractional second of a stride. The act of reaching for a lighter or a spoon is familiar routine, yet we hardly know what really goes on between hand and metal, not to mention how this fluctuates with our moods. Here the camera intervenes with the resources of its lowerings and liftings, its interruptions and isolations, it extensions and accelerations, its enlargements and reductions. The camera introduces us to unconscious optics as does psychoanalysis to unconscious impulses.

In her analysis of how Siegfried Kracauer and Benjamin relate the camera to cultural memory, Alison Landsberg remarks, “Benjamin sees the camera as a privileged device for making visible what, through repression, remains invisible to the naked eye” (17).

If the camera is a psychoanalytic and interrogative tool for mass culture, then the augmented reality apparatus would the be equivalent for the internet and the society of the spectacle—the electrate camera for a mass network culture. In the introduction to Ecosee, Sean Morey warns of the dangers inherent in visual technologies through his ecocritical work, arguing that electracy and the ecosee point to a cultural and communicative shift toward image culture. Morey argues, “those that control what the images ‘say,’ the pictorial manipulators that give picture/speech acts their illocutionary force and perlocutionary effects, are hegemonic structures that are able to determine what gets shown” (41). This threat that hegemonic image distribution poses to a cultural subconscious is an obvious likelihood for AR, as the creators of expensive tech so often tend to consist of corporations and those with strictly monetary agendas. As such, it is necessary to begin formulating a critical methodology for approaching mixed reality mediums, especially now that the first truly mainstream use of augmented reality has been realized with the smartphone application Snapchat.

Snapchat’s AR “lenses” became a cultural phenomenon almost overnight after the company covertly acquired the facial recognition “selfie” app Looksery for $150 million in September 2015. Snapchat is perhaps the builder of AR most relevant for critique due to the technological and political implications of the company’s enormous mainstream success and global reach, as well as for how it allows us to recognize how AR serves as an ideological construct.

Independent of Snapchat’s new AR experience, it is important to note that the app is in many ways an outlier when it comes to social media. While similar tech juggernauts like Instagram and Facebook serve as endless digital archives for seemingly infinite amounts of permanent photo storage, Snapchat promises services that do the exact opposite. Instead of posting photos to a page or “roll,” users set a timer for their photo or video, select their recipient(s), and send it away. Once a recipient views the sent photo or video for the predetermined amount of time, it instantly deletes itself and cannot be viewed again.^[9]See Orality and Literacy: The Technologizing of the Word and Preface to Plato, respectively. Similar to the ways in which Instagram manufactures nostalgia with filters that mimic the aged discolorations of Polaroid and instant film, and the vignette-causing light leaks of the cheap consumer Holga camera, Snapchat offers its users artificial ephemera by rebuking the powerful tools of networked archivization that their peers in the tech industry so often rely on. Just as a digital augment disappears when the viewer looks away from an AR installation, so too does an image on Snapchat.

On September 16, 2015, Snapchat began offering visceral mediated experiences to its users with the introduction of augmented reality features in what they call “Lenses.” These lenses work in a similar vein as Makeup Genius, where a smartphone’s forward-facing camera allows the screen to mimic a small mirror, which in turn allows the user to cycle through various augmentations that are mapped onto the face in real time. Users can distort their faces with a never-ending deluge of new lenses (which update everyday), and add bodily effects, like making their foreheads comically large, morphing their face into a monster, making themselves cry with cartoon tears, and even vomiting rainbows. “Premium Lenses” can be purchased for $0.99, and, much like the corporatization of augmented space that occurs in Tim Maughan’s cyberpunk short story “Havana Augmented,” businesses are even encouraged to sponsor lenses, for between $400,000 and $750,000 a day (Kosoff). Among the first sponsors were Fox Studios, advertising The Peanuts Movie with interactive dancing Snoopy and Woodstock 3D characters, and McDonalds, which is also one of the actual sponsors Maughan dystopically portrays in “Havana Augmented.”

It was once popular for users to wield the application’s crude color palate with a finger and paint onto their environment and even themselves, adding drawings to their “selfies” or cartoony additions to their surroundings, but with the introduction of “Lenses,” these interactive, creative processes are now predetermined through Snapchat’s predesigned AR overlays. And in a move similar to that of the nefarious Sakura corporation in “Havana Augmented,” the company has also implemented “geofilters” and “snapcodes,” features that allow specific AR content to be unlocked only within specific physical locations, such as overlays that show that a user is in San Francisco, or sitting inside of a McDonald’s. Snapchat’s intermixing of digital content with physical space mirrors what Jones refers to several times throughout Emergence of the Digital Humanities as a “willingness to engage,” where AR technology uncovers a deep desire to engage with digital content in physical ways. So what makes Snapchat’s playful entry into this game of desire for a digital-physical interaction so important? Even though it is the first mainstream contender into a game of this kind of interaction, it is crucial to remember that what Snapchat offers is also a corporatization of this AR remediation process. While other AR smartphone apps such as Layar and Aurasma^[10]Aurasma is now called HP Reveal. offer users the ability to augment physical space with layers of their own design, what Snapchat offers is prepackaged AR that can only be consumed, never remediated. .^[11] Or as Aurasma/HP Reveal calls them, “auras.” Much like a McDonald’s hamburger, what users do create with this tool is cheap and quick content that can only be sent and consumed once, so that new content must be constantly prepared and served anew. And while the service is marketed as though the images and videos are deleted immediately after being viewed, in truth these photos remain, haunting Snapchat’s servers long after their short life on recipients’ phones. The Snapchat ghost holds significance when considering what AR really is—a medium of creating and controlling apparitions that only select few people are able to see.

As I have argued already, many of the primary workings of AR are processes of literalization and eversion, and so it is interesting to note that Snapchat began the development of its mixed reality functionings by purchasing Vergence Labs, a company working a pair of networked sunglasses (Wong). While those sunglasses had more in common with the non-AR peripheral Google Glass, the sudden jump from software to hardware does seem to follow the logics of eversion and digital materiality, and hints at a move towards a more streamlined digital-physical apparatus.^[12]While Google Glass might seem like an early iteration of AR, the digital contents of its screen never actually interacted with its surrounding non-digital, “physical” environments. I would … Continue reading In “SnapGlass? HoloChat? Snapchat is Secretly Hiring Wearable Technology Experts,” Sean Hollister speculates about a different kind of cognitive augmentation, arguing, “as much as Google Glass was demonized, one of its most useful features was a camera that was always ready for action and captured exactly what the wearer saw. A pair of Snapchat glasses could do away with the extra steps involved in snapping a shot, and that could encourage people to use the service even more than they do now.” It is not uncommon to stare at something important for an extended period of time to create a more vivid memory of it, and it’s becoming increasingly common for people to take a photo instead. But what Hollister suggests is that Snapchat could be working to merge the two, to create a memory or communicative prosthetic, hardware that augments human cognition. In Hayles’ caveman example, these glasses become part of a “smarter environment” that helps negotiate the development of more object-oriented, posthuman modes of memory and discourse.

If augmented and other mixed reality mediums were once all about large headsets and cumbersome hardware, now they are being covertly slipped into our phones. While current uses of AR lean heavily towards industry, military, pragmatic, or commercialized uses, a deconstruction of AR’s logic and machinations reveals the ways in which our tech experience is streamlined and concealed, as its performativity so often relies on the functionality of a visible/invisible dichotomy. What makes the AR medium/tech so ripe for techno-critique and critical examination is the fact that it operates so uniquely close to human cognition. AR operates as a “natural” and invisible tech; by its very nature, it could be considered one of the “most human” pieces of digital tech as it is so often concerned with practical solutions to problems and interactive, digital play. But as far as the medium relates to ways of extracting processes of imagination and critical tool building, it is the responsibility of the humanities to divert a techno-future away from the “sealed,” authoritarian use of the tech depicted in dystopic sci-fi like Black Mirror, and towards the hacktivist, open source use of AR speculated in works like “Havana Augmented.”^[13] Particularly the episode “White Christmas.” I have been using the term “prosthetic,” but through a more posthuman lens it is not difficult to imagine this type of digital tech, this AR poetics—the entire spectrum of AR—being harnessed into a more naturalized understanding of techno-human materiality.

Consider John Tinnell’s useful critique of the term “augmented reality.” By applying examinations of the ways vision relates to “reality” in psychology, Tinnell contends,

We never experience a reality that is somehow prior to or purged of images. In this sense, our experiences of “reality” have always already been “augmented” by imaging. One could say that imaging, like writing (if we even distinguish between them), is an originary supplement to perception. Thus conceived, the term “augmented reality” becomes a senseless, empty signifier. That world browsers and smartphones bring this capacity into the mass-market, leading to the creation of massive global networks, is precisely what denaturalizes the philosophical stance implied by labeling such technology as augmented reality (All the World’s a Link).

The true power of the sunglasses then becomes not the ability to make the invisible visible, or to uncover masked ideologies, but to see tools of meaning making, tooled discursive practices, as part of the fluidity of posthuman being. That is, as Tinnell observes, AR is just another writing technology through which we augment the world around us, albeit a new one that allows for cogent new avenues of discourse. When AR becomes a prosthetic for a cognitive process we never thought we had, it reveals an ontology where cognition and reality have always been one and the same.

Works Cited

Baum, L. Frank. The Wonderful Wizard of Oz. Waiheke Island: Floating Press, 2009; 1900. Print.

Benjamin, Walter. “The Work of Art in the Age of Mechanical Reproduction.” In Illuminations, trans. Harry Zohn, New York: Schocken Books, 1968.

Bolter, Jay David, and Engberg, Maria. “Cultural Expression in Augmented and Mixed Reality.” Convergence: The International Journal of Research into New Media Technologies, Vol. 20(1). 2014. Web. 6 October 2018.

Borsuk, Amaranth. Between Page and Screen. SpringGun Press, 2016. Print.
Carr, Nicholas. “Digital Dualism Denialism.” Rough Type. Rough Type, 27 Feb. 2013. Web. 30 Sept. 2015.

Craig, Alan B., and Inc. Books24x7. Understanding Augmented Reality: Concepts and Applications. Amsterdam: Morgan Kaufmann, 2012. Print.

Dam, Andries van. “Post-WIMP User Interfaces: The Human Connection.” Communications of the ACM. Vol. 40, no. 2, 1997. Web. 28 Nov. 2016.

de Souza e Silva, Adriana. “Mobile Narratives: Reading and Writing Urban Space with Location-Based Technologies.” Comparative Textual Media: Transforming Humanities in the Postprint Era, edited by N. Katherine Hayles and Jessica Pressman. Minneapolis: University of Minnesota Press, 2013. Print.

Dobrin, Sid, and Sean Morey, eds. Ecosee: Image, Rhetoric, Nature. New York: SUNY Press, 2009. Print.

Debord, Guy. The Society of the Spectacle. New York: Zone Books, 1994. Print.

Deleuze, Gilles, and Félix Guattari. 1977. A Thousand Plateaus: Capitalism and Schizophrenia. Trans. Brian Massumi. Minneapolis: University of Minnesota Press.

Geroimenko, Vladimir, ed. Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium. Switzerland: Springer International Publishing. 2014. Print.

Gibson, William. “Burning Chrome.” The Oxford Book of Science Fiction Stories. Ed. T. A. Shippey. New York: Oxford University Press, 1992. Print.

—. Spook Country. New York: G.P. Putnam’s Sons, 2007. Print.

Gold, Matthew K., ed. Debates in the Digital Humanities. Minneapolis: University of Minnesota Press, 2012. Print.

Havelock, Eric Alfred. Preface to Plato. Cambridge, MA: The Belknap Press of Harvard University Press,1962. Print.

Hayles, N. Katherine. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago and London: University of Chicago Press, 1999. Print.

—. How We Think: Digital Media and Contemporary Technogenesis. Chicago: University of Chicago Press, 2012. Print.

Heidegger, Martin. Being and Time. Translated by John Macquarrie and Edward Robinson. New York: Harper and Row, 1962.

Hollister, Sean. “SnapGlass? HoloChat? Snapchat is Secretly Hiring Wearable Technology Experts.” CNET. CNET, 11 March 2016. Web. 18 March 2016.

Jones, Steven E. The Emergence of the Digital Humanities. New York, Routledge, 2014. Print.

Jurgenson, Nathan. “Digital Dualism versus. Augmented Reality.” Cyborgology. The Society Pages, 24 Feb. 2011. Web. 30 Sept. 2015.

—. “Responding to Carr’s Digital Dualism.” Cyborgology. The Society Pages, 1 May. 2013. Web. 30 Sept. 2015.

—. “The IRL Fetish.” The New Inquiry. The New Inquiry, 28 June 2012. Web. 30 Sept. 2015.

Kosoff, Maya. “Snapchat is on track to generate $100 million in revenue.” Business Insider. Business Insider, Inc. 1 Oct. 2015. Web. 2 Feb. 2016.

L’Oréal USA. “Makeup Genius.” Smartphone Application. Apple App Store. Vers. 2.2. 2014. Web. 15 July 2015.

Landsberg, Alison. Prosthetic Memory: The Transformation of American Remembrance in the Age of Mass Culture. Columbia University Press, 2004. Print.

Maughan, Tim. Paintwork. Bristol, England: Tim Maughan, 2011. Print.

Manovich, Lev. “The Poetics of Augmented Space.” Visual Communication. Vol. 5. SAGE Publications, 2006. Print.

McLuhan, Marshall. Understanding Media: The Extensions of Man. New York: McGraw-Hill, 1964. Print.

Molla, Rani. “Smartphones are driving all growth in web traffic.” Recode. Vox Media, Inc., 11 Sept. 2017. Web. 1 April 2018.

Morey, Sean. “A Rhetorical Look at Ecosee.” Ecosee: Image, Rhetoric, Nature, edited by Sidney I. Dobrin and Sean Morey. Albany: State University of New York Press, 2009. Print.

Morey, Sean and John Tinnell, ed. Augmented Reality: Innovative Perspectives Across Art, Industry, and Academia. Anderson, SC: Parlor Press, 2017. Print.

Ong, Walter J. Orality and Literacy: The Technologizing of the Word. New York: Routledge, 1982. Print.

The Pervert’s Guide to Ideology. Dir. Sophie Fiennes. Feat. Slavoj Žižek. P Guide Productions, 2012. Film.

Postman, Neil. Amusing Ourselves to Death: Public Discourse in the Age of Show Business. New York, N.Y., U.S.A: Penguin Books, 1986. Web.

Rosenbluth, Reed, Jonathan Dubin, Tom Cattalo, and Alex Crits-Christoph. Brand Killer: AdBlock for Real Life. Software. Devpost. PennApps Winter 2015, 2015. Web. 15 July 2015. .

Skwarek, Mark, and Joseph Hocking. “The Leak in Your Home Town.” Smartphone Application. 2010. Web. 15 July 2015.

They Live. Dir. John Carpenter. Universal Pictures, 1988. Film.

Tinnell, John. “All the World’s a Link: The Global Theater of Mobile World Browsers.” Enculturation: A Journal of Rhetoric, Writing, and Culture. 14 December 2011. Web. 2 March 2016.

—. “Computing en plein air: Augmented reality and impressionist aesthetics.” Convergence 20.1. SAGE Publications. 20 December 2013. Web. 2 March 2016.

Ulmer, Gregory L. Avatar Emergency. Anderson, S.C.: Parlor Press, 2012. Print.

—. Electronic Monuments. Minneapolis: University of Minnesota Press, 2005. Print.

—. Internet Invention: From Literacy to Electracy. New York: Longman, 2003. Print.

Weiser, Mark. “The Computer for the 21st Century.” Scientific American. Web. 28 November 2016.

Wills, David. Prosthesis. Stanford University Press, 1995. Print.

“White Christmas.” Black Mirror. Zeppotron. Endemol UK. 16 December 2014. Television.

Wolfe, Cary. What is Posthumanism? Minneapolis: University of Minnesota Press, 2009. Print.

Wong, Raymond. “Report: Snapchat is assembling a super team to build smart glasses.” Mashable. Mashable Inc. 11 March 2016. Web. 18 March 2016.

Yúdice, George. “All the World’s a Cloud.” Born Digital: Reformatting the Humanities in the 21st Century, University of Miami. Stanley and Jewell Glasgow Lecture Hall, Miami, FL. 20 March 2015. Keynote Address.

Žižek, Slavoj. “Denial: The Liberal Utopia.” Lacan. Lacan Dot Com, 2013. Web. 22 Sept. 2015.

---

Biography

Jason Crider is a Graduate Research Fellow in the Department of English at the University of Florida, where he researches and teaches at the intersection of rhetoric, ecocriticism, and digital media studies. His recent work on augmented reality and haptic media appears or will soon appear, in ISLE, Kairos, and Textshop Experiments. He is the managing editor of Trace: A Journal of Writing, Media, and Ecology.

© 2018 Jason Crider, used by permission

---

Technoculture Volume 8 (2018)