Published: Digital Da Vinci: Computers in the Arts and Sciences. Springer, New York, USA, pp. 99-111. ISBN 978-1-4939-0964-3 (2014)
1. Introduction
The computer, while not a new concept, has in its modern form transformed the way we disseminate ideas, interact with one another and enhanced our capacity to acquire information. From the artist’s perspective digital imaging presents opportunities for visual invention and challenges in how visual form is mediated. In my artwork I use the computer as a means to create imagery that would not be possible without its use, and that enables exploration of an artificially created space that enhances spatial awareness and challenges our perception of what we encounter. The computer enables the use of real time and recorded moving and still images to be embedded within artwork previously limited to still imagery and makes possible the fragmentation and reconstruction of the picture plane into multiple moving images with a remarkably high degree of resolution.
So why would being concerned with the way an image is presented be significant and how might considering this question help us to advance pictorial invention in a digital age? How we encounter the world is essentially a very personal act and we do this in a way that is both knowing (based on prior experience) and questioning (open to new experience). In choosing to make an artwork I am seeking to explore both of these perspectives and in doing so one of the big challenges is how to make an illusion of space appear real and to find a way to make spatially comprehendible what is in essence an invention. To achieve this it is possible to be creative with spatial arrangement in an artwork that, while clearly defined, is also open to interpretation resulting in illusions of space that are intriguing for the viewer, enhance engagement and challenge spatial perception.
There is a different sensibility to image generation on a computer compared with the articulation of visual ideas made by hand and a graphic medium. Our impulse to make marks is evident in the long history of image making dating back to the earliest pictographs and petroglyphs to be found drawn and carved onto natural surfaces using little more than pigment and rock. Programs for drawing digitally onto a screen or tablet have been around for a while but the computer screen offers a singular format for creating and presenting these images. While our modern mathematically proportioned paper formats do in some part offer a parallel, the range of format and adaptability available to the artist using different media to create their work is substantial when compared to the commercially available screen in its present form, particularly when wishing to create images not contained within a standardised shaped surface.
When Brunelleschi stepped back into the door whose frame outlined the view of the building he wished to represent, the fixing of the edge of the observable space defined it as a picture plane onto the three-dimensional space beyond, marking a moment where the framing element established a means to create a spatially measurable illusion of reality (1). However, it also defined a boundary between the real world of the viewer and that of the image and ensures our perception of what is depicted remains consciously an illusion.
We take this invention for granted today and are familiar with looking at artworks that represent to us a likeness of the world we are familiar with in a way that is spatially convincing, allowing us to engage with the illusion of space as an extension of our real world reality.
The creation of an illusion of space that can be interacted with on a human scale can be seen where the properties of art, architecture and effects of light combine. Historical examples of how we invent pictorially to create the illusion of reality with light, surface and material in an immersive way are visible in Japanese fusumaē (screen paintings), Neo-Assyrian wall reliefs, Roman mosaics and Renaissance frescoes, to name a few (2).
Artists have also embraced lens based technologies to enhance both their perception and understanding of what they see and to work with creatively in constructing their images (3).
There is now considerable interest within the neuroscience community in researching how the brain works when perceiving visual stimuli (4, 5). Visual forms that play with spatial ambiguity may offer insights into the way the brain functions, but artists have known for a long time that we are intrigued by images that appear spatially irresolvable and I am interested in extending this in ways that also play with our preconceptions of how imagery is presented.
While the picture plane may be viewed as a window onto an illusion of space, it is also the surface upon which spatial reality may be rebuilt. In questioning what we observe we must inevitably have recourse to our experience of the physical world around us and the pictorial offers us a way to explore these experiences and reinvent the world we see. Illusion of depth may be conceived as being built upon more than one plane or in such a way that a surface may appear to have many layers of reading. Inventing with the visual topology of the surface, while retaining its flat physical characteristic, has enabled me to create images which alter in their illusions of three-dimensional spatial readings when viewed over time or with varying light conditions and projections.
2. Perception and Visual Experimentation: Incorporating Computer Technology in Recent Interactive Artworks.
In seeking to invent spatially beyond a conventional pictorial format I have been developing the integration of digital content in artworks incorporating interactive Retro-reflective Projection Technology and real time/time delayed video installations.
Figure 1: Nathan Cohen 'Intangible Spaces', RPT Installation, Aisho Miura Gallery Tokyo 2010; computer screen showing format of video for wall projection
I have found that Retro-reflective Projection Technology (RPT) can be used to create visual images on a larger scale with embedded digital projections. By constructing the artwork in particular ways it is also possible to integrate digital content so that it does not appear to be constrained by the fixed framing format of a screen edge.
Tachi Lab (a leading laboratory researching Virtual Reality and Telexistence directed by Professor Susumu Tachi, now located in the Graduate School of Media Design, Keio University, Tokyo, Japan) has developed technology to support research into ‘optical camouflage’ and ‘augmented reality’ (6, 7) which I am working with in the development of new visual forms that can also be applied at an architectural scale. This has resulted in the creation of large scale publicly sited artwork installations, displayed at Ars Electronica, Linz, Austria (2008), and at the Aisho Miura Gallery in Tokyo, Japan (2010), with ongoing development (see Figure 1).
In creating my artwork I am looking to find a dynamic balance between the real world we live in and the world that may exist within an image and it is in seeking to find ways to link the two that a dialogue between the imagination and reality takes place. The issue of how the brain determines the difference between an object seen and its reflection or likeness is one which has preoccupied those who have studied the nature of visual perception from the neurobiological perspective (6). To find a form of visual expression that relates to the way we see encompassing a panoramic view and at a scale which encourages a more immersive and interactive experience is an exciting proposition. When combined with the possibility of incorporating moving imagery inventive potential is further developed. Enabling the image seen to be interacted with directly by the viewer, even locating their presence within the image itself in real time, confers a further layer of reading and interpretation and this is a process that has been achieved with the use of computers.
3. Interactive wall installation.
The interactive artwork presented at Ars Electronica incorporated three layers of reading with two layers of projection. Initially, the artwork may be viewed in its own right and is composed of irregular four-sided elements made from black and white card and a dark grey and light grey retro-reflective material, all fixed directly onto the wall surface. In normal ambient light conditions this gives the appearance of an image constructed from a specific combination of elements whose tonal values and composition create the illusion of a three-dimensional structure projecting from the wall surface, although it is flat.
When looking through the projector that faces the artwork at a 90 degree angle the viewer initially sees a static structured grid corresponding to the artworks construction, a first layer of projection that reveals more detail about its spatial composition and how this was conceived at the drawing stage.
A motion sensor attached to the front of the projector records with a slight time delay the silhouette of people walking in front of the artwork and this causes a second layer of video projection to be revealed beneath the first layer, in the area rendered visible defined by the edges of the moving silhouette. I had intended that this second layer of video projection would be a view captured in real time of the space behind the wall on which the artwork is installed, further questioning the relationship of the viewer to the seemingly solid surface of the wall on which the artwork is installed and revealing what exists in the space beyond. However, as the view behind the wall was not particularly engaging for the purpose of this exhibition it was necessary to use a video recording I had made earlier in Trafalgar Square, London.
The artwork spanned several meters across a wall and the projector was placed at sufficient distance to allow for the projected images to cover several of the retro-reflective elements composing it, and for observers to walk between the projector and the wall which also triggered the motion sensor.
The projection process has to be constructed with a computer programmed to drive the video sequences and modifying the projections in relation to the sensor input, although the viewer will not necessarily be aware of this as the computer is not visible as part of the display. The results are engaging and stimulated considerable response on the part of the viewers who interacted with the video projection and realized that in the viewing of the artwork they were active participants in the experiences of others who were also observing over time. This occurred in two ways; viewing through the projector as others walked up to and around the artwork, and being observed by others when approaching closer to the artwork.
The physical properties of the RPT reflective material offer a natural extension to the media familiar to the painter and allow for pictorial invention at a human and architectural scale. It can be cut into different shapes, applied directly to and bent around surfaces of different size and when viewed in ambient light conditions the light and dark grey of the RPT materials provide a tonal contrast that creates a balanced scale between black and white. Being highly light reflective when a more focused light is projected onto their surfaces the tonal scales shift, with the brightest element becoming the lightest of the grey RPT materials and the white non-RPT material surface seeming grey by comparison. This offers another means of playing with perception of space and depth within the artworks’ construction and can alter the illusion of the three dimensionality of the image given varying light conditions and projection.
In seeking to invent in my own work with images that appear three dimensional while retaining pictorial form (i.e. they are flat) and that have irregular boundaries I have created structures which are not confined to a picture window illusion of space. As the image also exists within the ‘real’ space of the viewer it becomes more architectonic in its spatial implications and activates a more dynamic relationship with the ‘real’ space within which it exists. When conceived on a human scale this allows for the creation of an immersive environment encouraging interaction with the artwork. When pictorial forms are combined in such a way as to challenge spatial interpretation a new dynamic unfolds resulting in images which are both intriguing and intellectually engaging. This can be developed overtly or subliminally, with the structure or physical construction of a work demanding visual dexterity in its comprehension while the light projected layering of imagery in selected elements offers multiple possibilities for spatial interpretation.
Figure 2: Nathan Cohen: art installation, Ars Electronica, Linz, Austria, 2008 (a) without projection
Figure 3: Nathan Cohen: art installation, Ars Electronica, Linz, Austria, 2008 (b) detail of projection layer 2
Consequently, I am investigating the potential for spatial invention offered by the digital projection of images, both still and moving, onto light reflecting material elements incorporated into my artworks. In their construction play is made between observing the artwork as a visual construction: (a) in its own right without projection; with certain areas exposed to projection, enhancing the three-dimensional reading of the space within the artwork (see Figure 2); (b) optionally a secondary layer of projection embedded ‘behind’ the first projection layer, revealed through the motion of viewers in front of the projector, all combining to offer extra dimensions to the viewing of the art work (artwork installation at Ars Electronica, Linz, 2008) (see Figure 3); and (c) the integration of real-time video projection (Aisho Miura Gallery, Tokyo, 2010: floor projection and real-time video wall projection) (see Figures 4, 5, and 6).
4. Intangible Spaces.
Tokyo provided the wider context for the exhibition ‘Intangible Spaces’ at the Aisho Miura gallery (2010). The city is a complex and contrasting environment where the new rubs shoulders with the old; buildings constructed using the latest technology can be seen alongside centuries old shrines and temples which are examples of the technological innovations of their day. In the art installations I created for this exhibition I wished to capture something of the essence of this experience, with the gallery itself once being a house used by the local temple, now converted into a modern gallery space. The imagery used in the recorded projections combined two views of scenes observed in Tokyo at the time of the show: for the wall projections I contrasted video sequences of Tokyo panoramas viewed by day from the top of a high building, enabling the complexity of the city to be viewed from an unusual perspective, with the summer hanabi fireworks displays, momentary bursts of structured light seen at night.
The artwork was constructed from irregularly shaped and alternating lighter and darker grey RPT material panels located at the corner of two walls resulting in an installation that appeared three dimensional and as if projecting into the viewer’s space, onto which the looped video sequence was continuously projected. RPT offers the potential for inventing with projection techniques on an architectural scale and my intention was for the distinctions between illusion and the reality of the viewer’s space to become blurred, with the artwork to appear to be a part of the real space experience of the viewer.
Figure 4: Nathan Cohen 'Intangible Spaces', RPT Installation, Aisho Miura Gallery Tokyo 2010
The design for the artwork was in part determined following initial discussions relating to the technical aspects of the RPT projectors and my decision to include real-time video in the process, which involved site visits to the gallery with the Tachi Lab team. The first practical stage for installing the artwork involved identifying the optimal projection locations within the gallery space, cutting the RPT material according to the designs and installing them on the floor and walls to allow for the projections to take place. Then the hardware, including 2 RPT projectors, video cameras, computers and cabling were installed. The final stage involved calibrating the equipment and preparing the video sequences I had shot earlier for the wall projection.
The RPT projectors had to be located at specific distances from the wall and floor and were oriented so that the viewer could easily navigate the space and view the artworks both with and without the projections. The floor piece incorporated a real-time video projection of the view as if seen directly through the floor to the room below, with the video camera located at an equivalent angle to the projector. This gave the viewer the impression they were seeing through the floor to the activity in the space below.
Figure 5: Nathan Cohen 'Intangible Spaces' (c) RPT floor projection, Aisho Miura Gallery, Tokyo, 2010
Both the floor and wall based RPT artworks required interaction on the part of the viewer to fully observe their pictorial potential, for while the installation included the wall and floor pieces that could be viewed as images that interacted with the architecture of the gallery space to create the illusion of three dimensional structures, when the projections were also seen through the RPT projectors an alternative reading of the space within the artworks was enabled.
This often resulted in viewers spending some time walking up to the artworks themselves, then back to viewing them through the projectors, changing their view to look again without projection, and so on, encouraging a high level of engagement in encountering and looking at the installation. The intention was also to challenge pictorial convention in the way the artworks were located in space, utilizing the corner of the room and floor, locations not usually associated with the display of pictorial images.
5. Encountering ourselves in the interpretation of an image.
We are very familiar with looking into an imaginary space, and developments in immersive technologies have sought to place the viewer in ever closer proximity to the action. A third artwork displayed in the Tokyo exhibition develops the idea of locating the viewer within the space of the artwork itself and utilizes an approach to visual invention which could only be achieved using the computer.
As the visitor to the exhibition climbed the stairs to the installation located on the second floor their progress was being recorded by a small video camera. A short time lapse of the resulting video image was combined with isolating the visitor’s image from the background and a moving video sequence I had made of the art installation itself. The result was a video projection, which the visitor could view as they reached the top of the stairs, of themselves ‘climbing into’ and located within the artwork they would not yet have seen. This formed the start of the interactive process of viewing the art installations.
Figure 6: Nathan Cohen 'Intangible Spaces' (c) still of real-time video wall projection, Aisho Miura Gallery, Tokyo, 2010
Keitaro Shimizu, who assisted with the installation projections, describes the computer software used to achieve this: ‘We could make this projection by use of a computer program utilizing ‘difference of background’, enabling differentiation between two pictures, allowing us in this case to determine which object is moving. Using this program we initially recorded the background image of the view of the stairwell with no motion which enabled us to compare the color of the real-time moving image, subsequently captured by video camera, with the color of the background image. Where a significant difference between the colors of two images occurs for the purposes of this display we can recognize it as motion, isolating the pixels whose colors have changed and combining this image with the background image. For this projection a slight a time delay of a few seconds was also incorporated enabling the viewer to see their progress up the stairs as they emerged into the gallery space art installation. After this the program resumed recording mode; if no motion was detected the projection replayed the last motion sequence until new motion was detected which then started a new cycle of recording and projection.’
Memory forms an important part in our interaction with the world around us. There are many ways in which this can be explored and artists have sought to create images and art forms that rely for their interpretation upon experiences we have gained prior to their viewing. I am interested in creating artworks that encourage the viewer to be an active participant in the experience of encountering what they see. In this artwork this relies on a technological application that combines a small time shift and its projection into the moment of engagement – the person climbing the stairs encounters themselves in this act as they arrive at the point of viewing the artwork for the first time, and this marks the initial moment of engagement with the exhibition.
The seemingly mundane process of walking up some stairs can be represented to the viewer in a form that is not conventional, but relies for its interpretation on understanding certain conventions relating to how we interpret what we see. The context for this plays a part in how we choose to interpret what is unfolding before us, and the viewing of the artwork in the specific space and context of the other artworks that are also displayed in this video sequence requires the viewer to engage with the totality of the experience to be able to comprehend what is seen. This is a process that unfolds over the time it takes to view all the artworks in the exhibition, and rewards a return to the initial video sequence as, having seen the exhibition, the viewer can now understand the context for the imagery they initially see themselves ascending into in the time delayed video sequence.
6. Back to the computer screen.
Up to this point I have been discussing the application of the computer in the context of projected imagery within an architectural space. The interactive experiences described in the artworks exhibited at Ars Electronica and Tokyo pose an interesting challenge in recording and documenting them as they are temporary installations. In considering this I have also been exploring the creative potential of the space offered by the computer screen and this has led to new artworks that engage the viewer dynamically in the act of viewing on a screen.
We are familiar with, and would normally expect, that an image fills a screen. We are also familiar with more than one image being presented on a screen simultaneously, most typically with the multiple screens replicating the rectilinear form of the screen itself – windows within windows. The interactive nature of computer technology combined with the limitation of the flat screen encourages us to layer our experiences, moving from one window to another often displayed simultaneously.
In considering how pictorial form can express different types of interaction with space it is necessary to look at the relationship of edge to surface, for the picture plane is often clearly defined by boundaries which separate it from the non-pictorial space of its surroundings, particularly on a more intimate scale.
It is also important to think about how we perceive moving images; the saccade sequences of moments in movement neurologically processed giving the sense that these are contiguous over time. Then there is also the question of focus and angle of vision to be considered – how close or distant we are from what we see, and how we are located in relation to an action or object of interest and how this might alter our understanding of it. Light conditions also vary our perception of space.
I have been experimenting with animated sequences of recorded video images that are combined and composed over time and are formally articulated on the computer screen in ways that shift how we perceive them spatially and inform our memory of the actions that occur.
Recent test pieces introduce fragmented and multi-faceted views of video sequences of human actions, such as walking through a space or cooking a meal. The shifting composition of these facets is intended to create a dynamic tension between the formal elements that compose an image and the edge of the space proscribed by the screen. With several views of the same activity displayed simultaneously within animated facets that shift over time it is possible to impart a sense of how it might feel to be engaging in the activity being viewed and to see this from multiple perspectives enhancing a sense of depth and movement within the picture plane of the screen and creating a more immediate presence of the activity. To this end I have found that by liberating the pictorial elements that compose this constantly changing sequence of images from the rectilinearly defined edges of the screen space it is possible to create a more dynamic sense of space, time and movement.
Figure 7: Images of cameras and computer set up
One example of the set up for generating these multiple images that compose these video animations consists of a group of video cameras placed in different locations along the path a person walks (see Figure 7). Each camera relays its input to a computer that has been programmed to generate a sequence of shaped facets that move and combine across the picture plane of the computer screen and which incorporates the different video camera images. This is an artwork that can be viewed in real time or as a recorded sequence, the former offering an interactive element that allows for the viewer also to become the subject of the recordings if they chose to walk through the cameras.
Figure 8: 4 screen captures of movement over time (each image is a view on the screen at a given point in the animated sequence) - sequence 1
In another artwork a similar approach to multiple cameras and viewing angles is employed with the intention of recording a sequence of events over a period of time (see Figures 8 and 9). Here I have composed an animated video sequence that conveys the experience of preparing and eating a meal. I have also extended the moving facets to meet the edges of the screen actively incorporating them into the spatial construction as each of the facets alter their position relative to one another in the unfolding sequence, rendering the screens edges a dynamic parts of the video imagery.
Different degrees of focus are also utilized between the cameras offering detailed and broader angles of sight. This is significant, for in our interaction with the environment we are constantly altering our point of view in order to negotiate and better comprehend what it is we are looking at and engaging with. (8) It is in the editing of these multiple strands of video in conjunction with the animated facets that the artwork takes form.
Figure 9: 4 screen captures of movement over time (each image is a view on the screen at a given point in the animated sequence) - sequence 2
As an artist I am seeking to interpret the nature of what I see through the artworks that I make and in the process encourage others to see the world afresh and perhaps in new and different ways. We are now living in an age where digital media offers for an artist another medium for visual invention, and in my utilization of computer articulated images in these artworks I am doing so with the purpose of enabling and enhancing perception. As Professor Susumu Tachi observes: ‘The word ‘art’, which currently refers mostly to fine art, also meant ‘technology’ in ancient Greece. In other words, art refers to both fine art and technology and it originally meant the act of creating something new.’
References:
- Kemp, M. 1990 ‘The Science of Art: Optical Themes in Western Art from Brunelleschi to Seurat’ Yale University Press
- Cohen, N. 2003 ‘Illuminating Thought: Imaging Reality’ http://www.nathancohen.co.uk/Nathan_Cohen_Web/writing/Nathan_Cohen/pages/page1.htm
- Steadman, P. 2001 ‘Vermeer’s Camera’ Oxford University Press
- Zeki, S. 1999 ‘Inner Vision’ Oxford University Press
- Purves, D. and Beau Lotto, R. 2003 ‘Why We See What We Do’ Sinauer Associates Inc.
- Tachi, S. 2010 ‘Telexistence’ World Scientific Publishing Company
- http://projects.tachilab.org/rpt/
- Henderson, L.D 2013 ‘The Fourth Dimension and Non-Euclidean Geometry in Modern Art’ Leonardo Book Series