Hank Dutt has played the viola in the Kronos Quartet, the world’s most famous avant-garde string quartet, for over forty years. His face is still gorgeous; his cheekbones sit as high as mountain ridges, casting shadows over the rest of his features, giving him an austere appearance even when he is not performing. But when he began to play Thursday night in Trevor Paglen’s Sight Machine, his eyes narrowed and his lower lip pouted out so far that his cheeks sucked right into his mouth, and he took on that alien expression of a classical musician completely focused. When the music’s tempo increased, he would scrunch in his shoulders and duck down, rocking back and forth. His upper lip curled over the wide ledge beneath it when his bow flew over the strings, wild.
Sight Machine’s premise is simple and strange. The Kronos Quartet performs on stage, and Paglen runs a series of computer vision algorithms on the live video of this performance; behind the musicians, he then projects a video that these algorithms generate. Paglen is an artist best known for his photography of spy satellites, military drones, and secret intelligence facilities – a wonderful retrospective of his work is currently at the Smithsonian American Art Museum through January 6, 2019 – and in the past couple of years, he has begun exploring the social ramifications of artificial intelligence.
However, videotaping a quartet may seem an odd way for him to visualize his concerns about the topic. After all, he is not running any algorithms on the music. But who moves as strangely as a string musician? The Quartet’s monstrous expressions and flailing arms become fodder for Paglen in a series of experiments that play with the ways that computers see.
At times, the results are stunningly beautiful. Early in the performance, the Quartet play “Tusen Tankar,” a Scandinavian folk song, sweet, slow, and mournful. Behind them, Paglen projects the results of an algorithm for detecting shapes and “regions of interest.” The images are abstract but recognizable. A few diamonds rocking back and forth must be Sunny Yang’s cello, and the thin boxes of her shoulders and her bow crisscross across the screen. It looks like the work of an idiosyncratic, immensely talented animator: a series of blue sketches designed to accompany a wonderful performance.
But this lovely harmony between Paglen’s projections and the Quartet’s music is rare in Sight Machine. More often, the images serve only as juxtaposition to the music being played onstage. The ugliest algorithm is known as Dense Optical Flow, and Paglen runs it on top of the Quartet’s faces while they play a Vietnam-era protest song. Dense Optical Flow algorithms have a simple purpose: they solve the difficult technical problem of following multiple different objects as they move over time. For instance, drones watching people from above sometimes use them so that they do not lose track of who is who.
Paglen’s visualization of this algorithm is visually very dull. Watching live video of violinist David Harrington contorting, little, green squiggles follow his bow, his nose, his shoulders across the screen. Then the video cuts out and we see only the squiggles— exactly as beautiful as a screensaver. What monotony, trying to see like this machine. Beyond the straightforward didactism of pairing drone tech with protest music, this presents one way a computer would watch the quartet perform: just follow which object is which as they move around. And it tries to get the audience to think about what it would be like to see that way. But why? This gets us to the central question of Sight Machine, a question Paglen asks with startling urgency: what does it mean to videotape a concert in the era of artificial intelligence?
This question is not incoherent technobabble. In his writings, Paglen has forcefully argued that the internet and computer vision should lead us to change the way that we look at images. The old vocabularies of art interpretation are no longer sufficient because “the overwhelming majority of images are now made by machines for other machines, with humans rarely in the loop.” Any account of what an image means must reckon with the fact that it is no longer exclusively humans that are doing the interpreting. Of course, people and computers approach images with radically different goals: computers use them not as a representation of an object, but as a tool for some operation.
Say someone uploads a photo to Facebook. What that photograph means includes not only how it will be understood by the other humans who then see it, but also what it will mean to the million algorithms that will pore it over. Paglen has repeatedly predicted that that image will soon influence the uploader’s credit score and her insurance rates, and a full interpretation of that image cannot neglect this fact. Way back in 2016, he noted on Twitter that a car insurance company had started to use AI on first-time drivers’ Facebook profiles to help guess if they will be a safe driver. Paglen’s comment: “This is just the beginning…” Pictures are the raw material of artificial intelligence’s decisions, and these decisions increasingly impact people’s day-to-day lives.
Artificial intelligence interprets images, and this is important because that interpretation is then used for something. However, for most of Sight Machine, Paglen remains laser-focused on depicting this process of interpretation, and temporarily puts aside how those interpretations are then used. This is also true of his recent photography, where the many uses of AI only creep in at the edges: the social ramifications of this machinery are only the menacing backdrop for his often goofy experiments with what it is like to see like a machine.
Indeed, if we focused too closely on his politically charged themes, we might overlook the playfulness and campy humor that run throughout his oeuvre. In a piece currently on display at the Smithsonian retrospective, he runs a facial recognition algorithm over a photo of Shoshone Falls. Viewers will recognize the resulting boxes from Facebook’s attempts to identify faces in order to “tag” people. Paglen’s picture of a roaring waterfall is really a puzzle, and it is fun to solve: step close to see how the wisps in this little square look kind of like a face. I look around the image for more wisps that seem to have eyes and that I would have put a box around if it was me, not a machine, doing the tagging. In the end, viewers approach “Shoshone Falls Hough Transform; Haar” with the single-mindedness of a child pointing at clouds, or of a computer. The significance of his lighthearted technique lies in this occlusion.
Sight Machine is full of these funny, little puzzles. My favorite was during “Zaghlala,” an energetic adaptation of a song by an underground Cairo deejay. The pace of the music meant that the Quartet were at their most animated, and behind them, Paglen lets the computer guess their gender, age, and emotions. As David Harrington contorts this way and that, what makes him, for a split second, very feminine? Why for a few moments does Sunny Yang look like an infant? The whole crowd laughed at that one. Of course, these silly algorithms can also be extremely useful to some institutions. In one essay, Paglen describes how some businesses “install cameras in malls and department stores” so that software can “track facial expressions to discern the mood and emotional state” of customers as they look at various products. Every day, the rigid categories of the machine – right now, violinist John Sherba is 48% happy and 12% confused – come to shape more and more of our world.
Towards the end of the performance, Paglen brings the political thrust of the piece a little more to the foreground. For a few seconds, instead of the Quartet, he shows us a drone’s view of a city, with all the tiny cars being tracked and numbered. The final song is the first section of “Different Trains,” a composition about the Holocaust and the relationship between technology and progress. As Kronos plays, the screen shows a short film, which moves through thousands of the images and videos that artificial intelligence uses to “learn” how to see. For instance, we see dozens and dozens of brief videos of people making the same series of exaggerated expressions. They are amusing at first, but then become unsettling. The song is almost nine minutes long, repetitive, and didactic. That this combination feels affecting at the end of Sight Machine, rather than boring and tendentious, is a testament to the success of Paglen and the Kronos Quartet. For an evening, at least, they begin to shift the way their audience approaches images and videos.
Sight Machine was performed October 25 at the Smithsonian American Art Museum. The Trevor Paglen exhibit, Sites Unseen, continues at the museum, located at 8th and F Streets and through January 6, 2019.
You must be logged in to post a comment.