Poiesis and Enchantment in Topological Matter

Sha Xin Wei

The M IT P ress

Cam bridge, Massachusetts

London, E n g l a n d
IJ
12
,M3
S� 3
ZO I �

© 20 1 3 Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or
mechanical means (including photocopying, recording, or information storage and retrieval)
without permission in writing from the publisher.

MIT Press books may be purchased at special quantity discounts for business or sales promotional
use. For information, please email speciaLsales@mitpress.mit.edu.

This book was set in Stone Sans and Stone Serif by Toppan Best-set Premedia Limited, Hong
Kong. Printed and bound in the United States of America.

Library of Congress Cataloging-in-Publication Data

Sha, Xin Wei.

Poiesis and enchantment in topological matter / Sha Xin Wei.
pages cm
Includes bibliographical references and index.
ISBN 9 78-0-262-01 9 5 1-4 (hardcover : alk. paper)
1. Art-Mathematics. 2. New media art. 3. Topology. I. Title.
N72.M3S53 20 1 3
701'. 5 1 --dc23
201 3005 784

10 9 8 7 6 5 4 3 2 1
Contents

Preface vii
Acknowledgments xi
Chapter Summary xv

1 Why This Book?

2 From Tech nologies of Representation to Technologies of Performance 19
3 Performance i n Responsive Environments, the Performative Event 63
4 Substrate 89
5 Ontogenesis 1 23
6 Topology, Manifolds, Dyna mical Systems, Measure, and Bundles 161
7 Practices: Apparatus a n d Atelier 203
8 Refrain 249
Epilogue: Conceptual Tactics 269

Afterword by Arkady Plotnitsky 271

Notes 293
Bibliography 329
Index 357
Preface

If asked what the world is made of, we can say it's made of objects, or we can say it's
made of stuff. This book takes the point of view of stuff, the stuff of which obj ects are
made. Moreover, the world changes-Eppur si muove, to repurpose Galileo's legendary
observation. So any response to the question of what the world is made of should take
history, dynamism, and temporality into account. In more precise terms, my project
is to understand distributed, field-based activity and materiality in rehearsed as well
as unrehearsed situations in the presence of responsive media. In this philosophical
and interdisciplinary investigation, the strategy is to suspend or bracket certain con­
ventions about what constitutes body, subj ect, or ego while trying to develop a
working understanding of embodiment and subjectivation-the formation of subjec­
tive experience. Movement, and in particular gesture, is an arguably essential aspect
of engendering human experience. But rather than taking "the body" or "cognition"
for granted as conceptual starting points, I attend to the substrate matter in which
gesture takes place-hence the interest in responsive and in particular computational
media created for sustaining experientially rich, improvisational activity. The investi­
gation also puts in play notions such as interaction, responsive media, and performa­
tivity, and so aspires to contribute to contemporary exchanges between art and
philosophy. The betweenness is most essential. Though it uses evidence and even bits
of argument, this book is neither a mathematical proof nor a philosophical argument.
Perhaps, as Wittgenstein said of a far more logically credible investigation, "[t]his book
will perhaps only be understood by those who have themselves already thought the
thoughts which are expressed in it-or similar thoughts . . . . Its obj ect would be
attained if it afforded pleasure to one who read it with understanding. " l On the other
hand, this is also written with an ear for poetry with equal calls upon rigor and affect.
In this aspect, I have little intention to convince you of an analytic method, but I will
share, as well as I can in this first essay, an orientation, a way toward thinking and
making things with people.
I argue for an approach to materiality inspired from continuity, field, and philoso­
phy of process, based on ethico-aesthetic as well as technoscientific grounds. This
viii P reface

proj ect investigates what could be implied by continuous, or more precisely topological,
approaches to media and matter in the concrete setting of installation-events. Another
motive is to explore the ethico-aesthetic consequences of topologically creating per­
formative events and computational media, drawing from the critical studies of science
and technology. This project is a philosophical investigation that is conducted in a
poetic mode of installation or event-based art and technology. This study of gesture
and agency is informed by scholarship in multiple literatures: philosophies of process
represented by Heraclitus, Henri Bergson, Alfred North Whitehead, Gilles Deleuze,
Isabelle Stengers, Gilbert Simondon; certain mathematico-poetic philosophies repre­
sented by Rene Thorn, Gilles Chatelet, Michel Serres, Jean Petitot, Alain Badioui and
theories of distributed agency represented by Humberto Maturana, Andrew Pickering,
Donna Haraway, Edwin Hutchins, and of course Gilles Deleuze and Felix Guattari.
Methodologically, its critical relation to psychology and cognitive science draws from
Ludwig Wittgenstein, Edmund Hussed, William James, Eugene Gendlin, and Felix
Guattari, with implications for other attempts to quasi-scientifically systematize prac­
tices such as design, engineering, or art.
However, to cite these authors does not imply a subscription to a school of thought
or a ready-made method, only that elements of these conceptual approaches have
proven fruitful in furthering the understanding of some aspects of my chosen phe­
nomena of study: field-based materiality and activity. One of the motivations for my
project, in fact, is to contribute coherently to this multipolar conversation by produc­
ing a genealogy of topological media. Using the word genealogy, I am mindful of
Foucault's critical and nonteleological approach to history. In an analogous way, this
project offers a detailed and critical reflection on theories of distributed, dynamical,
and processuaI matter that have been of interest to humanists over recent decades.
I try to discover, critically, the antecedent assumptions that have evolved into
certain conceptual frameworks that are taking hold in contemporary academic
approaches to media and art and literature, especially as they appeal to nearby fields
of design and cognitive science. The critical proj ect reflects not only on what concepts
of plenum materiality and distributed agency are being constructed and deployed, but
also on how they are being constructed and deployed, by whom and with what effect.
As such, this project should contribute to philosophy of process and subjectivation,
philosophy of art and technology, as well as historical and critical studies of technol­
ogy and science as practiced by Isabelle Stengers, Ian Hacking, Bruno Latour, Tim
Lenoir, Niklas Damiris, Helga Wild, Brian Rotman, Steven Shapin, Donna Haraway,
Kavita Philip, Mario Biagioli, Peter Galison, David Bloor, Lucy Suchman, Kriss Ravetto,
Mike Fischer, Doug Kahn, Frances Dyson, and many other scholars.
In recent history, there is an equally distinguished and diverse set of thinkers, such
as Nietzsche, Freud, Lacan, Levinas, Bataille, Blanchot, Derrida, and Lyotard, who have
worked profoundly with concepts such as difference, disconnection, discontinuity,
P reface ix

and atomicity that are complementary to the approach of this book. One can situate
this book with respect to those thinkers in several ways. First, while pragmatically
respecting their complementary and profoundly humane insights, one can perhaps
see a basic difference in "first philosophy. " Anthropocentric thought begins with the
human condition and with relations between human subjects, and then proceeds to
think about the world. By contrast, a nonanthropocentric philosophy starts with ques­
tions about the world, and then considers the social or the human via a cosmological
perspective. This project takes the latter approach. Second, as I said at the outset, this
book is neither a mathematical proof nor a philosophical argument. Rather than
debate, I would prefer to make an alternative art or technology. I do not presume to
explain what the world really is made of or how the world really works, or what it
really means to be human. Indeed, the work that I have done with speculative artists,
philosophers, and technologists does not debate, but gives a sense of how one might
regard with a certain " as if. " Inspired by the tactics 'of a de Certeau or the situationists
vis-a-vis their city, or Grotowski's nonperforming performance laboratory, we've found
a few conceptual tactics over the years, a set of orienting tropisms, what Stengers and
Whitehead have called lures for feeling and thinking. They are particularly elaborate
lures, informed by political, artistic, and technological practices. But they are not
recipes or methodologies. Comparing and contrasting these orientations against
apparently competing domains of thought, while valuable as a scholastic exercise,
would eat a great deal of patience and energy that may be better reserved for trying
on this book's alternative orientation for fit.
Although this is a project of reflection whose main tangible product is a book, it
draws on a critical familiarity and engagement with recent material practices in the
mise-en-scene of installation art and performance: computational video, sound,
sensors, active textiles, and so forth, as well as specific experimental researches in
performance, movement, and visual arts. I draw not only on my own work, but also
on a set of ongoing professional conversations with Michael Montanaro, Toni Dove,
Joel Ryan, Tirtza Even, Laetitia Sonami, Michel Waisvisz, Sponge, and FoAM, and
informed by other contemporary artists such as Ann Hamilton, Kiki Smith, Mona
Hatoum, Janet Cardiff, Dan Graham, and Robert Irwin. To understand, ai"l:d to feel,
how these arguments matter at sufficient scope and depth requires an intimafe engage­
ment with experimental performance or installation-events, and with specific techno­
scientific research programs. This investigation accompanies, situates, and reflects on
the speculative material practice. I hope to recirculate the conceptual fruit of this
investigation in the communities of allied artists and technologists, and am most
grateful to the many fellow scholars, artists, activists, and students who have traveled
with me in these past two decades of speculative practice.
Acknowledgments

This project is a philosophical investigation that happens to be conducted in the mode

of performative arts and technology. Therefore it owes a great debt to an unusually
diverse number of fellow scholars, artists, and technologists.
First I thank the university hosts who have generously welcomed me as a visit­
ing scholar to work on this manuscript: Tim Lenoir, History and Philosophy of
Science, Stanford University, Peter Galison, Department of the History of Science,
Harvard University; Mike Fischer, Science and Technology Studies Program, MIT;
Robert Harrison, Department of French and Italian, Stanford University; Chris Fynsk,
Centre for Modem Thought, University of Aberdeen. I thank Concordia University
and the Faculty of Fine Arts for their continuous support since I assumed the
Canada Research Chair in New Media, and in particular Dr. Louise Dandurand, Vice
President of Research and Graduate Studies; Dean Catherine Wild, Associate Dean of
Research Lynn Hughes, Lyse Larose, Mich Sardella, Kathy McAleese, and my faculty
colleagues.
One foundation has been my teachers and mentors in mathematics, induding
Andrew Gleason, David Mumford, Bob Osserman, Brian White, Rafe Mazzeo, Leon
Simon, Rick Schoen.
This project has taught me much about the philosophy of process and the historical
and critical studies of technology and science thanks to the scholarly company of Tim
Lenoir, Isabelle Stengers, Niklas Damiris, Helga Wild, Kavita Philip, Ben Roblnson, Ann
Weinstone, Ron Broglio, Hugh Crawford, Ken Knoespel, and many others. I am
indebted to conversations enlightening, delightful, and often both with Anne Balsamo,
Peter Galison, Satinder Gill, Friedrich Kittler, Bruno Latour, Arkady Plotnitsky, Kriss
Ravetto, Arielle Saiber, Steve Shapin, Lucy Suchman.
These theoretical discussions find earth and nourishment in professional con­
versations with Tirtza Even, Laura Farabo, Maj a Kuzmanovic, Michael Montanaro,
Toni Dove, Joel Ryan, Chris Salter, Scott Minneman, Laetitia Sonami, Michel Wais­
visz, Sam Auinger, Ingrid Bachmann, Jason Lewis, Joanna Berzowska, Sponge,
FoAM, the Blue Riders chamber ensemble, and other artists who have generously
xii Acknowl e d g m e nts

shared their artful lives. The other side of material practice includes a select group
of engineers and technologists who have achieved a humane wisdom all the more
remarkable because it has been cultivated in the face of present inhumanity. These
inspiring people include Mike Cooley, Anatol Holt, and Karamj it Gill. For generous
and clever company, I also thank Larry Leifer, Blair MacIntyre, Irfan Essa, Aaron
Bobick, Thad Starner, Peter Grogono, Jeremy Cooperstock, Sid Fels, and Sabine
Bergler.
Inspired by a couple of decades of activism, this book is partial thanks for the
company of many courageous people and passionate scholars, including Eulalio
Balthazar, Amy Rocha, Miriam Patchen, Ben Robinson, Diane Nelson, Kavita Philip,
and Ward Smith.
Perhaps the most gratifying aspect of this project has been the nurturing and
growth of the garden that has been the Topological Media Lab. The atelier owes its
good earth largely to the wide-ranging intelligence and competence of Harry Smoak,
who started with the lab in Atlanta and built it as a research-creation atelier in Mon­
treal, together with Josee-Anne Drolet, the heart of the lab.
I thank the students and independent spirits who found their way to this social
institutional experiment, sharing so much art, intelligence, trust, craft, muscle, humor,
and love: Freida Abtan, JoDee Allen, Yannick Assogba, Nadine Asswad, Jim Bell, Chloe
Bertrand, Dave Birnbaum, Nina Bouchard, Laura Boyd-Clowes, Noah Moss Brender,
Christoph Brunner, Judith Penney Burton, Yvonne Caravia, Niomi Anna Cherney,
Soo-yeon Cho, Sarah Choukah, Jill Fantauzza (Coffin), Graham Coleman, Erik Conrad,
Maria A. Cordell, David Covo, Meredith Davey, Anahita Dehnonehie, Kiani del Valle,
Jerome De LaPierre, David DeMumbrun, Hugo Desmeules, Lina Dib, Steven Dow, Olfa
Driss, Josee-Anne Drolet, Patricia Duquette, Alexis (Laura) Emelianoff, Omar Faleh,
Vincent Fiano, Julien Fistre, Louis-Andre Fortin, Michael Fortin, Karmen Franinovic,
Elena Frantova, Adrian Freed, Rosalie Dumont Gagne, Charles Gagnon, David Gau­
thier, Chaim Gingold, Toby Glidden, Orit Halpern, Ayesha Hameed, Patrick Harrop,
Jason Hendrik, James Yu-Cheng Hsu, Stephen Ingram, Jhave David Johnston, Katie
Jung, Zohar Kfir, Rodolphe Koehly, Shirley Kwok-Choon, Anne Lambert, Valerie
Lamontagne, Justyna Latek, Seungyon Lee, Maroussia Levesque, Jason Levine, Flower
Lunn, Zhiming Ma, Joel McGinnis, Jehan Samir Moghazy, Delphine Nain, Navid
Navab, Anne Nigten, Magdalena Olszanowski, So-Young Park, Kevin Quennesson, Filip
Radonjik, Jim Ransone, Wolfgang Reitberger, Troy Rhoades, Jean-Sebastien Rousseau,
Tristana Martin Rubio, Yu Satow, Shermine Sawalha, Yoichiro Serita, Elliot Sinyor,
Harry Smoak, Jen Spiegel, Kevin Stamper, Julian Stein, Morgan Sutherland, Tim Sutton,
Kulavee (Kat) Tej avanija, Emmanuel Thivierge, Jane Tingley, Junko Tsumuji, Tyr
Umbach, Doug Van Nort, Yon Visell, James Wang, Matthew Peters Warne, Magda
Wesolkowska, Pegah Zamani.
Acknowledgments xiii

I thank my editor Doug Sery and the editorial staff at the MIT Press for an abiding
interest in this project. This book owes significant improvement to the reviewers who
generously and carefully provided detailed, constructive advice. I thank Harry Smoak
for helping with the textual revisions, and losee-Anne Drolet with the images and
figures.
And I thank my family for unconditional support throughout the years.
Chapter Summary

Chapter 1 : Why This Book?

What is at stake: the potential for ethico-aesthetic experiment. The first chapter orients
the book for readers concerned with what is at stake given computationally augmented
and nondigital responsive media and responsive environments. The book appeals to
artists and philosophers of media who are concerned with ethico-aesthetic as well as
political implications in contemporary material practices in media and the technolo­
gies of performance. Setting aside transcendentalist appeals to universal immortal
frameworks structuring our experience, and in the absence of any Archimedean point
external to subjective experience upon which we can lever social and ethico-aesthetic
judgment, what remains? How can any sense of sociality and pathic subjectivity
emerge? This chapter introduces the argument for a deeper approach with the poetic,
rather than instrumental or technical, use of continuous topology and related modes
of nonatomistic articulation. The argument is substantiated and informed by specula­
tive projects over the past twenty years that challenge existing paradigms in compu­
tational media technology and media arts.

Chapter 2: From Technologies of Representation to Technologies of Performance

This chapter rapidly recapitulates what I consider the most salient forces motivating
the move from technologies of representation to the technologies of perform'ance. The
forces derive from critical history as well as engineering advances in computational
media technology. We review some of the core crises of representation that thread the
modernist and postmodernist moments, through the lenses of Bruno Latour and Akeel
Bilgrami. With the advent of electronic computation, representation in its particular
form of the scientific model comes alive in the mode of numerical simulation and
graphical visualization. I concretize this in the context of the role of musical notation
in twentieth- and twenty-first-century performance, and the impact of computer
technology. A key phenomenon here is how the non-real-time computer model as a
xvi Chapter S u m mary

tool of scientific analysis has transformed into a real-time instrument for live perfor­
mance, thanks to the increase in computer hardware power, the enrichment of the
operators, and the transformation of attention from modeling in virtual computer
space to shaping, manipulating, and articulating the material world live, in real time,
Le., the technologies of performance.

Chapter 3: Performance in Responsive Environments, the Performative Event

If we are to create events that are not allegorical, and that have an authentic and
immanent rather than representational relation to their content, these events ought
to be constructed not using technologies of representation but rather technologies of
performance. Moreover, if we aspire to create events with affective and sociopolitical
power, then it matters how we fashion our environments. In other words, unless the
techniques and the technical practices are also, to use a shorthand expression, topo­
logical, creating representations of topological events using conventional atomizing
schemas and obj ect-oriented technologies merely produces simulacra of play, which
has the same effective constraints as the most restrictive, disciplinary games.
We return to a fine-scale, process-oriented approach to distributed agency, inten­
tional or nonintentional gesture and movement. And we investigate concretely the
experience of rich, corporeal, live events in built environments or installations filled
with thick, responsive media. The canonical examples come from a family of related
installation-events envisioned and built over the past decade. In such installation­
events, we discuss questions of superposed agency, of collective versus individual
action, of correlates (rather than certificates) of intentional gesture, and other topics.

Chapter 4: Substrate

DHourning Antonin Artaud's call for attention to the materials of performance, after
interpreting performance more broadly via the technologies of real-time, live perfor­
mance in responsive environments, we argue for a turn to examining the substrates
in which events and obj ects take shape. In place of epistemic and hermeneutic inves­
tigations that require explicit analytic obj ects like Subjects, Egos, or Roles, organized
into a priori taxonomic structures, we start with an experientially continuous ontology
of plenum or field. This requires unpacking distinctions between the discrete, the
algebraic, the atomic versus the continuous, and developing some notions of the field,
material plenum, substrate, and tissue. This discussion traces a history of arguments
that includes Heraclitus, Spinoza, Leibniz, and Whitehead in the West. Of course,
bracketing obj ects does not deny that obj ects exist. It shifts the ground to considering
how obj ects come to be, i.e., to ontogenesis. In order to articulate plenum and onto-
Chapter S u m mary xvii

genesis, we turn next to a rich set of concepts from point set topology, topological
dynamics, and deeper branches of continuous mathematics.

Chapter 5: Ontogenesis

Armed with the concepts of the previous chapter, we can turn properly to ontogenesis
without a priori obj ects, developing our consideration of distributed matter, substrate,
plenum, tissue. In this chapter we consider some philosophies of material process­
those of POincare, Whitehead, Stengers-and propose an approach to process-namely
dynamics-consonant with such process philosophies, informed by the more precisely
nuanced articulations afforded by concepts from topological dynamics and other
poietic arts, as well as by the technologies of performance in responsive media.

Chapter 6: Topology, Manifolds, Dynamical Systems, Measure, and Bundles

This is the core chapter, introducing concepts that articulate continua, continuous
substance, and continuous process. These concepts find precise and deep forms in
point set topology, topological and differentiable dynamical systems (qualitative,
topological, and geometrical approaches to systems of ordinary differential equations),
and the much more sophisticated perspectives of differential geometry and fiber
bundles. We introduce basic poetic concepts such as the open (closed) set, neighbor­
hood, map, space, continuity, connectedness, limit, convergence, compactness, and
so forth. Along the way, we consider the work of Brouwer, Thorn, and Petitot and
prepare the reader for a critical encounter with Petitot's program on ontogenesis.
Articulating matter with such anexact concepts seeds the ground for an alternative,
nonreductionist approach to ontogenesis.
Certain terms used in earlier chapters for their intuitive senses, such as continuous,
limit, dense, etc., will now be presented more rigorously, so that they can be used with
more precise connotations and conceptual purchase after this chapter.

Chapter 7: Practices: Apparatus and Atelier

The motto " art all the way down," which harkens to the amodern working ethos of
the preindustrial atelier, the Bauhaus fusion of craft and art, and the plenist ontologi­
cal commitments driving our object-free approach to ontogenesis, prompts us to
examine how such art practice and the critical studies of media arts and sciences can
be sustained in the sociocultural and capital economies of the arts and the academy.
What sort of working ethos can we derive to sustain the work of atelier-studio-labs
like the Topological Media Lab or FoAM and their kin? We derive practices that draw
xvii i Chapter S u m m a ry

from the collectivist practices of the engineering laboratory and the theater, as well
as the more solitary aestheticoeconomic practices of the art studio.

Chapter 8: Refrai n

Answering the challenge t o do art "all the way down," in place o f anthropocentric art
and science can we build world-oriented art and engineering? This motivates the
creation of events and technologies with a nonconventional notion of agency sans
agents. We harvest the implications of the previous chapters for articulating and
inhabiting the world as quickened matter. In particular, we consider materiality and
lifelikeness of objects as effects of process, rather than predicates on objects. Neverthe­
less, objects are not epiphenomenal, because they and the processes under which they
emerge as invariants are immanent in the substrate that constitutes the world. Fur­
thermore, when we articulate and inhabit the world in such a mode, the world
becomes as rich as we imagine, but without boundless complexity. This profoundly
motivates field-based rather than object-oriented or ego-oriented social technology
and technologies of performance sustaining ethico-aesthetic play.

The Role of Mathematical Notation in This Book

One of the exhilarating strengths of the Interaction and Media Group seminar at the
Stanford Humanities Center ( 1995-1 997) was the principle of drawing from all the
conceptual resources available around the table to gain purchase on our phenomena
of study (the nature of interaction, digital media) : whether it was contemporary theo­
rists such as Derrida, Kittler, Lakoff, or Foucault; or performance work by William
Forsythe and Dumb Type; or mathematical poetics like topology and differential
geometry. Two of our implicit working principles were a "principle of charity II and
"no dumbing down./I By a "principle of charity II I mean the starting assumption that
even if I don't know what you are talking about, I believe you do and that you are
saying something significant; so I will continue the conversation. By "no dumbing
down" I mean that if you do not share my area of expertise, I will not feed you super­
ficialities used only for advertising my discipline to "outsiders "; I will present habits
of thought that experts would consider significant as well, in notation that adequately
articulates the thought, yet is cleared of what (even) a master of the discipline would
think of as technicality.
It is in the same spirit of adequating language to thought that Heidegger con­
structed his neologisms to notate his philosophical concepts, to the benefit of those
who would work productively with those concepts. My ambition is much more
modest: I do not presume to invent so much as to adopt notation already well polished
by use. Leaving some concepts in their idiomatic notation, I give you access to some
C h a pter S u m m a ry xix

of their articulation so you may, if you choose, accommodate and adapt these habits
of thought yourself, rather than refer to them from a distant gloss.
However, anglophone critical and humanities studies, even the philosophical litera­
ture reflecting on mathematics, have tended to avoid the use of mathematical nota­
tion, taking Derrida's comment about the " silent" mathematical sign as a limit to
rather than an instrument of critical reflection. To some extent, this has been the
unfortunate and complicated reaction against the structuralist interpretations of mid­
twentieth-century students of human phenomena such as Jacques Lacan and Rene
Thom.
Roger Penrose cites Stephen Hawking about using mathematical notation in
"popular" physics books: every equation cuts the audience in half. I share Penrose's
respect for the reader, rather than the presumption made by most popularizers of
physics. As Derrida and Roy Harris recognized, mathematicians have invented signs
for two millennia to best articulate their ideas in their practices of thought. With
Penrose and Heidegger, I trust that if you are inspired by the aspirations of this work
as a whole, you will appreciate having some well-crafted notational handholds to
avoid "verbal" circumlocutions that obscure as they gloss. !

Time-Based Media References

Although one feature that paper enj oys compared to time-based media is a material
durability, this book would make more sense in tandem with media references to the
art and research on which it reflects. In the spirit of what mathematicians call a con­
structive proof, ten years ago I decided to build and find working indicators of what
could be the case, starting with a different sort of laboratory-the Topological Media
Lab-modeled after theatrical production, engineering research lab, and the preindus­
trial atelier.2
1 Why This Book?

In recent years I've taken to asking students and colleagues, " Why do you do what
you do?" Although that question is not the same as "Why do we live?, " it is not
unrelated, because how we live would be part of my own response to the question of
why we live. It's a phenomenological question about the experience of life, but I would
like to answer it in a poetic way in the context of contemporary and emerging tech­
nologies of performance, where performance is construed generously beyond the
domains of performing and performance arts.
One may aspire to do philosophy in the mode of poetry again, a Laozi multiply
transposed. But didn't Plato throw out the poets from the Republic because they oper­
ated in the realm of the fictive imitative, thrice removed from the truth, and therefore
were not to be trusted with the proper affairs of the polis? I'm writing this as an exercise
in philosophy in the mode of art, trusting that it can be done, that it matters not only
what we say or do, but how we say or do it. I'm wagering that both truth effects and
ethico-aesthetic1 paSSions can be accommodated in the same breath, the way math­
ematicians construct truths. However, mathematicians are not scientists, because their
theorems do not claim anything about the "real world. " Therefore they do not write
under the sign of empirical truth. Mathematicians prove theorems true or false within
propositional systems that they themselves construct. Therefore their constructions
are works of imagination. Writing neither under the sign of truth nor of fiction, math­
ematicians create truths via imaginative processes that can be regarded· as poetic
processes.
It is in this spirit that I would propose exploring some questions refined over the
years from their sources in crude, concrete, and technical craft. Together with fellow
artists, engineers, and scholars, I have explored those questions via a hybrid of mate­
rial and phenomenological experiments which have been built in the Topological
Media Lab and by affiliate art groups, notably Sponge and FoAM. Most importantly
this book shows how questions of craft, under inspection and reflection, can become
refined into philosophical questions. Under rigorous inspection, questions how
become questions why as well. Questions of philosophy in turn can provide heuristics,
2 C h a pter 1

Fig u re 1 . 1
Visitors, dressed in projected and sensor-laden costumes, playing in a TGarden responsive media
environment. Dutch Electronic Art Festival, Rotterdam, and Ars Electronica, Linz, 200 1 . Side view
and overhead view. Images courtesy of FoAM and Sponge.

though never blueprints or methodologies, for craft. The most compelling reason for
refining technical challenges into philosophical questions is to accommodate value.
Given that we can engineer A, B, or C, the question we ought to answer first is why
A, B, or C? Such an ambition places this book in the area of the critical studies of
media arts and technology. The book provides a thoughtful place, more ample than
the confines of a technical j ournal article, in which to resituate the work, and to
provide some sense of how some approaches to art and technology may be more fertile
than others.
However, this project of constructing a genealogy of topological media embodies
a more radical ambition, which is to produce matters of value as well as matters of
fact. To make sense of how we may approach the production of matters of value
occupies the central chapters of the work. Mindful of Foucault's view of history as
punctuated by rupture, my account of topology and potential reenchantment pre­
tends no progressivist history of ideas. The discursive field linking, say, Heraclitus,
Leibniz, Spinoza, Whitehead, Deleuze, and Stengers is just as present as the discursive
field of Democritus, Frege, Newell, and Simon. In any event, we have always been
topological. You may adopt various positions with respect to the concerns of this book,
and with them you may develop alternative conceptualizations of art and technology,
and alternative approaches to the material practices of artist and engineer at micro,
meso, or macro scales of process.
Why This Book? 3

Where Does It Spring From? Why Does It Exist?

Twenty-five years ago, in a letter I wrote to R, I metaphorically drew people as distri­

butions and argued that no synchronic sets of characteristics could ever determine
whether two people could be compatible together, that the real-time process of living
would answer itself. Already then, I was convinced that something smelled wrong
about the synchronic notions of modularity, compatibility, and fit, but I lacked the
analytic terms to describe the origin of the odor, and only inklings of the alternatives
based on history, evolution, flux, and process.
Perhaps the core of this work is a search for a way to live gracefully, but by grace
I mean something like and unlike Simone Weil's catholic sense of grace, like and unlike
the grace of Kierkegaard's knights of infinity, who hesitate infinitesimally just at the
moment of landing on terra {irma. Living well is a matter of why as well as how. It is
also an unfolding in temporal processes, in psychological, biological, historical, cin­
ematic, videographic, ecological, evolutionary, cosmological times. So we need to
approach the art of living as process, and imagine what the processes of living offer
us. What sort of process am I concerned with?
To answer the children (borrowing a convenient label from Isabelle Stengers) who
cite Clausewitz to justify war as a mode of political intercourse, I recall Felix Guattari,
who, at the end of his Chaosmosis, asks whether art is the appropriate mode of radical,
ethico-aesthetic experimental subjectivation. Guattari's hyphenation-ethico-aesthetic­
invites us to articulate together what Plato sundered: the arts of poetry with the arts
of truth. It matters not only that something works or is said, but how something works
or is said. What is done or uttered is inextricably the same as the manner in which it
is done or uttered. Even more radically, what is done cocreates what could be done,
or could have been done; in other words actualization coconstructs the potential. Ever
since I came across Guattari's third ontology of an asignifying stratum, a plenum in
which subjectivities form and dissolve in a magma of machinic assemblage, I have
tried to elaborate how that sort of magma would work, and to explore and make pos­
sible in detail the nuanced forms of such free, deschematized, and rematerialized
ethico-aesthetic gesture.
But what sort of art would it be? Certainly not art as self-expression or ego therapy,
nor even, pace Krzysztof Wodiczko's early work, a way to cope therapeutically with
the social world, as if the world were an illness. Should we insist that social critique
is nothing but collective therapy after capitalism and schizophrenia? That would be
a position just one crisis away from nihilism, a position I consider not worth the effort
of living a life.
So what would the object of art be? Under capitalism, does contemporary art neces­
sarily function in its residual forms as mimesis, hagiography, illustration, or social
memory, and above all as commodity? Can art be about making things with all the
4 C h a pter 1

aesthetic and critical contextualization before and after its obj ect? Can it be essentially
concerned with unmooring us from our literal and denotative or smug expectations?
Although art could help us reimagine the inhabitation of our built spaces, perhaps we
ought not begrudge people the need for comfort. Art could be simply about material
play. (If even mathematics and words have their materiality, then mathematics and
poetry can be performed as art as well, but we'll come back to that.) And fundamen­
tally it seems to be about objects, rather than process. It's telling that, aside from practi­
tioners themselves turned teachers, the most durable representatives of process art of
the 1 9 70s and 1 980s are the documentary images that we have. (Perhaps this is the
fate of every process, every performance, that its representatives have the last word.
As Derrida noted in his essay on Artaud, the representation of performance is its first
word as well, in an endless circle.)
Even as mathematics, engineering, and scientific business management have drawn
more and more upon abstraction, we've seen a sequence of critical moves away from
the abstract: the linguistic turn, the semiotic turn, the structuralist turn, the materialist
turn (again), and the turn to body. In each case, the turn goes through a naive phase
and a reductionist formulation. Take the turn to the material in its special case: the
body. In the simplest case, the turn to the body is a reduction to naive biologism (as
if a curled lip were fu lly determined by honest glee) .
Now what if we give up our conventions of body, ego, agent, object, but still wish
to understand and work more deeply with embodiment, with desire, with intentional­
ity and texture? What if we unmoor ourselves from our barnacle dependence on
objects and predicates and networks, to swim through our world as a dense, plenist
flux? How could we ever navigate, fashion, inhabit, form subjects, attachments, desires
in such a fluid world?
Now have I slipped art and philosophy into the same bed, as if by cohabitation I
expect them to produce whole offspring the likes of which we have not seen since
Zeus split the round atom, and Plato drove the poets from the Republic? In mundane
terms, this is the idea for collaboration, a sexual union of disparate species. But if we
honestly suspend our reliance on objects, on things in themselves with predicates, on
actual occasions or atomic events, then we ought not appeal to a model of work in
which artist and philosopher are separate species.
This is a methodological point, and an important one because it gets at the heart
of "how we get there from here." Honoring the American pragmatic turn, I feel that
even, or especially, in a book about philosophy as art, we need to say something about
how truly fused dispositions and approaches may offer more than juxtaposition or
collaboration. In 1 9 7 6 David Bohm, the physicist and philosopher, published a slim
volume titled Fragmentation and Wholeness in which he succinctly observed how our
modern rational analytic power to divide ourselves from our environment and to
divide the world into disconnected domains has fractured our life in the world:
Why This Book? 5

The process of division is a way of thinking about things that is useful mainly in the domain of
practical, technical and functional activities . . . . However, when this mode of thought is applied
more broadly to man's notion of himself and the whole world . . . then man ceases to regard
the divisions as merely useful or convenient, and begins to see and experience himself and his
world as actually constituted of separately existent fragments.2

At that time, the postmodern wave of rupture and arbitrary juxtaposition was still
cresting. But bricolage has had its day, and now we must gather its shards and make
an alloy of the pieces.
But this motivates us to appeal to flux, transform, stuff as a way to come up with
fresh-that is, poetic-ways to play in the magma of ethico-aesthetic activity and
gesture, collective as well as individual, diffuse as well as sited. In order to do so, we
should examine more closely the magma itself and see how we can play in it. It is for
that purpose that I construct a genealogy of topological media.

What Is Topological Media?

Topological media for me is a set of working concepts, the simplest set of material and
embodied articulations or expressions that allows us to engage in speculative engineer­
ing, or philosophy as art, and to slip the leg irons and manacles of grammar, syntax,
finite symbol systems, information and informatics, database schema, rules and pro­
cedures. I argue that topological media is an articulation of continuous matter that
permits us to relinquish a priori obj ects, subj ects, egos, and yet constitute value and
novelty.
Topology provides alternative, tough, durable, supple, and-to use Deleuze's term­
anexact concepts with which to articulate the living world, concepts like continuity,
open set, convergence, denSity, accumulation and limit points, nondimensional, infi­
nite, continuous transformation, topological space. To play on a motto from Latour,
we have always been topological. It's only in modern, or I should say modernist, times
that we've been so enamored of digital representations, discrete logic, digital computa­
tion, and quantization. I believe these concepts of continuity, openness, and transfor­
mation also can inform how we evaluate art and technology and enrich ti1� way we
make them. There is nothing mathematically fancy about the elementary 'topology
with which I begin, and this accords with my aim to make richness without complica­
tion. Nonetheless, impelled by the way we approach ethico-aesthetic creation, we will
appeal to significantly more developed mathematical patterns, most of which rigor­
ously and poetically exceed the digital, discrete, computational domain.
The discrete drops out as a special case, by the way, so we are not losing anything
of the graph theories (from syntax parsing trees to actor network theory), but just seeing
them in their place would be enormously useful. The space of discrete graphs is so
sparse as to be measure-theoretically null, entirely negligible at the human, meso scale.
6 C h a pter 1

It could be that one of the lures of the discrete has been the notion of choice,
discrete choice, which in turn has been associated with freedom. But choice "# freedom.
And indeed superfluity of choice may simply obscure freedom.
The lure is the possibility that these concepts could provide material and embodied
ways to shape, unshape, rework, knead the world. Contemporary engineering is not
based on the noncomputable, infinite, and continuous; therein lies the conceptual
and technical challenge and interest.

For Whom Is This Written?

I'm thinking with (penser avec, to use Stengers's beautiful notion) philosophers who
practice in the mode of art. Perhaps the most consistent way for me to do this would
have been to make an event out of this book, something more like the 24H Foucault,
organized by Thomas Hirschhorn as part of the Nuit Blanche in Paris in 2003 on the
proposition that Foucault was a philosopher who practiced in the mode of art. And
in a sense, I have, in directing the atelier-studio-Iaboratory for creation/research called
the Topological Media Lab, and in a decade of work as a member of the Sponge art
group. So this book can be viewed as an utterance, a long thought in motion rippling
out beyond the reach of the drops of material speculative installations and instruments
that my collaborators, students, and I have created over the past decade.
I am writing this as an attempt to think with the process philosophers: Heraclitus,
Laozi and Zhuangzi, Marx, Foucault, Whitehead, and with Deleuze and Guattari, to
make philosophical concepts as art. I am writing this as a letter to MK and other fellow
artists who ask, Why should we even try to create anything beautiful or j oyful in this
world? I'm writing this to articulate to my students and my friends a way of being in
the world-of creation of art, poiesis . . . not so much a definition of representation
or imagination but as permission and as ways to imagine other than the actual. (I say
"to" not "for" to be mindful of Stengers's observation that we can speak in front of
but not in place of those without vOice .)3 And I'm writing this for my son who
has asked me, ever since he was seven, why do we live, and what is the purpose of
our lives?

What Are the Stakes?

Give me a place to stand and I shall move the Earth.

-Archimedes

Since 1 848, utopian narratives of emancipation and liberation have been balanced by
criticisms of transcendental frameworks built around notions such as God, Nature,
ego (man), and now bit, gene, and network, from which there is no appeal.
Why This Book? 7

What are the transcendentalisms against which I'm guarding, underwriting histori­
cal categories such as class, race, gender, nation? These include database (with con­
comitant schema drag); naturalization; proceduralism and structuralism (with
concomitant brittleness); problem solving; shrink-wrapped designer speech and
behavior.
These transcendental frameworks, far from being abstract, have had enormous
material effect, especially as interpreted by their priests and revolutionaries.
Before we go further, why do we guard against transcendentalism or reductionism?
An important part of the twentieth-century motivation for this has been to resist the
inquisitional dogmatism, faScism, totalitarianism, and now fundamentalism in whose
names so much blood has been shed. As Simone Weil wrote in Oppression and Liberty,
" only priests can claim to measure the value of an idea by the amount of blood it has
caused to be shed, " and went on to question the "revolutionaries" of her day who
shed their own blood as copiously in the service of a "shade of Helen."4
Democratic politicS, as Ernesto Leclau and Chantal Mouffe pointed out in Hegemony
and Socialist Strategy,S is at heart based on the infinite continuability of debate. But as
we know, beyond the formal Habermasian requirements for effective communication,
what we need are the principles of charity and of balancing destructive with construc­
tive criticism, which in turn arise from a prior sense of care. But how does care appear
in the world?
Can we recover or construct solidarity, mercy, or interest, without a priori subjects?
If we let go our clinging grip on transcendental verities, how can we still create ethico­
aesthetic value?
Biopower today no longer acts only at the scale of docile human bodies (as Rabinow
and Dreyfus characterized Foucault's study of power), but has dispersed into the back­
ground texture of social and political life. So if our critical technologies, whether they
be technologies of entertainment (iconically the personal computer and the iPod)
or the psychiatric and public technologies of patient or citizen, articulate egos only
in the form of Adam or Eve, then they lie very far from where the contest really
takes place.
To anticipate the arguments of this book around topological media, cab there be
continuous, distributed agency, and what ethico-aesthetic invention would that
enable? How free can gestures be in reflexively responsive media? How can and how
do people improvise collectively meaningful gesture? Such questions crucially moti­
vate the study of media from a continuous topological perspective.
As Akeel Bilgrami succinctly put it in an essay on the modern roots of what he
termed thick rationality:
The metaphysical picture that was promoted by Newton (the official Newton of the Royal SOciety,
not the neo-Platonist of his private study) and Boyle, among others, viewed matter and nature
8 C hapter 1

as brute and inert. On this view, since the material universe was brute, God was externally con­
ceived as the familiar metaphoric clock winder, giving the universe a push from the outside to
get it in motion. In the dissenting tradition-which was a scientific tradition, for there was in
fact no disagreement between it and Newton and Boyle on any serious detail of the scientific
laws, and all the fundamental notions such as gravity, for instance, were perfectly in place,
though given a somewhat different metaphysical interpretation-matter was not brute and inert
but rather was shot through with an inner source of dynamism that was itself divine.6

Eighty years ago Max Weber famously argued that modern rationality, by separating
the religious from the rational, removed magic and myth from our world, which he
called the disenchantment of modern society. ? Perhaps modernity is not so monolithi­
cally successful as Weber claimed: what is thrown out by day returns with the night.
However, instead of accepting a split into rational and irrational life, instead of resort­
ing to magic tricks or to transcendentalist and fundamentalist retreats, we ask: Can
we make cracks in material, ordinary, physical situations in which extraordinary,
nonteleological poetic activity can emerge? We emphasize that we are not designing
experiences, or images or replicas of experiences, but the material background condi­
tions of the built environment; hence our resort to computational media and active
materials as substrates of performance rather than technologies of representation.
Reductionism is not merely judicious applications of Occam's razor. Nor ought its
opposition be simply a hearty wallow in arbitrary pools of superfluity. (Burning Man
is merely the antipode to industrialized property economy that reinscribes technologi­
cal excess.) In the terms invoked by Bilgrami's observation, what's at stake is the
reenchantment of matter.

The Main Argument of the Book

Papa, did you know when I sit down on the ground, I'm already touching the whole world?
[How is that, Gabriele?]
When I'm sitting on the floor, the floor is touching the earth, the earth is touching everyone,
so I'm already touching everyone . . . and the whole world!
-Gabriele Weimin Carotti-Sha

Despite the range of art, technology, and thought through which my account will
travel transversally, but non trivially, this book is a single thought. Therefore let me
condense the thought of the entire book into one paragraph. This underscores that,
despite the apparent diversity of diSCiplines and practices due to their accidentally,
historically evolved boundaries, the thought has a coherence and compactness. On
the other hand, one should hardly expect to grasp the book's thought expressed this
way since it comes here ahead of all the development of intermediate observations,
Why This Book? 9

reflections, and most importantly the evolution of a notation adequate to the thought.
(I say "notation" instead of "vocabulary, " for reasons that may become clearer in
chapter 2.)
[fwe set aside transcendentalist appeals to universal immortal frameworks structuring our
experience, and in the absence of any Archimedean point external to subjective experience
upon which we can lever social and ethico-aesthetic judgment, what remains? How can any
sense of sociality, solidarity, pathic subjectivity emerge? Not from an atomic world, because
we run into complexity and the problem of intersubjectivity-the problem of how monads or
groups of monads sum to one society. However, if we start with a plenum-already one
substance-then we have, not a starting place-an Archimedean leverage point-but a
magma of costructuration that can be the substrate of subjectivation. This magma is already
continuous and laden with value, saturated with time and all other quality-creating processes.
This magma is not reductionist because it admits infinity and the imaginary-with bound­
lessly many modes of potential being. All monads, being formed in/out of this magma, are
already touching, therefore making ethical action possible. The dynamical behavior of the
world's distributed media is costructured with our noematic experience of the world. Hence
the apparently simultaneous emergence of shared patterns of behavior or recognition. The con­
temporaneity is an artifact of the contemporaneous time slice (or Poincare section) of the
evolving world. It's the very acausality of that contemporaneous region coimplicated with
the nonforced, nondeterminist realm of action that is ethical.8
This book provides the motivation, background, mode of articulation, elaboration,
and implications of the preceding paragraph. (I prefer to say articulation rather than
"context" or "language" to avoid falling back onto the very same crutches of repre­
sentationalism, linguisticism, anthropocentricism that have hobbled thought.) This
investigation is a philosophical, not a scientific one, because it makes no claim to
verisimilitude with respect to some naive empirical notion of nature external to and
divided from subj ective experience. Nor is this a methodology: it prescribes no recipe,
no rule-based procedures to govern social, political, economic, or design practices. Yet
I do pose approaches to practicing art and engineering in a mode of rigorous specula­
tion most closely aligned with creative, speculative mathematics. To call these
approaches "principles" would be presumptuous; what I suggest are an o'Nen set of
attitudes toward the material and practice of art and engineering that ar � critical,
poetiC, and informed by an inside knowledge of artists' and engineers' experience.
Ten years ago, I decided to publish written arguments and perspectives in tandem
with making exemplary instances of this approach to articulating the world-media,
performance events, installations, software algorithms and instruments, workshops,
institutional organisms. This decision aligned with the pragmatic spirit infusing the
late twentieth-century United States. It also constitutes a material analog to what
mathematicians call a proof by construction, a constructive proof.
10 C h a pter 1

What Experiences I nform This Book?

At heart, what I describe is not a set of technologies that would homogenize practice,
but an attitude toward the design of technology, a disposition with respect to living
in the world and shaping it as more than a set of ready-made recipes or synchronic
schemas. This approach is substantiated by nearly thirty years of work in various
domains of art and engineering, two fields in which practitioners make a virtue of
material work and substantiation of concept in alinguistic creative processes. Although
the works9 have a continuous history intricately intertwined with the conceptual
development over the same period of time, I'll introduce the earlier works ( 1 984-1 993)
in this first chapter and present the later works in chapters 6 and 7, after we have
some concepts that will make sense of their approaches. The earlier works include a
series of physics simulations and social and historical simulation games, and the
applications of the MediaWeaver distributed obj ect-oriented multimedia management
system. The later works (2000 on) include speech recognition in public urban spaces,
responsive media environments, live (real-time) gestural media, media choreography,
and soft architecture. Essentially the dividing point is the great die-off in the diversity
of the applications of computational and network technologies that took place when
HTML and httpd spread like kudzu around the world.

Bias Cabrera and Andrei Linde's Physics S i m ulations and Visualizations

BIas Cabrera and Andrei Linde's physics simulations and visualizations made palpable
via the computer physical realities inaccessible to our ordinary vision and touch.
Rather than merely present animation of canned physics, Cabrera's goal was to create
computational microworlds that numerically simulate aspects of the physical universe
normally inaccessible to the human, and allow the human to conduct virtual experi­
ments in the simulated world. Students were required to build analytic, mathematical
models and at the same time create and observe experiments in the simulated envi­
ronments that we built.
In 1 984, Apple and IBM seeded several universities, including Stanford and MIT,
with a revolutionary new personal computer, the Lisa, to discover what could be done
with these graphical user interfaces on small computers that could not be done with
computers or any other antecedent technology in the world of teaching and research.
They also provided funds to hire the first generation of programmers for this new
operating system and programming environment.
For a heroic decade, we extended our practice of microworld simulation to interac­
tive simulacra of historical and social microcosmoi, a move profiled by Jean Baudril­
lard.10 We extended our computational visualization software technologies to present
images of differential geometric and topological structures that one could never
encounter in the flesh (or so I thought at that time) . For generic system X, however,
Why This Book? 11

a simulation of X is not the same as X, and, practically by definition, one's experience

of X is not the same as one's experience of a simulation of X. l l

MediaWeaver Distributed M ultimedia, 1 993-1 995

Ten years later, the developers of social simulations were spending up to one year
creating media-rich applications, with complex interlocking rules on what media had
to be presented under what conditions to the user. Everything, from the creation of
the media to the logic, was custom-crafted for the particular form of the simulation,
whether it was a videodisc-based conversation between the student and a fictive story
space, or a graph-based visual programming interface to create general physics simula­
tions, or a 3D lighting instrument plot program to help a lighting designer visualize
and plan a light plot for a theater. The interactive narrative projects seemed ripe for
some optimization, because that class of applications seemed to share a common work
flow, from the basic research into a set of social relations, media asset collection and
creation, coding of simulation logic, and presentation in a multimedia screen-stage,
etc. Major logistical challenges included coordinating teams of creators and program­
mers on networks of computers, rewriting logic which meant recoding, repurposing
media, and handling new media formats. Ten years later, such problems transposed
themselves to the industry of game design, which has begun to approach the complex­
ity of film production.
For the MediaWeaver, I imagined the earthwide network of stored data as a single
ocean of bits on which multiple structures and lenses (optics) could be overlaid. (In
fact, in 1995 this was much more than a metaphor. MIT's Andrew File System or AFS
j oined a set of hundreds of UNIX computers around the world to present a single,
unified file system. Sitting at my desktop, I could drag not just a document but any
visible file from a computer in, say, Japan onto my own desktop. This file-level unifi­
cation was much deeper than httpd's sharp restriction only to passive documents that
had to be structured as HTML text files, and could only be viewed in a special applica­
tion, a "browser, " that rendered HTML.) A set of bits could be interpreted simultane­
ously as an image, or a sound file, or as strings of characters, or even as an operation
to be performed on other data. The multiple structures could be provided hy distrib­
uted relational databases and obj ect-oriented media archives plus metadata markup.
Alternatively the structures could be custom patterns imposed by commercial or per­
sonally written "editor" applications. At a finer grain, tools provided "lenses" through
which one could view the media. The emphasis lay on multiplicity of interpretation,
and on an unboundedly rich space of operators on data.
The conundrum was how to make available to the authors of these socially and
culturally rich multimedia simulations the tools that could register the predicates and
relational database schema structuring their media ontology, while at the same time
retaining all the expressive power of the media editing tools with which they were
12 C h a pter 1

familiar. Further, the forms of representation and protocols never remained static but
would evolve over time. The MediaWeaver was designed to provide the infrastructure
that would allow the composition and population of rich media environments that
could sustain events ranging from physics simulations to reenactments of French
theater from the Renaissance through the twentieth century in hybrid physical­
computational built space. It used a multipronged strategy to accomplish this. ( 1 )
Designers could use not just one structured schema but a multiple and dynamically
variable number of databases to describe the relations among its set of media. (2) Each
object could be represented by an equivalence class of concrete media proxies of any
type-text, image, sound, stream, executable code, and so forth, even types yet to be
invented. (3) The system provided a set of services (dynamically supplied from a global
network) that could convert media obj ects from type to type, for example deriving a
paragraph of text from the audio channel associated with a video clip as a summary
for a client application that needed text. This followed the principle that the space of
transformations of a base set of objects is at least as important as the base set itself. As
I will summarize it at the end of this chapter, the tactic is to move from working with
nouns to working with verbs. (4) No interface was imposed, but rather the Media­
Weaver managed and supplied these media objects, links, and metadata using stan­
dard commercial applications as well as a set of interface kits, under UNIX, Macintosh
Hypercard (the precursor of Director / Flash), and NeXTStep (precursor to Mac as X),
and to the World Wide Web via httpd and CGI.
The MediaWeaver database presented a limit case of relational databases and an
object-oriented approach to handling the mutability and interconvertibility of
humanly parsable media. 12

A Remark from Sociology of Technology

In that early epoch of finding ways to use "personal computing" in humanistic, liter­
ary, if not aesthetic applications, it was enlightening to witness how scholars who
were clients external to the new technological arts and sciences progressed through
an arc of attitudes toward computational technology. The successive attitudes could
be described as remediation, awe, disillUSionment, false expertise, and, very rarely,
virtuosity. Over my decades of working with individuals and professional communi­
ties, it seemed that mathematicians, artists, poets, historians all went through at least
a few of these attitudes, especially infatuation and disillusionment. The wish for reme­
diation was expressed by a demand such as: "I just want to write a book, and have
the computer tum the pages. " The graphical user interface fed and accentuated this
conceit. Some encounter with the unique and eye-opening features of desktop com­
puting, and later of networked computing, led some to an attitude of awe: "The
computer can do ALL. " But the experience of the rigidities of technology, and the brittle­
ness in particular of software riddled to a Heideggerian depth by bugs, plus the
Why This Book? 13

long development times required to make something sufficiently robust and rich for
ordinary use, led to an attitude of disillusionment. Depending on the person, their
encounter with computational technology could also lead to a false sense of expertise.
Individualists might insist on their idiosyncratic inefficiencies and circumlocutions
and view their continued cottage industry as vindication of their DIY (do-it-yourself)
method. DIY practices have run the gamut from writing personal applications to do
what can be done with off-the-shelf commercial software, to creating custom lan­
guages within which one can express a certain computation that extends an applica­
tion. Brilliance, a quality abundant among mathematicians and literary scholars, can
accentuate the tendency to DIY. On the other hand, collectivists rush to standardiza­
tion, the more global the better. What I call the tendency to "reach for your ISO"
(International Standards Organization) percolates into almost every large symbol­
processing industry, including electronic documents (SGML-XML), 3D graphicS
(VRML-X3D), and video (MPEG-I-MPEG-2 1 ) . Beyond skill and knowledge lies virtuos­
ity: consummate skill with its particularity, plus consummate knowledge which brings
professionalism and perspective, plus an expressive leap that finds fresh but idiomatic
ways to use techniques not as black-boxed technology, but as developable ground for
prepared improVisation.

G eometers Workbench and the Holy G rail of the Magic Blackboard, 1 998--2000
After about a decade13 of working with different computational tools for doing research
in differential geometry and topology, I wondered why it was, after fifty years of work
in logic programming, automatic theorem proving, 3D graphics, and numerical simu­
lations, that computers were so unuseful for the actual day-to-day work of creative
mathematical work. This may seem surprising, but the bulk of the free creative math­
ematician's activity has little to do with calculation and graphics, as conceived by
computer scientists and programmers. Taking a step back from both logicians' and
programmers' externalist cartoons of mathematicians' practice, I studied in particular
what really existing differential geometers do in their native habitat, in front of black­
boards, talking over coffee, and typing in TeX. In fact I looked at their gestural activity
as much as their verbal activity, trying to bracket linguistic assumptions �bout how
signs are used, yet paying attention to the differential geometer's phenomenological
experience of differential geometric entities: the constantly evolving tissue of defini­
tions, theorems, proofs, estimates, conj ectures about objects, functions, classes of
entities, etc. This led me to realize that the most basic activity was traced in a mode
of nontelementationalist writing, writing that I argued constituted mathematics rather
than "represented" preexisting, transcendental forms. My key interest here was to shift
the perspective from tools for representing idealized, crystalline mathematical objects
to tools for creating or fashioning them, tools of mathematical performance. Thanks to
Terry Winograd and colleagues in Stanford's Information Mural research group, in
14 C h a pter 1

particular Franr,:ois Guimbretiere, I was able to realize a "blackboard" that mapped

freehand gestures to algebraic and differential geometric operations. I'll say more
about this in chapter 3 .
The key insight here i s that writing can b e more usefully understood a s a collective
processual constitution of fresh entities and relations rather than a static representa­
tion of transcendental objects. One could transpose this approach to all sorts of sign­
making activity, in fact to the entire domain of semiotics, and beyond. In fact, I
transposed many of these questions about writing and poiesis to the domain of visual
arts, media art, responsive environments, when in 2001 I founded an atelier called
the Topological Media Lab to study such questions experimentally. 14

H u bb u b Speech-Sensitive U rban S u rfaces, 2000-2002

Continuing in this vein of evaluating and conceiving the technologies of writing as
performative technologies, I imagined an installation based on speech recognition.

F i g u re 1 .2
Hubbub installation, speech-animated dynamic glyphs. Projection onto steel cloth, Chrissy Field,
San Francisco, 2002. Photo by the author.
Why This Book? 15

What if, I speculated, surfaces in public spaces were to register fragments of text from
casual spoken conversation, so that ephemeral speech would acquire some of the fixity
of writing? Moreover, what if these glyphs were to dance and reshape themselves
according to the timbre and dynamics of the voices that speak the words, so that the
glyphs acquire some of the prosody of speech? How might social spaces thicken in
the presence of such partial condensation of speech in shared spaces? Over three years,
first as artist in residence at Jason Lewis's Arts Alliance Lab in San Francisco, I created
a series of speech-sensitive installations in public spaces, in San Francisco, Brussels,
and Atlanta. These installations sidestepped the problem of " surveillance" by the
idiomatic capitalization of the very errors and ambiguity of the technology. The design
took advantage of the errorful speech-to-text transcription to detach the sign from the
lips of the speaker. Moreover, the glyphs circulated through a given public space
according to dynamics that were predesigned for the site, and so this further material­
ized the autonomy of the glyphs. The early Hubbub experiments allowed a carefully
prepared but playful relation and projection between the intention of the speakers
and the latent, responsive dynamics of a speech-sensitized site.
Each of these projects (MediaWeaver, Geometer's Workbench, and Hubbub) consti­
tuted an extensive response to and against prevailing technical conceptual frame­
works, and as such each constituted a fairly elaborate probe into the sociotechnical
and associated cultural, ethico-aesthetic milieu. Each probe was a diagnostic embody­
ing internalist but critical response to simulation and what would become the trope
of virtual reality; to multimedia and what was to become the trope of the World Wide
Web; to a limit case of the augmentation of knowledge via computer representation;
and to pattern recognition as a technology for public discipline. Building on the cri­
tique, these responses constituted computational technology invented according to
scientific, humanist, and artistic desiderata rather than market or industrial norms,
and they constituted material interventions based on expert internal knowledge.

Conceptual lily Pads, landing Spots

Out of this spiral of work from physical and social simulations, through geometrical
and cosmological visualizations and distributed media archives, to media art and
technologies of performance based on real-time media resynthesis from gesture and
movement, emerged vignettes and meditations, concepts, arguments, rants, and judg­
ments that inform this book. Together they constitute an adventure in experimental
phenomenology.
I'm trying to discover and mix together mathematics as materials that are adequate
to life, because mathematics has a peculiar power to intertwine the imaginary and the
actual. It could be sharply different sorts of poetic, symbolic matter: continuous topo­
logical dynamics, geometric measure theory, or even fancy stuff like noncommutative
16 C h a pter 1

algebra and etale cohomology. But I choose to start with the simplest symbolic sub­
stances that respect the lifeworld's continuous dynamism, change, temporality, infi­
nite transformation, ontogenesis, superposability, continuity, density, and value, and
are free of, or at least agnostic with respect to, measure, metric, counting, finitude,
formal logic, linguistics (syntax, grammar), digitality, and computability, in short the
formal structures that would put a cage over all of the lifeworld. Simplicity here is not
a requirement of the theory (no Occam's razor here) but merely an acknowledgment
that I do not understand enough about the lifeworld to bring out fancier stuff yet, of
which there is so much more up the wizard sleeves .
The fundamental difference i n this approach is t o use mathematics a s substance i n
a workmanlike way, patching here and there to see what values ensue, a s a trellis for
play, rather than a carapace, but always considering whether the poetic material
accommodates transfinite, incommensurable, immanent passion. Totalizing carapaces
like Wolfram's computational equivalence principle, which at bottom is a transcen­
dental atomic metaphysics founded on making counting sacred, would hammer us
into a very sparse ontology. And to a hammer everything is a nail.
Why mathematics? Mathematics is conventionally cast as the quintessence of cer­
tainty, which is equated with dry rigidity. It has, however, the advantage of being a
mode of articulation that escapes (and exceeds) the linguistic, a mode of argumenta­
tion and disquisition that escapes the legal and the political, and a mode of measure­
ment that escapes the naive notions of the senses and sense data. Then what value
lies in looking to mathematics? Isabelle Stengers wrote, in her essay " A Constructivist
Reading of Whitehead's Process and Reality":
Abstractions, for Whitehead, are not "abstract forms " that determine what w e feel, perceive and
think, nor are they "abstracted from" something more concrete, and, finally, they are not gen­
eralizations . . . . [A]bstractions act as "lures", luring attention toward " something that matters",
vectorizing concrete experience. Just think of the difference between the mute perplexity and
disarray of anybody who faces a mathematical proposition or equation as a meaningless sequence
of signs, as opposed to someone who looks at this same sequence and immediately knows how
to deal with it, or is paSSionately aware that a new possibility for doing mathematics may be
present.
In order to think abstractions in Whitehead's sense, we need to forget about nouns like "a
table" or "a human being", and to think rather about a mathematical circle. Such a circle is not
abstracted from concrete circular formsi its mode of abstraction is related to its functioning as a
lure for mathematical thought-it lures mathematicians into adventures which produce new
aspects of what it means to be a circle into a mathematical mode of existence. IS

And farther on:

Such is the power of what Whitehead called propositions, luring abstractions which are not to
be confused with sentences (which eventually serve as their vehicles), that their acceptance into
Why This Book? 17

experience may disrupt social order. "When a non-conformal proposition is admitted into feeling
. . . a novelty has emerged into creation. The novelty may promote or destroy order; it may be
good or bad. But it is new, a new type of individual, and not merely a new intensity of individual
feeling. "16

In these passages, Stengers has described an essential feature of the pleasure and
consolation of doing mathematics, which is the constant reaching via a rigorous
imaginary beyond the actual, and beyond matters of fact. Mathematics hitched to
utility can be as rigid and asphyxiating as any schema. But mathematics, as Stengers
recognized in the practice of mathematicians in their own terms, is indeed a performa­
tive art, and it is in this poetic and poietic mode that I will articulate some of my
arguments and expressions by adapting the concepts and theorems of topology, dif­
ferential geometry, Lie theory, and dynamical systems.
One final qualification may bear repetition throughout this book. I do not use
mathematics for instrumental purposes, e.g., to measure obj ects or to model some
phenomenon. Nor do I aim to construct a philosophy of mathematics or physics,
making judgments about, say, the metaphysical status of mathematical objects, or a
theory of agency explaining mathematicians' discursive agency. Nor do I intend to
mine mathematics for metaphors, e.g., using fractal geometry to stand in for nature
or art. My interest lies in seeing how certain mathematical concepts can inform philo­
sophical insights. Antecedents adopting a similar approach to mathematics include A.
N. Whitehead, Rene Thom, Gilles Deleuze, and Alain Badiou, with a fountain of
diverse results. However, standing more with Stengers than with Badiou, being well
aware of mathematics's coherent power, I would urge us to go slowly, thinking that
we may not have concepts adequate to the phenomena and to our concern. As
Stengers wrote in her essay "Beyond Conversation: The Risks of Peace" :
A s Deleuze said, t o think (or create) i s t o think " i n front of" o r "for" analphabets, dying away
rats or alcoholics . Which does not mean addressing them, or helping them, or sharing hope or
faith with them, but not insulting them with our power to justify everything. Thinking with
them in front of us means thinking with the feeling and constraint that we are not free to speak
in their name or side with them . . . . What would be a conversation "in front" of all the unknown
people our words so easily disqualify as a matter of fact, even when those words speak, of mutual
appreciation, respect and love? Deleuzian tradition, with its built-in decision to side with the
damned, may help process people to "stammer, " or "quake" when trying to produce the words
for a sorely needed "relational worldview. "1 7

Stengers's caution for those who would "produce words" holds equally for those of us
who would tap mathematics for philosophical or artistic inquiry.
Some people say that ideas are cheap, that making is hard. But we know very well
that humans create and rework concepts with just as much effort and rigor and mate­
rial discipline as the making of a physical installation. It's just that the young domain
18 C h a pter 1

of media arts and sciences has not enjoyed the lUxury of alloying and working out
concepts as thoroughly as, say, biotechnology or Renaissance literary history. Domains
of practice that benefit from billions of dollars or centuries of investment develop
practices that exploit the making and composition of concepts based on antecedent
literatures, intricate dependencies and interrelationships of publication and citation,
the social networks that give meaning to concepts, and procedures of evidence and
argument and generative logics indigenous to the epistemic culture.18
2 From Technologies of Representation to Technologies of

Performance

This chapter shifts perspectives from media technologies of representation to tech­

nologies of performance. It revolves around the h�nge of notation, initially regarded
as representation, then as a part of the means of articulating performance. The larger
shift in perspective, intertwined with the introduction of computational technologies,
moves from the design of notation, to the design of instruments, to the design of
responsive media.

Two Motivating Stories

In 1 922 Laszlo Moholy-Nagy wrote:

So far it has been the job of the phonograph to reproduce already existing acoustic phenomena.
The tonal oscillations to be reproduced were incised on a wax plate by means of a needle and
then retranslated into sound. An extension of this apparatus for productive purposes could be
achieved as follows: the grooves are incised by human agency into the wax plate, without any
external mechanical means, which then produce sound effects which would signify-without
new instruments and without an orchestra-a fundamental innovation in sound production (of
new, hitherto unknown sounds and tonal relations) both in composition and in musical
performance.
I have suggested to change the gramophone from a reproductive instrument to a productive
one, so that on a record without prior acoustic information, the acoustic phenomen ltitself origi­
nates by engraving the necessary etched grooves. 1

One could question the potential effectiveness of this technique-why go to all that
effort of impressing a pattern to be produced by electromechanical means, when one
could use perhaps more expressive or precise musical controllers, for example a j oy­
stick or hand waving over some antennae? Nevertheless, Moholy-Nagy anticipated
turntablists' reuse of the gramophone, imagining an even more radical hack. Whereas
turntablists essentially work with macroscopic samples, Moholy-Nagy's proposal
20 C h a pter 2

would have the performer create sound from the far more primordial level of manipu­
lated wax substrate.
***

When the creators of the TGarden came together for a two-week workshop in the
Banff New Media Institute to come up with a design for the responsive play space, we
organized a series of very different modes of ideation reflecting the radically different
modes of practice represented by the eight members of the creative team. A musician
with twenty years of experience in electronic music and live improvisation with live
musical and dance performance demonstrated examples of both sampled and synthe­
sized sound. A PhD in computer science gave a series of talks about the state of the
art in particle systems and computer graphics. An experimental theater artist who had
formed three companies pioneering performances blending everyday and prepared
sites, incorporating video personae with live actors, led some narrative exercises based
on imaginary scenarios and language games. Some of the artists also brought samples
of visual and written work from scrapbooks that they had been keeping for the months
prior to this retreat. Some of the group leaders believed that our goal was to come up
with a common multimodal language in which the next version of the TGarden could
be described and designed. I didn't share this belief. After Wittgenstein's demolition
of positivist theories of language as representation, after all the experiments with
artificial languages from Leibniz to Orwell, one might better expect that a common
language would evolve in the crucible of the making of an event. And moreover, an
" accurate" formal representation of the structure of an event and the modes by which
it was created would be possible only after the event was created.
One might believe that a graphical interface would be more universal, but Witt­
genstein's infamous arrow, duck-rabbit, and other graphical examples show us that this
is no less chimeric than believing in a universal or even a transversal (verbal) language.
This is not to discount all the elaborate technologies for making images and sounds
and texts and things that we have developed. What I propose in this chapter is simply
to shift how we regard these technologies, to see how they can be used not to represent
facts or knowledge but instead to create events . In short, I propose to shift the perspec­
tive from representation to performance. By technologies of representation I mean those
technologies designed for creating media that are later perceived by a spectator in an
edited form that does not vary according to what the spectator or environment is
doing during the playback of the recorded media, whereas by technologies of perfor­
mance I mean technologies that vary media by design according to contingent condi­
tions and activity.
This chapter is by no means a survey or critique of representation. It is not a his­
toriographically systematic bit of archival research on the history of musical technolo­
gies and notation. My purpose is to reorient our view of media technologies, especially
From Technologies of Representatio n to Technologies of Performance 21

computational media technologies, toward their performative, expressive, improvisa­

tory uses. The concrete performative and improvisatory uses will become clearer in
the examples of responsive environments that I describe at the end of this chapter,
and in the chapter on performance.
Briefly, the reorientations in this chapter are: ( 1 ) from technologies of representation
to technologies of performance; (2) from the point of view of the composer or critic to
that of the performer; (3) from an instrument model of sound synthesis to playing
sounding, responsive media; (4) from distributed agency (the ensemble) to plenist
magma, material medium, stuff.

N otation

Given the emphasis on computational media, it makes sense to consider how comput­
ers are used as the quintessential symbolic processing machines. It follows that it's
reasonable to consider the medium of signs. The big shift in perspective is to see how
the signs constitute things in a nondualist or, to anticipate Whitehead, unbifurcated
ontology, rather than represent (point to, refer to, correspond with) some discon­
nected part of the world. (How the sign is disconnected from its worldly referent
depends on the analytic conceits in force in the situation.) This perspective may prove
useful for understanding how we make musiC, dance, as well as mathematics, all
regarded as equally performative and creative modes of human expression, even
though they create radically distinct kinds of things. Grammar and notation are tech­
nologies for structuring symbolic media as much as brush and canvas structure paint­
ing. (By structure, I mean nothing deterministic but not random either, more a matter
of conditioniug.)
The more profound difference, from a semiotic point of view, is what constitutes
an interpretant of the sign. As many have observed, from Turing and Wiener to Fried­
rich Kittler and Katherine Hayles, computers foreground the possibility of machinic
interpretations of these signs in tandem with human interpretation. I will tell the
story from the point of view of the " symbol machines" that play a role in the genera­
tion of performative events. (For the purposes of this chapter, events are � anges of
material state, and performative events are events in which affective as wen as sym­
bolic transformations play a leading role in those changes of state.)
At the end, we will have in hand some theoretical takeaways and concepts . We
will consider a notational system's expressive power, which depends on several factors­
resolution, syntactic denSity, completeness, range, nuance, and connotative potential.
Following the tactic of starting outside the domain of the computational in order
to get some parallax on the computational, we will consider technologies associated
with two areas of performance: ( 1 ) musical performance and sound improvisation, (2)
dance and movement. In each domain of practice, I will trace the move over recent
22 C h a pter 2

decades from conventional notation to computer technologies. We will trace the

transition scaffolded by advances in computational technologies from offline model­
ing (special effects) to real-time synthesis and modulation of media. Then I will make
some observations transversal to those domains of practice about the computational
technologies of performance.

M usical Performance
What does a musical score enable? What parties are expected to use it to coordinate
thoughts and actions-composers, instrumentalists, theorists? Typically, the audience
is not expected to read a score.
What does the seemingly universal musical notation for European art music do?
For the purposes of this section I will intend twentieth- and twenty-first-century nota­
tion as used in European art music, referring to the contemporary conventional system

RITMO 0 1 QUATTRO BATTUTE areo

I �. ..
Vlolons I
� semprepp
pp
f! areo
" ... .
Violons I I
III
pp semp'repp
arco
... -� ...
Altos
pp S
arco
t:� l� ��-- b1'- �1!= �1!= �
Violoncelles :
� 1!=
? �t:� � �.. .-

pp
arco ;F;
semprepp
I I. .
Contrebasses :

v pp sempre
[1 2 3 4 2 3 4 2 . . .J

F i g u re 2 . 1
A standard classical musical score: Ludwig van Beethoven, Symphony no. 9, second movement.
(Source: http: //commons. wikimedia.org/wiki/File:Beeth_2_Th_strette. jpg.)

II f:\

v
o
I" Ie
ff
F i g u re 2.2
The final bars of Richard Strauss, Don Juan, op. 20, 1 889, notated as whole notes, but played as
quarter notes with rests in the player's interpretation by percussionist Anthony J. Cirone.
From Technologies of Representatio n to Technologies of Performance 23

of written musical representation inherited from seventeenth-century Europe: five

horizontal lines-the "staff"; a "time signature" indicating the basic durational unit
and how many units make up each minimum formal segment-called a "bar"; a "clef"
indicating a general pitch range; a "key" indicating the scale of pitches to be standardly
used; and a vocabulary of terms that have acquired canonical meanings for perfor­
mance practice, such as " sforzando, " "morendo, " and "G.P. "
The key difference between a literary text and a musical text-a score-is that the
latter is written by the musical artist-the composer-for an intermediate interpretant­
the musician-who in turn produces the sound to be experienced by the audience. So
the musical event conventionally bounded as a " concert" is coproduced by a complex
of the score, musician, instrument, audience, and physical site. I'm deliberately con­
fining this description of the production of a musical event to obvious, tangible ele­
ments for the purposes of this chapter. Ignoring the enormous components of
performance practice, publication and dissemination systems, and sociological and
phenomenological modes of experiencing music, I focus on the technologies of per­
formance most directly bound up as musical notation and musical instrument.
Adj oining the systems of signifiers coevolving with SOO years of instrument making
and schooled performance practices reflexively refined around the interpretation of
musical text (the classical musical score), we can see musical notation as part of a
highly refined technology of performance.
But none of these aSSOciations, however well sedimented by custom and history of
compositional and interpretive practices, are uniquely determined.
Musical notation only partially registers the acoustic Signal, and in fact, before the
age of mechanically recorded sound, there was always an aspect of the imaginary
implied by writing music. There are in fact uncountably many interpretations possible
for a given score.

Contemporary M u sical N otation in (European) Art M usic

Broadly speaking, notation can represent:
1 . The corporeal, kinesic manipulations relative to a physical instrumept (in the
I,
context of computer music technology, a controller); .
2. The sound (e.g., acoustic: pitch, loudness, tempo descriptions like "andante");
3 . Conceptual, formal structure, metaphor.

N otating Kinesics
Examples of notations of corporeal manipulation range from very common tablature
diagrams for guitar and lute that show where to place the fingers on the fretted fin­
gerboard to one-of-a-kind diagrams such as Mauricio Kagel's drawings of how to place
the hands on a balloon. Kagel's score of Acustica is a pictographic depiction of how
24 C h a pter 2

Mit einem, zwei oder mehreren Fingern

@ der gleichen Hand tiings des BalloDs reiben.
LUFTBALLON

BaUon mit einer Hand drehen uod mit

® der anderen leicht beriihren .

© Beide Hande altemierend: Ballon leicht

driicken uod gleichzeitig etwas drehcn.

@) ® und © in beliebiger Reihenfolge ausfUhren

'" Ad lib,; wCibrend dieser Aktionen rubt der Luftballon auf einem
FelJinstrument (z. B,
Grope Trammel).

F i g u re 2 . 3
Mauricio Kagel, Acustica.

to manipulate an obj ect. But Kagel's diagrams are compound; he also incorporates
representation of the desired acoustic effect to be achieved-a graph showing the
aggregate "intensity" of manipulation over time.2 Of course this sort of representation
is not restricted to European art music. The most notable treatises by Arab theorists
(al-Kindi) established the principal Arab non-Pythagorean tone system by showing
how to finger the tones on the ud (lute) .
Andrea Valle writes, "Musical notation makes body . . . most substantially, with
which one intends by music: in this study it truly is the case that knowledge is sedi­
mented in notation."3 For example: Anthony ] . Cirone, a distinguished member of
the San Francisco Symphony Orchestra, published a set of notes for fellow percussion­
ists about how to interpret the less precise indications for percussion instrumentalists
in European art music of classical and postclassical eras. About performing Richard
Strauss's Don Juan, Cirone advises the percussionist: ''It is important to muffle . . . by
using both arms to silence the instrument. " 4 For playing Mahler's Symphony no. 2,
he provides such detailed comments on kinesthetic aspects of the interpretation of
the score as: in bar 9, "Stroke edge of cymbal following, " and in bar 1 3, to add a bit
of drama, lift cymbal in view of the conductor.s
From Tech nologies of Representatio n to Tech nologies of Performance 2S

N otating Sound
The second mode of musical notion describes the acoustic pattern-the sound-to be
produced. Cirone remarks that scores for percussion often lack notation for phrasing,
whereas for other instruments phrasing is represented by an arc extended over all the
notes to be played as if they were sung in one breath. Even though percussion instru­
ments are not played by breath, they can certainly be played with nuance that matches
the variations in connectedness and fluidity that accompany the phrases of a piece.
So adding such arcs even to keyboard-like representations introduces, suggests, reflects
shape added by phrasing.6 The obvious pOint here is that musical written notation
can never completely describe the j oint musical intention of the composer and the
performer. In fact, as Valle also remarks, any (synchronic) semiotic system of repre­
sentation radically undercodes the corporeal actions to be done by the performer in
order to create the scored music in a way that would be accepted by the complex of
composer, performer, and audience within the unron of their musical frame. But as
Valle observes with abundant examples for twentieth-century art music, notation also
radically overcodes performance as well.
Let us start with relatively straightforward elaborations of conventional scores, for
example bars 27-32 from Gyorgy Ligeti's second string quartet ( 1 968) / Ligeti chooses
to augment the conventions of musical notation of the day with very detailed notes
to the string instrumentalists about how to interpret the notation (" [graphic marking]
always applies to the pitch not the finger"), the physical movements (sempre suI tasto­
always near the fingerboard, a nonconventional area on which to bow), and the
desired musical effect (leggierissimo) .

�
- -@ ------@ - - - - - - - - - - - - - - - - - -@- --- - -@ - - - - - - - - - - - - -- - - - - - - - - - -�. L
•• � ...u,_ @+@+@
- ... - ... � ... - - - - ... � - - - - - - - - - - - - - - ... � .. - - - - ... � .. - - - - - - - - - - ... ,:.. ... - - rn... ,;.J �¥� ,��
- -
'" = = += L!J += � l1= ..

J. ! n�
sempre ppp

- -
- - - - - - - - - - -e- - _ _ _ _
\
_ _ _ _ _
_ -

- t�"'" - - - - - ."'" - - - _ _ _ _ _ _ _ _ _ - - - #-0 - - _ _ _ �-o - _ _ _ _ _ _ _ _ _ _ _ _ Ito - - - _ _ _ _ _ _ - - -

Uggi.riss;mo, {.
fi1 su:.taSto i'i (�;)
E

...) 8 " " ' gill sletsjUr die TMlMhe, nichtfor de.. tIriff. ""'lIre ppp
�..J.-�'I" , .. �
8 " " ' altua!fs applies 10 til.pil<h, Hut to tIt'lingering.

Figure 2.4
Gyorgy Ligeti, String Quartet no. 2, bars 27-32.
26 C h a pter 2

But his organ work Volumina8 has a much more unconventional score-in fact a
one-of-a-kind notation, similar to a spectrograph-a display of frequencies present in
the spectrum at a given point in time. Ligeti instructs the organist to press fingers,
but also elbows and whole forearms onto the keys in order to create masses of sounds­
tone clusters and tone masses, which suggests a set-theoretic conception of sound.
In any case, such scores presume a classical (pre-quantum mechanical) relation
between the composer-performer and the musical work: there is the presumption of
a definitive entity-the musical work-the text as conceived by Ligeti and "realized"
by the properly prepared organ and properly prepared organist.

Notating M usical Event Structure

Notation can also prescribe, in the most flexibly abstract way, musical event structures .
One example is the notation o f a key, which selects a set o f pitches that will actually
occur in the sound and sequences of sounds to be produced. Another example might
be paratextual glosses like "Sonata" or "Overture. " ("Text" in this chapter refers not
to ordinary-language words but to the musical score.)
As Valle put it, "A notation, like every sign, is a crystallization of knowledge not
reducible solely to predispositions of the signifier [segnalel , and consequently it is
impossible to speak of notation without entering into aspects of compositional tech­
,,
nique. 9 Ligeti's Volumina shows how a notational scheme can fuse all these modes of
articulation at once. Of course with contemporary computational means we can syn­
thesize the sonic clusters, but it makes a tremendous difference that the performance
is done by a human body, which conditions the production and the perceived quali­
ties of the sound in a radically different phenomenal way. Stravinsky relied on a similar
performative tension when he scored the Rite of Spring s opening melody to be played
'

by a bassoon far above its customary range, precisely in order to hear the strain of being
outside the normal range of performance practice of professional bassoonists of the
day. There are uses in performance for performing at the limits of the capacity of
the technical ensemble (to use Simondon's terminology for the complex of technical
object and technical individual, discussed below) .
Krzysztof Penderecki's score for Threnody to the Victims of Hiroshima shows how the
24 cellos are grouped into clusters of voices, and also when they enter staggered across
time to build the mass of sound. 1o
Issam EI-Mallah's diagram depicting the formal structure of the sotll goes to one
extreme of being primarily a tool for analYSiS, not a part of the instrumental apparatus
for performers to produce a musical event. This is an important distinction when we
consider a notational system for dance.
Myron Levine's score for Parentheses is a diagram in the form of a matrix, with
arrows showing possible transitions to neighboring cells in the matrix. We can easily
interpret this as a graphical program in which each cell is a possible state of the musical
From Technologies of Representation to Tech nologies of P e rformance 27

� � J � ""f � ,; Edition Pefers No. 5983

: '� 'f: /I' .: 1 )l to Copyright 1967 Henry UtoUFs Verla.g. Frankfurt.
...
r _ _ _ .... _ _ _ _ _ �_X_t_':::=-�� Riprodotro su aurorizuWone di Petcr.s Edition Limitoo, London.

�
•
·
•
<
!t
<

® .;
�
f
�
...

";
�

�
"
�
'"
"
'\\"
•
'.
1::
�

�
.:!
...
""

� •
�
X
�

�"
•

,(
'"
'i
....

\
:; -oS
j. �
'"
� -:r:;

F i g u re 2,5
Gyorgy Ligeti, Volumina for Organ, tone cluster strips for right and left hands and pedal.
28 C h a pter 2

© Copyright PWp, Warszawa

lOVe

F i g u re 2.6
Krzysztof Penderecki, Threnody to the Victims of Hiroshima, for 24 cellos.
F rom Technologies of Representation to Technologies of Performance 29

event (its contents indicating orchestration and voicing, and so forth), with adj acent
cells as the possible successor states and arrows indicating transitions. So it is a paper
version of a piece of code, except that this is meant to be read by a human performer
rather than a computer.

N otating Metaevents
John Cage gave up a deterministic coupling to corporeal production and the resulting
sound. One could, with Valle, think of "notation as operation, "12 but with Cage and
subsequent composers the operation works more at the level of musical event as
cybernetic system. Notation serves more as a metascore, a set of instructions to the
performer on how to arrange the elements provided in order to assemble a score that
can then be interpreted. For example, in Fontana Mix the musician is told to slide a
graph-ruled transparency across a printed paper, and then to read out the resulting
"line" constituted by the superposition of the grid over the printed drawing as a nota­
tion to be interpreted for playing musical instruments.13
Most radically, Cage gave up the presumption of a stable entity called the musical
work that predates the moment of performance. A critic like Genette may claim that
Cage's strategy was a historical anomaly, and that art obj ects are largely determinate
or become determinate. But as Guattari pointed out, borrowing from Lacan's objet petit
a , 14 every art object enjoys this quality of being only a partial object, its radical inde­
terminacy underscored by the arbitrary and boundless ways in which it may be "com­
pleted" contingently in every encounter with an observer. (In this chapter, we pursue
this to its material limit of what I will call substrate.)
Just as with Levine's Parentheses, Cage's metainstruction is another "analog" analog
to a set of instructions-a program-that instructs its reader to execute a set of opera­
tions according to the potential logic encoded.
This anticipates the transition from human-legible scores to be interpreted by
human performers to machine-readable scripts or code (software) to be executed by
electromechanical machines. Looking ahead, I will multiply the notion of score into
code versus script, code being more allied with numerical and logical instructions to
instruments, and script more allied to "semantic" instructions for playing instruments.
Alongside this I will transition from thinking scores as representing some 'entity to
the instrumental performance, instruments, apparatuses, environments, and media in
which these performances are constituted.
To gain some parallax on this, let us look to a neighboring domain of movement
and time-based expression: dance.

Movement Arts, Dance: Rudolf Laban and D a n ce as an Art

Why do dancers not use a written notational system such as Laban notation whereas
European classical musicians do? Of course, contingent histories play a major role in
 30

Ch ap te r
2

'.
•••• •
... .. .. :
.�......

. \.

. . ...

Edition Peters N
o. 671 2,
Riprodo
tto su au ©
o,Pyright 1
toriz
zazione di 960 Ben
Pete m.,. Press
rs Editio In c., New
n Limite
Fig ure d, Londo York.
2. 7 n.
Jo h n Cag
e, Fonta
na Mix .
 STUDY IN TOUCH AND SLIDE
Suggested music : An Italian tarantella .

4 t!
..... <> () .t!
...
�'--)(a "
� �
\)(.-/

3 8 8

<>

2 7 7

1
I
6 6 '*-
9
0

7_+-'-i-r-+-- 4

5'_�+-r-+-- 2 2

3'_+-r-�+--

5 _--,-__,,-,-_

F i g u re 2.8
Laban notation : touch and slide study, from Ann Hutchinson Guest, Dance Notation (1 984), 143.
32 C h a pter 2

the adoption or nonadoption of a written system of notation to represent some social

practice. Hungarian-born Rudolf Laban ( 1 8 79-1958) created a notational system in
order to provide a way not only to record but to compose dance with as much reflec­
tion and citationality as literary forms. The main purpose of what Laban called kinetog­
raphy is "making the movement event precise through analysis and freeing it from
being blurred, which makes a language of dance appear on the one hand unclear and
on the other monotonous. " I S Although it is true that Laban studied crystal symmetries
for formal purposes of understanding spatial rhythm and space harmony, it would be
inaccurate to describe his notation as a description of positions in space relative to
the center of gravity of the body. In fact, for Laban movement was primary. One can
see this in his sketches of a dancer's movement-he attached labels not to positions
of body parts, but to traj ectories of movements in the sequence in which they occur,
a thoroughly dynamical attitude. Laban wrote: "The conventional idea of space as a
phenomenon which can be separated from time and force and from expression, is
completely erroneous . . . . Movement is the life of space. Dead space does not exist
for there is neither space without movement, nor movement without space . " 1 6
We often mistake a thinker's scaffolding for h i s o r her conclusion, o r pedagogical
rhetoric for actual practice. Christopher Alexander's approach and practice cannot be
reduced to his set of "patterns" without entirely desiccating the approach and turning
it into its deadly opposite. Likewise, Laban's approach cannot be reduced simply to
his system of notation. A common move by technicians is to reduce Alexander's or
Laban's compositional approaches to a fixed schema or typology and try to build entire
worlds by combinations of a set of primitive elements. In the illustrations and frag­
ments of texts published from his notebooks by one of Laban's students, we can see
how he started from the full richness of bodily movement and moved in a double
spiral toward geometrical notation. One branch of this spiral was progressively codi­
fied movement, and the other a progressively refined system of signs. I ?
Laban was sensitive to the dynamical and full richness but availed himself only of
piecewise linear geometry. So, for example, on page 3 1 of the sketchbook, he started
with a fully drawn body and placed it inside a bounding polyhedron. But even in this
Euclidean cage, the body was drawn in the middle of a movement rather than some
straightened, symmetrical profile. And the vertices are formed out of the extremities
so that the bounding shape is clearly changing all the time in the course of the move­
ment. Laban also built from simple lines to more complete movement-skeletons: from
1 -simplexes l 8 to ribbons centered on such complexes of lines l 9 to polyhedra.20 But to
sketch the full movement, he also drew the dynamics in a much more evocative way,
such as in one figure with the swirls traced by a body in motion.21
The actual syntax derives from multiple sources in European arts ranging from
musical staff notation to solid geometry, streamlined over thirty years into a dense,
compact notational scheme. In 1 928, Laban laid out the ground principles for his
From Technologies of Representatio n to Technolog ies of Performance 33

notational scheme (figure 2.9) . The basic durations are denoted by length. (Laban
credited the woman who collaborated on his project for this innovation from his
original token-based representation.) The final form of the notation was oriented verti­
cally to map more naturally the bilateral symmetry of the upright body. There is an
important point to the vertical, bottom-to-top syntax for writing and reading a line
of Laban notation. Together with the mapping of left-hand signs to left parts of the body,
and likewise with the right, the vertical orientation of the syntax matches the embodied
perspective of the performer. Here is a notation system designed not for the convenience
of the European typesetter or literary reader, but for the dancer. The photograph of a
dancer walking in line with the score on the floor (figure 2 . 1 0), along with the
sequence of examples from Ann Hutchinson's definitive text, illustrate this point.22
Laban himself was not interested in propagating his approach as a methodical
system, but rather as a reorientation, a modernist awakening of sensitivity to corporeal
kinetic movement in human life generally: in science, art, spirituality, and health. For
our purposes, the dance notation, stripped of the obvious and obscuring indirection
through mechanical instruments, is a particularly clear example of a notation constituting

- - -

i
__ �I
•
__ :r---- _---....--
.
.....

F i g u re 2.9
Rudolf Laban, Ground Principles.
34 C h a pter 2

F i g u re 2. 1 0
Walking a prototype notated floor, from Rudolf Laban, Schrifttanz, vol. 2.

a technology of performance as I have characterized it in this chapter, the medium

being the movement of human bodies in an event.
Of course, in the past half century, and especially in the past twenty years, move­
ment artists have begun to augment the moving body with electromechanical instru­
ments, and to augment scenography and costume with computational media-video
and sound. Notable examples include the work by the media group Riverbed (Paul
Kaiser, Shelley Eshkar, Michael Girard) with Merce Cunningham's Biped, and Robert
Wechsler's work with Palindrome in Germany.23
Let us turn to the computational present.

Computational Media: Sound and Video

Furnished with these examples, we can think of notation as a part of a technical
ensemble-to use Simondon's well-developed framework24-for modulating kinesics,
sound, and structures of sound in a musical event. In this context, the technical ensemble
comprises technical objects-musical instruments, figures of musical notation-and
technical individuals-trained musicians . The composer, the technical objects, and tech-
From Technologies of Representation to Tech nologies of Performance 35

nical individuals are informed by the score. Simondon's notion of informing is a pro­
found amalgam of theoretical knowledge, corporeal performance practice, and matter flow.
Notation suggests both retentively and protentively how memory and anticipation
shape the musician to her sounding instrument, and vice versa. One of Simondon's large
challenges to prepare for a technologically informed humanism was to invent what I
would call modes of articulation interrelating the evolution of humans and machines.
Musical notation serves as one of the richest examples of such intercalating modes in
the contemporary age of computationally mediated sound and musical performance.
Computational techniques for recording, processing, and synthesizing sound make
it possible for musicians to vary from analog concepts of record and playback, and
put in play such categories as score, performer, spectator, sound, composer, instru­
ment. What the computational adds to the paper-based machines of musical notation
are the prolongation of gesture, pluralizing of agency, blending of instrument design
with composition, and blending of composition with performing, via making code.
Decisions may be deferred in any number of ways, by simple delays, by Boolean logics
that depend on contingent conditions to be triggered, by parameterizing the resyn­
thesis of sound depending on continuously changing conditions during the event, by
calling on random processes, and so forth. Even in the case of appealing to chance
operations, the computer can help flexibly and even programmatically control the
degree and timbre of indeterminacy.2s
Prolongation of gesture means deferring action by computer-controlled electrome­
chanical means, using for example the mechanisms of indirecting or deferring action
by: ( 1 ) Boolean IF <conditions> THEN do <action>; (2) table look-ups (tax tables, MIDI,
scales); and (3) computable functions (compare figuring out the taxes you owe by
looking it up in a table with precomputed values versus using a formula) .
One lens through which we can view this is the transmutation from the role of
notation as (part of) a machine for modulating sound to the ways that score, script,
and code co articulate practices in the technical ensemble that produces computational
organized sound (see figure 2. 1 1) .
The impetus i n the mid-twentieth century toward "preparing" conventional instru­
ments in European art music has extended in the present epoch of computational
software-hardware to electromechanical instruments that are so continuaIly repro­
grammed as to be essentially unique for each performance. On one hand this extends
the compositional control into the very capacities of the instruments, but on the other
it severely limits the capacity of the human performer to develop what would consti­
tute musical virtuosity in pre-electronic instruments whose playing characteristics did
not change appreciably over the period of an individual musician's career.
So, in place of the musical score that intercalates the composer's actions with the
performer's, we have at least three significant intercalating semiotic systems: the score,
the code written by a software programmer-engineer to activate the electronic instrument,
36 C h a pter 2

Com poser

�
h u m a n readable score
" m usical n otati o n "
/'

/
Performer ---. physical i nstru ment

/ "
c o m po pco g " \,'d " , r dwa re E n g i neer

� �
h m a n � rea d a b l e sc re mach .i n e-read a b l e
'

,e lectro\
m u sical n otation .
scri pt

" Performer n i c i nstru ment

I
sou n d p rod uction devices

F i g u re 2 . 1 1
Coordinating relations among musical agents.

and the machine-readable script written by a composer to parameterize the execution

of the electronic instrument, which may also be manipulated by a live performer in
the musical event.
This multiplication of types of notation serving to mediate human as well as
machine processes calls for a more careful look at how these semiotic systems work,
before we continue our reorientation away from what Karen Barad calls representation­
alism to the performative. The other reorientation that I will suggest later in this
chapter, and in chapter 3, is from instruments (and other objects of musical production)
to medium.

Theoretical I nterlude: The Problem with Representationalism

Why did Laban's and his collaborators' thirty years of titanic effort not yield a more
universal language for movement and dance, a written form in which choreographic
art could develop? Certainly historically profound changes in the diffusion of new
movement forms, new performing arts industries, new mass media industries displac­
ing live arts, new recording technologies, the diversity of ways in which performers
learned to dance and transmit dance, and so forth all played a role in making it practi­
cally impossible to induce a universal written notation for movement and dance. But
perhaps also there is the intricate fusion of body, movement, and thought that is only
now being split apart and infiltrated by computational technologies, along with the
mercurial evolution of ever-changing, ever-unfolding practice that escapes every time-
From Tech nologies of Representatio n to Tech nologies of P e rformance 37

invariant system of representation. In this last respect, notating dance is no more and
no less impossible than fixing the written representation of all human speech prac­
tices. This points to much deeper problems with any sort of representation, whether
of movement or of other living or social activity. For this we draw summarily from
Wittgenstein's Philosophical Investigations.

Wittgenstein against Positivist Representationalist N otions of language

What does musical notation represent? Nothing in itself, the late Wittgenstein
would say.
The basic claim is that a time-invariant system of representation cannot be formally
specified and adequate to any living phenomenon of languaging. By a series of care­
fully, devilishly constructed examples in the Philosophical Investigations, Wittgenstein
exploded whole genres of theories of language and meaning, including those based
on correspondence, rule, formal or axiomatic method, and prototype. This is not the
place to rehearse the critiques of representation and representationalism that have
occupied modernist art and philosophy for at least a century. But for efficiency, rather
than endlessly debate what one might call, after Wittgenstein, "ungrammatical ques­
tions" let me just summarize a few useful observations from his interrogation of how
we think about language and representation.
The first of these is that correspondence is a poor anchor for meaning. One
common way to define a term, one might think, is to point to it. But not every refer­
ent can be the target of such action. Wittgenstein gives an apparently Carrollian
example, with a serious consequence:
I say "There is a chair. " What if I go up to it, meaning to fetch it, and it suddenly disappears
from sight?-"So it wasn't a chair, but some kind of illusion. "-But in a few moments we see it
again and are able to touch it and so on.-"So the chair was there after all and its disappearance
was some kind of illusion. "-But suppose that after a time it disappears again-or seems to disap­
pear. What are we to say now? Have you rules ready for such cases-rules saying whether one
may use the word "chair" to include this kind of thing? But do we miss them when we use the
word "chair"; and are we to say that we do not really attach any meaning to this word, because
we are not equipped with rules for every possible application of it?26

One might say that meaning can be assigned by some rule. But there are many
problems with this hypothesis. Even the apparently simple task of teaching someone
to write down a series of numbers by mimicry is not unambiguous. A pupil can be
tested to interpret a rule as the teacher would for, say, the first 1,000 integers, then
interpret it differently beyond 1 ,000, yet come up with a rule that agrees with the
teacher's interpretation for integers below 1 , 000.
No course of action could be determined by a rule, because every course of action can be made
out to accord with the rule . . . . If everything can be made out to accord with the rule, then it
can also be made out to conflict with it. And so there would be neither accord nor conflict here.27
38 C h a pter 2

One might argue that rules describe an ideal situation and perfect form of imperfect
practice. But we could take the practical response of the ontological principle, that we
start from what there is, not from ideal types:
[I] n philosophy we often compare the use of words with games and calculi which have fixed
rules, but cannot say that someone who is using language must be playing such a game.-But if
you say that our languages only approximate to such calculi you are standing on the very brink
of a misunderstanding. For then it may look as if what we were talking about were an ideal
language. As if our logic were, so to speak, a logic for a vacuum.-Whereas logic does not treat
of language-or of thought-in the sense in which a natural science treats of a natural phenom­
enon, and the most that can be said is that we construct ideal languages. But here the word
" ideal" is liable to mislead, for it sounds as if these languages were better, more perfect, than our
everyday language; and as if it took the logician to shew people at last what a proper sentence
looked like.28

The key point is that the interpretation of rules is conventional.

Is what we call "obeying a rule" something that it would be possible for only one man to do,
and to do only once in his life?-This is of course a note on the grammar of the expression "to
obey a rule. " It is not possible that there should have been only one occasion on which someone
obeyed a rule. It is not possible that there should have been only one occasion on which a report
was made, an order given or understood; and so on.-To obey a rule, to make a report, to give
an order, to play a game of chess, are customs (uses, institutions) .29

Another family of methodological problems goes under several names, such as

reification error or Whitehead's fallacy of misplaced concreteness. Just because your theory
has a name for something does not mean that something exists as a coherent thing
or concept, not to speak of coherence or adequacy for your theory.
There are no absolute "primitives. " Wittgenstein's famous example for this is the
sole colored diagram in the book (see figure 2 . 1 2) . And so for instance the sentence
" RRBGGGRWW" describes an arrangement of this sort:
Here the sentence is a complex of names, to which corresponds a complex of elements. The
primary elements are the coloured squares. " But are these simple?"-I do not know what else
you would have me call "the simples," what would be more natural in this language-game.
But under other circumstances I should call a monochrome square "composite, " consisting
perhaps of two rectangles, or of the elements colour and shape. But the concept of complexity
might also be so extended that a smaller area was said to be 'composed' of a greater area and
another one subtracted from it. Compare the 'composition of forces', the 'division' of a line by
a point outside it; these expressions shew that we are sometimes even inclined to conceive the
smaller as the result of a composition of greater parts, and the greater as the result of a division
of the smaller. But I do not know whether to say that the figure described by our sentence consists
of four or of nine elements! Well, does the sentence consist of four letters or of nine?-And which
are its elements, the types of letter, or the letters? Does it matter which we say, so long as we
avoid misunderstandings in any particular case?30
From Technologies of Representation to Technologies of P e rformance 39

F i g u re 2. 1 2
Diagram by author of colored squares (R = red, B = black, G = green, W = white), based on Witt­
genstein, Philosophical Investigations, §48.

This raises questions about Genette's claims to the objective existence of stock phrases
and building blocks for improvisation.
Perhaps the most fundamental contribution of the Philosophical Investigations is to
deflate the positivist project of looking for meaning encoded in a sign (raising insu­
perable problems of taxonomy, context, and interpretation), and looking instead
to how people language with one another in practice .31 In brief, meaning comes
from use.
Valle recognizes linguistic and nonlinguistic conceptions of notation, in which the
linguistic frame, characterized at a most fundamental level, is the unidimensional
sequencing of phonological units.32 Text, regarded as a transcription of speech, inherits
such a syntactic structure, but it need not do SO.33
Many dance notations use linguistic models, even though dance is far from ame­
nable to a reduction to a unidimensional continuous index (like clock time) . Rudolf
Laban rejected the necessity of aligning dance with mUSiC, with verbal linguistic
pattern, and in fact with any reduction to a unidimensional structure. Yet the dance
scores published in Schrifttanz are all indexed to musical lines.
For more than a decade, computers gathering dozens and hundreds of sensor chan­
nels have been generating "motion capture" data as humanly unreadable scripts of
movement.34 It is only more recently that performers have begun to use the same
data-collecting apparatus but map the sensor data through to sound in real time so it
can be modulated in concert with the movement of the performer. This transmutes
the technology to a technology of performance.35
After this detour into the inadequacy of approaching notation as a technology of
representation, can we seek certainty by appealing instead to the musical object in
itself?
40 C h a pter 2

A Classical Positivist View: The Art O bject

Objects
For Gerard Genette, in his Work of Art: Immanence and Transcendence, improvisations
or executions of music and dance steps are acts, and thus facts, where a fact is a prop­
erty of the world, about which one can make a statement with truth value, true or
false.36 For him there are two kinds of art objects (of immanence) : ( 1 ) "real" factual
obj ects, and (2) evental obj ects, performances.
A performance is a physical event, and as such is a unique autographic object.
Recordings make multiples of an autographic art object (an "object-multiple"), whereas
iteration (as in a run of a show on Broadway) yields a "plural" autographic art obj ect.37
Following Nelson Goodman, Genette spends a fair amount of effort classifying and
distinguishing various types of "autographic" and " allographic" art objects, using the
test of counterfeitability. An allographic obj ect is one that cannot be forged, as is the
case when a copy with identical artistic effect can easily be made. He also devotes
considerable effort to the status of the multiple, but from the perspective of perfor­
mance all such questions miss the point, which is the durational (appealing to Bergson)
experience of the event. Indeed, in chapter 5 of his book, Genette turns his attention
to performances in order to subsume them also as stable objects as well.
Having posed factual and evental objects of immanence, Genette claims that there
are stable building blocks, and presumably therefore stable objects. Regarding building
blocks in the setting of performance, Genette claims, setting aside free j azz as "uncon­
vincing, " that even improvisation uses stock phrases, formulas, to render performance
"more facile" for musicians as well as audience.38 In other words, he claims that there
are primitives in the ontology of j azz music. In what manner is free j azz unconvinc­
ing, as music or as freeness? In any case, Genette's claim immediately falls under
Wittgenstein's challenge to the coherence of the notion of the analytic primitive.
Genette claims, further, that ( 1 ) every improvisation is simultaneously created and
interpreted; and (2) a "text" emerges from every performance. As evidence, he claims
that any performance can in principle be written out by anyone with "competence"­
much less than the competence needed for performing a text-in the "notation proper
,,
to the art in question. 39 A professional j azz musician will write out his prerehearsed
written solo, note for note. Genette observes that any "competent" musician can score
a performance of Charlie Parker, note for note. Although conventional musical scores
cannot record the muscular motion, embouchure, etc. that Parker used, nonetheless
another trained performer can imitate his style, and with practice can record as ade­
quate a "cover, " a reprodUction, of a Charlie Parker song as required.
However, it would be naive to believe that a score could possibly encode every
macroscopic aspect of a performance, a fact borne out by the great effort almost any
child must expend to learn how to sight-read European art musical notation. Further-
From Technologies of Representation to Tech nologies of Performance 41

more, transcription competence could itself arise from a dispersed and largely tacit
field of performance practices in which there are no stable music objects.
Again, it helps to look at recent (twentieth-century) Arab musical practices as docu­
mented by Issam EI-Mallah in his monograph Arab Music and Musical Notation. In Arab
practice, there is no standard tonal system, because such systems vary by region and
by musical "philosophy. "40 In fact, the instruments are tuned at each performance to
the singer's condition at the moment, not to a universal scaleY
The melodic scores that have been evolving in the latter part of the twentieth
century, borrowing from European notational systems and under the impact of record­
ing technologies that supported the development of a whole new economy of indus­
trially produced and marketed commercial music as well as heritage recordings, record
only the skeleton of the melody. But several severe artifactual problems have arisen
with this introduction of notation, beyond the by-now-obvious political, economic,
and social effects.
One problem is that the very act of scoring some songs imposes a selection effect:
while in practice there are no definitive versions of a given song, by scoring one
version, the one that happens to be performed for the notator, one version becomes
definitive. Secondly, what is notated is merely the skeleton, and omits all fine structure
of the melody because of the variety and "microtonal" pattern of Arabic tonal systems
that have no representation in the relatively coarse Pythagorean system codified in
European art music. For example, Arabic tonal scales include 1 7 and 24 steps, not all
of which are equal. And thirdly, the very notion of ornament as superfluous is incon­
sistent with a musical practice which makes no such distinction. As EI-Mallah writes:
"For the Arab artist, ornamentation does not represent an addition or a dispensable
and replaceable element of his art. It is the material itself, out of which the artist creates
infinite forms" (emphasis added) .42 Moreover, the nature of the ornamentation is that
it does not consist only of substitutions into a prescribed region of a form (an algebraic
syntax) . It is in fact variable in an endlessly variable way: even given a repeated form,
the performers can vary in any segment, and a given nuance can vary through a con­
tinuous range. EI-Mallah cites an early musicologist, R. G. Kieswetter: "The Arab singer's
character aims to ornament a melody with numerous neighboring tones.';43 We will
return to this notion of "the material itself" in continuous (and therefore' infinite)
variation when we turn to substrate in chapter 4.
In traditional Arabic musical practice, a note is not defined as an absolute frequency
but by the manner in which it is produced on an Ud,44 the lutelike principal instru­
ment of Arabic music, which in turn is tuned to the condition of the singer at each
particular performance. As this music is a primarily vocal art, its instruments model
the human voice, analogous to the practice of bowed stringed instruments like the
Eastern European gusle or the Chinese =tJl erhu. So the note is not defined as in Euro­
pean musical convention, by its position on the musical staff paper with an "absolute"
42 C h a pter 2

nominal frequency. We can interpret this as a radical illustration of Wittgenstein's

observation that the musical (which is not the same as the literary or cognitive)
"meaning" comes only through use. This seems surprising only before we who have
been disciplined under what Brian Rotman would call the alphabetic regime have been
infected/cured by exposure to the Philosophical Investigations. In a passage characteristic
of his argument against the coherence of definitions and rules, Wittgenstein writes:
When one shews someone the king i n chess and says: "This is the king, " this does not tell him
the use of this piece-unless he already knows the rules of the game up to this last point: the
shape of the king. You could imagine his having learnt the rules of the game without ever having
been shewn an actual piece. The shape of the chessman corresponds here to the sound or shape
of a word.
Consider this further case: I am explaining chess to someone; and I begin by pointing to a
chessman and saying: "This is the king; it can move like this . . . and so on. "-In this case we
shall say: the words "This is the king" (or "This is called the 'king"') are a definition only if the
learner already 'knows what a piece in a game is'. That is, if he has already played other games,
or has watched other people playing 'and understood'-and similar things. Further, only under
these conditions will he be able to ask relevantly in the course of learning the game: "What do
you call this? "-that is, this piece in a game.
We may say: only someone who already knows how to do something with it can significantly
ask a name.4S

The point is that the meaning or significance of a piece only comes about through its
use. Rules and definitions come after.
Moreover, rhythm is barely represented. This is not for lack of rich theorizations
and highly subtle reflections on rhythm over a thousand years of performance prac­
tice. EI-Mallah argues that this is due in large measure to the fact that rhythms and
melodies coarticulate one another. For that reason, a percussionist or any member of
the audience can j oin in beating or clapping in tandem with whatever is being sung
without being told in advance what rhythm to produce. In fact, there are even aspects
of the rhythm that cannot be deduced form the acoustic data of a performance because
it is interrelated with the way the bodies of the participants move during the course
of the performance. EI-Mallah observes that this is evident only from the video
record.46
Beats and rhythms are all named using syllabic sounds that have developed by
convention over several centuries, using syllables that are reproducible in all the dia­
lects of the Arabic countries. Rhythm operators such as subdivision are named by
extending the syllable names by some other syllable. This very elaborate naming
convention is more an algebraic assignment of a sonic signifier to some rhythmic
practice than onomatopoeia.47 But as Wittgenstein pointedly said, to name a thing­
i.e., to associate a verbal or written sign with a thing-is not at all the same as using
it or knowing how to use it. In this sense, naming rhythms and operations upon
From Tech nologies of Representation to Tech n o logies of Performance 43

rhythms bears analogy with mathematical notation and how it functions in deploy­
ment as instantiations, rather than representations, of mathematical objects and
processes.

Texts
Genette insists that a "text" is an interpretation of rule.48 However, he allows that the
text may be created in the moment of the performance. He claims this is true even
when the speaker Mirabeau exclaimed in the passion of his historic, unrehearsed
speech on June 22, 1 789: "We are here by the will of the people . . . "-in which case
that extemporaneous exclamation was not the interpretation of a preexisting text.
However, after-the-fact memorialization of an event does not imply the existence of
a well-defined object coincident with the performance, except by convention. In
principle, the difference is whether a litext" preexists or is created in the event, in the
moment of the performance. Genette asserts that this difference is "irrelevant to the
way we describe the mode of existence of the work of performance considered in and
of itself, that is, without regard to whether it is pre-existent or not. "49

Notation
Citing Goodman's definition that linotation is any unambiguous system for identifying an
object, " Genette exhibits the point of view of the critic or literary scholar, rather than
the composer or performer. He cites Goodman's claim that it is necessary that liiden­
tification of the or a [possibly unique] instance of a work be independent of the history
of productioni a notation as much codifies as creates such an independent criterion. "sa
This is a very useful way to interpret an event's litext, " but if we more rigorously pursue
the tactic of moving from nouns to verbs, we need not start with representations of
any obj ect, whether a song or a text, but with performances instead.

Objects, Again
An allographic art object can have both immanent and ideal manifestations. Genette
gives a particularly enlightening example: a particular copy of a novella can have
200,000 words and be 400 pages long, whereas the ideal text also has 200,000 words
but is no pages long. Genette diagrams this general scheme in all modes of art, includ­
ing musical performance, in which case the scheme takes the form shown in figure 2 . 1 3.51

M usical text
/' �
Performa nce Score

F i g u re 2. 1 3
Gerard Genette, musical text, performance, score. Diagram by author.
44 C h a pter 2

But as the extensive counterexample of nine centuries of Arabic music indicates,

this is by no means a necessary ontology. EI-Mallah devotes an entire chapter of his
monograph to explaining why this music has no written notation. We can turn that
around and use that fact to learn something about the relation between musical per­
formance and its technologies. EI-Mallah summarizes Arabic musical practice as:
(1) an "oral" cultural practice;
(2) collectively performed-all people corporeally participate in the construction of
the musical event. A performer plays a large or even identifying role in coconstructing
a musical work together with the composer and noninstrumentalist participants; and
(3) improvisatory "Arabic music is about variability, not variation" (emphasis added) .52
-

From Object to Deep Structure of Objects

Beyond the putative eternal object (using Whitehead's terminology), what Genette
calls the "immanent" "text" of a performance, one could also claim that there is a
deep structure invariant beneath all particular texts and performances. Valle cites
Reginald Smith Brindle:
It can be said that where notation is used, the general norm is the free twelve-note language,
while where notation is not used (in graphic scores, for example) still at some point the score
has to be realized in sound, either by improvisation or in the player's own performing version,
and again the free twelve tone idiom would almost certainly be employed-unless the composer
specified other means, which is extremely rare . . . . [So even with the graphiC innovations of the
1 9 60s and 1 9 70sj "the situation has been much more static than at first seems apparent, " because
"the basic language-free twelve-note music has remained the same . . . . [Ij f that which counts
is only a phenomenologically accurate transcription of the time development of sounds . . . ,
modern musical notation in use from 1 600 is demonstrated to be a perfectly functional instru­
ment even for improvised music. "S3

Although Smith Brindle's observation may be true of European art mUSiC, this is
false in general. It is certainly not the case for Arabic music since the ninth century,
considering diachronic evolution of the 1 7- and 24-interval tonal systems and the
synchronic variation of idiosyncratic tunings from region to region. Indeed, Valle
allows for this, admitting a possible ethnocentrism in such a claim. 54
So, retreating from this stronger claim about deep structure, what about Genette's
weaker claims about the ontological if not chronological priority of an object, a "text"?

Toward a Nonclassical Theory of Performance: Partial Objects and Partial Actions

A fundamental problem with Genette's " obj ects of immanence," " real, " factual, or
evental, is that he makes an a priori distinction, weighted toward a plastic object, as
From Technologies of Representation to Tech nologies of Performance 45

against performed, temporal process. He subscribes exactly to what Simondon identi­

fied as the ontological reversal of prioritizing the individual object over the process
by which it comes to be. ss But instead of trying so hard to turn a performative event
into a synchronic /ltext, " why not focus on the instructions to the maker for use in
the making? We will be shifting our focus from objects like a text or a musical work
to the performer's performances and improvisations.
Think of the historical Arabic characterization of a note (pitch) in terms of how it
is fingered on an ud. The instrument in turn is tuned not to an absolute frequency,
but to the regional conventions and the singer's condition at the moment. Moreover,
it is by no means necessary to accept such a universal claim as the existence of a
/lmusical text" immanent to every instance of a performance. As Antonin Artaud
wrote, /lInstead of continuing to rely upon texts considered definitive and sacred, it
is essential to put an end to the subjugation of the theater to the text, and to recover
the notion of a kind of unique language halfway between gesture and thought. This
language cannot be defined except by its possibilities for dynamic expression in space
as opposed to the expressive possibilities of spoken dialogue. "s6
Against Genette's claim that text is interpretation of rule,s7 we have Wittgenstein's
trenchant skepticism and EI-Mallah's histories of Arabic musical practice. EI-Mallah
reports two stories. The first is the example of an /linformant, " a local Arabic musician,
hired by an ethnomusicologist to /lwrite" down a song central to his community. The
informant comes back several times for more money, till at last it is revealed that he
was making it up. The point of course is that the musician could not have done oth­
erwise. The other example is the story of a prince who wanted a particular song from
a neighboring court, and sent a spy to steal this song by committing it to memory.
The spy was repeatedly frustrated because every time he returned with a different song.
The point is that every time the song was performed it changed in substantive ways:
there was nothing like a canonical version with variants.
Genette's assertion that the difference between a text preexisting or coconstructed
in an event of the performance /lis irrelevant to the way we describe the mode of
existence of the work of performance considered in and of itself" is tautological.
Moreover, this ignores the fundamental aspect of performance as event rath� r than as
text. And it is not surprising that Genette, a literary theorist, seeks to reify "the text"
so insistently against performance; following Walter Ong, this is in fact a hallmark of
literate culture .
And finally, the perspectives maintained by Genette and Valle are the perspectives
of the spectator or the composer, but not the "first-person" experience of the per­
former. Genette accepts the theatrical convention, the same irreducible dual that ]erzy
Grotowski insists on: an actor and a spectator. But whereas they insist on a classical
distinction between these as two distinct, fleshy bodies, I mobilize these as dispositions
that can be taken on by any body, even by one and the same body.
46 C h a pter 2

M usical N otation as a Semiotic System for Performance

Valle sees notation as an essentially temporal semiotic system:

Notation reflects constitutively a temporal instant, an aspect of transition of time from an initial
situation to a successive one, in which it is necessary to conserve something which precedes it.
In that way, the concept of notation centers on a characteristic of mediation, the transitoriness
between states, connections between elements . . . .
A note is a special sort of sign. Every note is an " act of notating. uSB

Yes, but by symmetrizing one's temporal perspective, not from t to -t but from Dt to
-Dt-or more vaguely but less reductively, the directionality, or temporality, of action­
every notation is also a performative act. In other words, the act of notating that the
composer makes, to use Valle's term, could be fruitfully regarded as a partial action (to
detourne Lacan and Serres) that is complemented by the corporeal action of the per­
former and perceiver in the production of sound of a musical event.
Using Peirce's semiotic framework, Valle remarks that notation exhibits an obvious
" secondness, " a distance or difference from that which the notation effects. Moreover,
it bears also a "thirdness." For Valle,
Two elements constitute every notation: ( 1 ) The secondness of a procedural simile as a difference
in space and time (that is, in the necessarily heterogeneous constitution of the note with respect
to that which is notated) [and] (2) the note's thirdness, . . . which mediates between two instants:
[he] who notes and that which the notation effects. 5 9

However, in so distinguishing these semiotic agents, Valle obscures the instrumentalist­

performer.

A N otation's Expressive Power

A notational system's expressive power, as I suggested earlier in this chapter, depends
on factors including syntactic density, resolution, range, completeness, degree of
nuance, poetic potential for connotation over denotation. No notational system can
completely and precisely describe or prescribe every nuance, and the mechanical
means merely exponentiate the complexity of the problem. Therefore, as a practical
matter, one can proceed with an eye to the minimal notational systems that ade­
quately scaffold coarticulation in the technical ensemble linking composers, instru­
ments, instrumentalists, and all the participants and media in the event.
Moving to the macroscopic social and historical scales, Valle reminds us that evolu­
tions of notation reflect changes in performance practice, but their histories are not
congruent.60 Valle quotes an essay on the history of musical notation: "only a funda­
mental break with established musical aesthetics and philosophies can bring about a
commensurate notational change, and such profound upheavals have been extremely
rare."61 He identifies three great semiographic shifts: (1) the passage from monody to
From Technologies of Representation to Tech nolog ies of Performance 47

polyphony in the ninth century with the adoption of diastemazia and mensura (mea­
sures, bars); (2) the shift from horizontal polyphony to the predominance of harmony,
which sees the substitution of unique parts; and lastly (3) in the mid-twentieth
century, the unprecedented augmentation both of precision and of aesthetic
ambiguity. 62
Valle makes a most significant observation: "the employment of very short duration
values which indicate in all probability a mode of execution similar to the speaking
voice reflected the rhythmic innovations of performers during the Ars Nova. "63 The
converse to Valle's point, that notational innovation can drive changes in performance
practice, is true as well. We see this especially clearly in twentieth-century European
art music in many instances. For his vocal-instrumental piece Ancient Voices of Chil­
dren,64 George Crumb used a custom notation in the opening movement asking the
soprano (the piece was written for Jan DeGaetani, a virtuosic and experimentally
adventurous singer) to sing sequences of notes of (continuously) progressively shorter
and then longer duration (figure 2 . 1 4) . Twentieth-century composers invented ever
more radical notational neologisms for performers to enact, sometimes far beyond
their abilities to reproducibly, let alone "expressively, " produce a sound. In fact, it's
useful to regard Cage's Fontana Mix6 s as an analog computer with a graphical interface.
His instructions are high-level "code" interpreted by human performers.
At an appropriate scale of historical time, one can see the intercalation of innova­
tions by performers and writers (composers and notators) . One can see an analogous
intercalation of innovations among Arab performers, composers, and "notators" from
the 1 940s, when European art musical notation was adapted to represent Arab music­
first by ethnomusicologists (echoing European colonial and postcolonial anthropol­
ogy) and then in the complicated social dynamics of accommodation, in which
musical notation was interpolated along with studio recording-based production and

F i g u re 2. 1 4
George Crumb, Ancient Voices of Children, score.
48 C h a pter 2

radio diffusion economies alongside traditional nonpublic, nonmarket practices of

music making.
As Valle described twentieth-century musical notation in European art music,
musical notation both undercodes and overcodes musical performance. This is not
only true for twentieth-century composers' practice vis-a.-vis instrumentalists in the
West, but is true in general of any notational scheme. For example, in his account of
how Arab musicians' orally transmitted practices adapted in the twentieth century to
European notational schemes, El-Mallah notes in detail how European notation radi­
cally underspecifies the tonal structure, the relation between rhythm and syllabifica­
tion of the Arab languages, rhythmic complexities, the continuous variability of the
practice, and most of all the radical contingency of the orally transmitted and
community-performed Arab music. To underline a point worth reiterating, because it
can be said of all live musical performance, Arabic musical performance consists of
continuous variability, not a permutation or substitution of one variation in place of
another in a form that is fixed across all instances of performances of a song.

Computational Technology

As musicians have adopted electronic technologies and then computational tech­

niques, new questions have emerged concerning proficiency, and rehearsal versus
tinkering. With computational techniques, notation evolved not only in degree­
temporal preciSion, tonal resolution, or fineness of instrumental manipulation-but
in kind. Who or what interprets the notation in the course of production of the sound
during a performance? As performance scores and scripts extend beyond the limits of
what a fleshly musician can realize, and with the augmentation of the performing
body, the locus of compositional invention has diffused into the design of electronic
and now computational instruments. These questions are deeply related to whether
the activity is about making experience for the spectator or for the self. We will take
up the question of what expressivity means in this context later, as we draw out what
serves in place of notation when we adjoin computational technologies to the produc­
tion of organized sound and video.

Off-line Model to Real-Time Modulation

In any case, the strengthening in degree and kind of operation sustained by exponen­
tial advances in computational technology has enabled a different obj ect of compu­
tationally mediated activity. Mathematical models once only realizable as retrospective,
"off-line" simulations of some physical phenomenon can now be used as instruments
of "live performance. " And the present state of computational media technology is
combining rich models with dense visual and sonic media. There is nothing except
convention to prevent us from using such instruments for expressive applications.
From Technolog ies of Representation to Technologies of Performance 49

Of course, the historical narrative of technology just sketched does not determine
the actual, critical, and symbolic argument that I trace in the next sections of this
chapter. It suggests the necessity but not sufficiency of the individuation of compu­
tational technology in the technological milieu of media arts and sciences, borrowing
Gilbert Simondon's terminology for understanding the evolution of technology.
With the advent of electronic computation, representation in its particular form of
the scientific model comes alive in the mode of numerical simulation and graphical
visualization. The key question here is the status of the (computer) model, which is
transformed by the increase in speed and the enrichment of the operators, enabling
but not determining the transformation of attention from off-line computer software
modeling to directly manipulating the material world.
In other words, with the increase in computing power as measured by arithmetic
operations per unit time, memory capacity, and scientific expressive capacity of soft­
ware programming languages and machine instruction sets over the past fifty years,
what could only be presented as a synchronic or retrospective model can now, in
certain domains, be presented as "real-time" computation that changes in tandem
with living processes. This enables, but does not guarantee, a "performative turn" in
the domain of computer music, witness for example musicians like Michel Waisvisz,
Laetitia Sonami, Pamela Z, Joel Ryan, and Carl Stone.

What Is a N u merical Model or a Simulation?

For our purposes, a model is a set of numerical data structures and operations coded
into a computer to represent some system or obj ects in the world. It is essentially a
snapshot, synchronic. A simulation is a set of processes in the computer that rep­
resent some changes of state in the world. Simulations are essentially temporal,
diachronic.
How is a mathematical, scientific, or computational model related to what the
model is supposed to represent? For my purposes, a scientific model is some combi­
nation of physical objects, fictional objects, set-theoretic structures, descriptions, and
equations representing portions or aspects of phenomena, data, or theory concerning
'
nature. Models intercalate observers, phenomena, and theories and can sometimes
be manipulated in regular ways. Some models, for example, in biology are discursive
narratives, whereas mathematical models have parameters that can be matched to
numeric empirical data. Such models may have algebraic or function-analytic struc­
tures that can be manipulated, given those parameters, to yield definite statements
that can be compared with empirical observation. Despite what one might believe
under the regime of digital computation, not all scientific models have such manipu­
lable parametric structure. (Physicists like to call them the constants in a theory,
such as the speed of light or the ratio between the electromagnetic force and gravity.)
And of those models that do have an algebraic or analytic structure, some fraction
50 Chapter 2

can be represented or approximated in a digital form, and therefore operated in a

computer. This means that the model must at bottom be a numerical or a logical
model, which covers a much smaller range of models than mathematical models in
general, still less than scientific models (looking back through biology and astronomy
for example) .
Fifty years ago, finite-element models of structures such as the steel skeleton of a
bridge could only be realized on a computer as a batch j ob, meaning that the engineer
had to write out elaborate encoded, textual descriptions of a configuration, along with
descriptions of forces, then submit the data and instructions on what to do with the
data into a device that transferred these instructions into some electrical states in the
machine, and then wait a considerable time (a "coffee hour") for the results. This
stop-and-go-and-wait necessarily divides the process of thinking about building a
structure into a step-by-step process with large gaps (hours or even days) in between
one configuration of thought and the next. However, when more powerful computers
allowed the adjustment of tensions in tandem with the change in some segments of
a proposed structure, the engineer could experimentally vary structures continuously,
and reason with the structure in continuous variation. Continuous variation allows a
qualitatively more flexible order of invention, and enables the exploration of a much
larger design space.
Computer processor speed alone does not suffice to transform the nature of com­
putationally augmented life. The type of operation must also be more expressive in the
senses that I sketched above: resolution, range, syntactic density, completeness,
nuance, connotative potential (poetry) .
What makes the representation of finite elements in engineering building structures
amenable to " real-time" manipulation by humans is the presentation of such informa­
tion not as coded algebra but as visual depictions of the geometry, along with ways
for the engineer to insert, tug on, and vary the "physical' elements of a structure, with
the computer visually presenting the stresses that it computes using its encoded
models of numerical physics.66

G raphics and Sound Editing Software as Technologies of Representation

Another, more everyday class of software is the genre of image and moving-image
editing applications. Consider the difference between editing digital and Foley sound
effects into a film or video that is perceived later by an observer who cannot intervene
in the sound as you make it when you speak. Analogously, consider the difference
between creating visible effects in a film to be viewed later by a spectator who cannot
intervene in the playback of the visual imagery and the play of the visual features
your face makes as you talk. Examples of the software typically used as technologies
of representation rather than as instruments in live performance include Photoshop,
Premiere, Final Cut, and After Effects.
From Technologies of Representation to Technologies of Performance 51

Again, for the purposes o f this book, we are not concerned with the synchronic
representation of media but with how it is created, synthesized, or modulated " a
tempo, " i n real time.

Real-Time Video and Sound I n struments as Technologies of Performance

Examples of software technologies of performance include VJ (video j ockey) software,

Ableton Live, and Max/MSP/Jitter, presently the lingua franca of live-media artists.
This class of applications characteristically presents images and sounds without expos­
ing what computational or mathematical models may be used to create or manage
them. Unlike the class of software used to manipulate models, this class allows the
user to manipulate bits of image and signal. What one loses in structure, one gains in
sensorial density. For example, one could represent the physics of a clamped plate as
a solution to a biharmonic partial differential equation on a simplicial complex
approximating a physical shape, then "excite" the physical model by presenting some
simulated stimulus (or an input from some sensor attached to some physical control­
ler) . That would require a complicated software acoustical simulation that would have
to be carefully optimized for particular hardware. Or one could bang on the physical
obj ect with that shape, and record the resulting sound via a microphone. The software
representation in the former case is a very complex set of models and algorithms
Simulating processes in those models. The software representation in the latter case
contains no such model of the physical obj ect but is merely a file with a stream of
numbers representing amplitudes as a function of time, an " audio" file.
Such software, especially if it includes compression-decompression algorithms, may
contain some sophisticated algorithms, but the algorithms are semantically shallow
in that they do not model humanly legible descriptions of a physical object, system,
or situation. Nothing in that audio data representation codes humanly legible sonic
semantics such as key, time Signature, instrumentation, or the name of the composer
or the form of the piece. The AI literature used to call such data-centered rather than
model-centered representations "opaque. "
The set o f softwares for modulating sound and video i n real-time performance
bifurcates into those that are shrink-wrapped as "applications" whose logiC or mode
of behavior is encapsulated into a few commands (e.g., Ableton Live), and those whose
logic is always editable (e.g., Max/MSP/Jitter or any "programming environment" with
access to media data) . The latter class permits (or forces, depending on your point of
view) the performer to also be an instrument maker. Whether you think this extends
or inhibits the performer's musical expression, in practice what we witness, in the
present era of computational programming environments for processing time-based
media for performance, is the performers' incessant tinkering with the "instruments"
that they play.
52 C h a pter 2

There is another, equally powerful consequence of Moore's Law-and that is the

enormous decrease in size and cost of the devices, the equipment needed to produce
these media effects. The history of such media centers, especially in the domain of
electronic music, parallels interestingly the rise and decline of high-energy physics
labs. Whereas in the 1 9 70s, supporting works by Nono and Stockhausen required a
whole institution with labs, space, and dozens of specialist technicians, similar work
can now technically be realized on a personal computer with some gear that can be
purchased by an individual artist or even hobbyist.

Four Shifts of Perspective

The rest of this chapter intertwines four shifts of perspective:

(S l ) From composer and critic to performer;
(S2) From tools of analysis to instruments of performance;
(S3) From designing sound objects to designing controllers-devices manipulable as
instruments;
(S4) From designing instruments, to designing the dynamical, material, palpable qual­
ities of the temporal medium itself, to manipulating stuff as instrumental media. An
example is the motto " shaping sound like clay" that titles Achim Heidenreich's history
of electronic music at the ZKM in Karlsruhe.67 This last constitutes a shift from the
perspective of instrument makers to that of makers of media, i.e., alchemists.

S 1 : From Composer or Critic t o Performer

Refocusing on the performer has been accompanied by a surge in gesture-based "inter­
action" and " controllers" and sensor engineering. But as Yves Abrioux has noted,
the fundamental conundrum of these engineered turns to the performative is that
,,
"meaning does not reduce to behavior. 68
Genette consistently considers the point of view of the author or composer, not
surprisingly given his home in literature. But the performing arts introduce a key dif­
ference, which is the role of the performer who intermediates the composer and audi­
ence, spectator, or reader.

S2: From Tools of Analysis to I nstruments of Performance

Returning to computationally synthesized sound and video, we can shift attention
from computer programs as tools of analysis or representation of "physical" sound or
"physical" musical performers or "physical" instruments (as if there were some onto­
logical priority to such things), to software applications that can be played as instru­
ments rather than as models of the real. Take OMax as an example, which bundles
learning algorithms like hidden Markov models (HMMs) built atop feature extractors
into live-play instruments.
From Technologies of Representatio n to Technologies of Performance 53

Although many of the learning algorithms used have been known for years, OMax
represents a qualitative shift in how they are organized into instruments for real-time
performance. The processing logic is both powerful and elegant, as succinctly pre­
sented by an image of loops of paper printed with strings of text (figure 2. 1 6) . The
instrumental interface is nonetheless complicated (figure 2. 1 7) . Moreover, such inter­
faces are by no means restricted to a virtual panel on the screen. The basic problem
with the WIMP interface is the assumption that there is only one point of interaction
(the cursor). But in a musical control, there are often dozens of parameters functionally

F i g u re 2. 1 5
OM ax Markov-model-based pattern recognizer. Arc diagram interface suturing intervals with
similar features.

Figure 2. 1 6
Analogy for OMax looping algorithm based on matching subsequences.
54 C h a pter 2

Fig u re 2. 1 7
Navid Navab, OMax-based instrument, TML 20 10.

mapped to hundreds of experientially meaningful parameters. Therefore performers

use combinations of screen-based displays with physical hardware with many parallel
inputs (hence the term "fader boxes").
Our earlier three-way application of notation maps interestingly to this case. Notat­
ing the kinesic manipulation becomes coding and twiddling parameters on the soft­
ware instrument. Notating sound becomes coding granular synthesis, FM synthesis,
physical models, some SuperCollider processes. Notating structure becomes coding
SuperCollider, or scripting in higher-level machine representation or languages: MIDI,
LOOPS, Kyma, OMax, and so forth.
However, notice that these are still discrete inputs. Next, let's consider a class of
"instruments" whose inputs are continuous fields.

Example: Responsive Media Environments

A responsive media environment is a physical space in which people's activity and
time-based media (video, sound, active materials) influence each other and evolve in
From Technologies of Representatio n to Technologies of Performance 55

F i g u re 2. 1 8
External hardware controllers supplementing the window-keyboard-mouse software interface
for live musical performance: faders and knobs, and pine cone plus microphone as an audio­
rate sensor.

concert to create an event. The people may or may not be expert or have rehearsed
in the space for the event. Contemporary techniques for sensing activity and coordi­
nating or synthesizing the media draw on a large variety of sensors, including position
or force sensors, photocells, air and contact microphones, and cameras. They can be
placed on or inside the body, in air, or on the floor, walls, or objects (props) . For our
purposes, the important point is that all these sensors transduce physical, analog
conditions into electrical signals that are then converted into numerical data. Along
the way, effectively continuous matter in motion is transcribed into noise to be neglected
and signal to be further processed. Sampling this signal at intervals further introduces
artifacts of discretization. A lot of fundamental reduction has already been committed
before the data even gets to the input to a real-time media programmer's software,
coded in real-time media programming environments such as Max, PD, VVVv, or
SuperCollider. A programming environment like Max/MSP/Jitter make idiomatic quite
different logics than procedural languages like Java, Expressions, or C. Rather than
thinking in terms of variables, iteration, loops, and cases, it's more idiomatic to think
of streams of values, sound, or images pushing through a net of operators �onnected
as if they were physical devices plugged together by physical cables. In fact\ the pro­
grammer must unlearn Boolean "if-then" thinking in order to write idiomatically,
more freely and expressively.
The major distinctions introduced by responsive software systems based on move­
ment and gesture include the following:
• Notation bifurcates into media that humans can manipulate, and code for machines.
Instruments, instead of being prefabricated, can be fabricated out of the responsive
medium: sound + sensors + software + gesture + controller, practiced together as a repro­
ducible "instrument" along with performance practices idiomatic to that instrument.
56 C h a pter 2

• A responsive environment supports the nuancing of any gesture to any degree of

fineness (e.g., in a continuous gesture space, in a sense to be made rigorous in chapter

6) . We evaluate a responsive environment according to the qualities we used to char­
acterize expressivity: syntactic density, resolution, range, completeness, nuance,
connotative potential. We leave open the question: For political, ethico-aesthetic
experience, do the inhabitants need to shape the notation, or the conditions of an
event, during or outside the event?

S3: From Designing Objects to Designing I n struments

The intricate history of electronic and now digital instruments exceeds the scope of
this book. But as we stated earlier, with the introduction of electronic and computa­
tional technologies, the locus of compositional invention diffused into the design of
new sensor- and software-mediated instruments.
Responsive-media environments allow people to play in concert with whole bodies
in free space. Composers of the responsive environment can condition the possible
events by introducing costumes, furniture, or other architectural constraints.

S4: From Designing Objects and I n struments to Designing Media: Alchemy

The TGarden contains already a general example of an apparatus sustaining cross­
modal temporal media whose qualities can be composed to vary over the course of
an event according to both the composer's imagination and the inhabitant's contin­
gent play.
The analogy would be to shift the perspective from the making of a particular vase
or its deployment in the interpretation of a particular script, to the shaping of the
clay from which vases or many other ceramic obj ects can be shaped. Alchemical art­
istry would change, for example, how clay responds to and remembers touch as it is
shaped on a wheel.

Example: I nterstitial
We can see clearly this sort of alchemical design in the much simpler example of an
exercise by three students in a movement + media workshop, called Interstitial.69
A dancer-acrobat stands behind an elastic, translucent screen and presses his body
into the elastic membrane. The elastic sheet constrains the free movement of bodies
to a neighborhood of a two-dimensional surface. Although it is two-dimensional, the
membrane is not a rigid plane; it continuously evolves, and palpably resists or aug­
ments movement by summing players' gesture with its own field of tensions. A trans­
ducer attached to the membrane transmits oscillations to sound feature extraction
and synthesis software that modulates to accompany the movement. The real-time
video instrument processes the live video of the infrared "shadow" of the dancer from
the membrane, and processes the image into a silhouette that is reprojected back onto
the membrane, illuminating, edging, and haloing the figure. Simple relays and feed-
From Technologies of Representation to Tech nologies of Performance 57

back create video as structured lighting that produces a rich range from highlights on
the body to coronas to doppelganger imprints.
In this example, a membrane constitutes a material, embodied, continuous mode
of expressing corporeal entanglement. A " score" can in fact be projected onto the
membrane not as a matrix of discrete icons or diagrammatic signs, but in the form of
a continuous field of a continuously running video image. In fact, we can regard the
processed video feedback from earlier videos of a dancer as in fact a " score" for the
current dancer. The same membrane that images the score simultaneously serves as
the dancer's medium of articulation. This membrane is an extension of his tissue, and
its oscillations instantaneously modulate the field of sound that saturates the space
of the performance. With this example, we have traveled a long way from the realm
of classical distinctions between composer and performer; sound, kinesics, and score;
instrument and body; notation and body.

Example: Constel l ation

Returning to sound technology, Meyer Sound's Constellation system uses the com­
pany's most advanced speaker arrays, acoustic analyzers, acoustic models, and sound
field synthesis algorithms and devices not to present a specific, spatial sound field

F i g u re 2. 1 9
Navid Navab, Jerome Delapierre, and Bruno Gagnon, interstitial sonic membrane experiment, 2008.
58 C h a pter 2

(e.g., a tableau of sound obj ects) but a set of latent acoustic responsivities charac­
teristic of the acoustics of various architectural spaces, in the computational model.
This is technoscientifically a most subtle and intricate accomplishment because the
system must map the response of another physical space so that any sound (object)-a
clap or a singing voice-can be preprocessed so that it appears to resound in that
other space, with all its material features (volume, wooden walls, carpeting, glass
windows, etc.) .

The Larger Stakes: From Representationalism to Performativity

I have deliberately focused my attention on expressive performance in the sound and

video media, in order to get some traction on the notions relevant to technologies of
performance. But their significance extends beyond media art and entertainment.
Genette's factitive obj ects and musical texts are, in the end, fetishes, or at least, as
Wittgenstein might put it, grammatical errors. Martin Heidegger, in his Metaphysical
Foundations ofLogic, criticized both the claim that there is an object at hand with every
subject and the claim that a subject is at hand with every obj ect, because the at-handness
already entails a givenness that does not require the other entity.70 Instead, we can
think of the software/hardware systems assembled for responsive environments not
as housing a pre scripted sequence of sound or image obj ects, but as sets of potential
performances. I will return to this in chapter 3 .
Much social, political, epistemological contest comes down t o the stabilization of
objects, or who gets to stabilize these obj ects and under what conditions. (I will expand
on this in chapters 4 and 5.) Graphic design, musical composition, film, and all the
arts of obj ect making do not necessarily grapple with this crisis. Why not? Perhaps,
despite the best efforts of the artists, we tend to elevate the obj ect itself over the process
of its gestation or construction. Of course, we could acknowledge with Baudrillard
that we live deep in an epoch in which objects have won over subjects, but this would
be too simple a response after quantum mechanics forced physics to blend objects
and subjects, the thing regarded with those who regard.
Latour, too, makes use of objects but in the sense of object of concern, calling upon
us as social actors and social scientists to shift from being concerned with making
matters of fact to making objects of shared concern. However, I think that reducing
questions of ontology to "What objects make up the world?" confers upon objects a
peculiar temporal invariance that effectively fetishizes them. Instead, my questions
turn on how objects materially form in the world. I'll take up the question of onto­
genesis in chapter s .
If the crises o f political and linguistic representation stymie us, can w e sidestep
these crises by sidestepping representation? Can we sidestep some of the rigidities
imposed by discrete symbol systems and rules (abstract machine) and schemas by
From Technolog ies of Representatio n to Tec h n o l og i es of Performance 59

playing in open continuous material that responds to our continuous action? After
all, if human experience is thick, dynamic, and continuous, then let's play in material
fields that are thick, dynamic, and continuous. By material fields I mean distributions
of matter, energy, and affect, such as people in a collective space, the sound of their
speech, the light incident on the space, or the sound in which they are immersed.
For more than a decade, the work of the Topological Media Lab has been motivated
by these concerns. But we decided to take an experimental, ethico-aesthetic approach
distinct from the direct approaches of political/cultural theory, critique, or activism.
Karen Barad succinctly motivates the turn from representationalism to performativ­
ity in her book Meeting the Universe Halfway:
Language has been granted too much power. The linguistic tum, the semiotic tum, the interpre­
tative tum, the cultural tum: it seems that at every tum lately every " thing"-even materiality-is
turned into a matter of language or some other form of cultural representation. The ubiquitous
puns on "matter" do not, alas, mark a rethinking of the Key concepts (materiality and Significa­
tion) and the relationship between them. Rather, it seems to be symptomatic of the extent to
which matters of "fact" (so to speak) have been replaced with matters of signification (no scare
quotes here) . Language matters. Discourse matters. Culture matters. There is an important sense
in which the only thing that does not seem to matter anymore is matter. . . .
A performative understanding of discursive practices challenges the representationalist belief
in the power of words to represent preexisting things. Unlike representationalism, which posi­
tions us above or outside the world we allegedly merely reflect on, a performative account insists
on understanding thinking, observing, and theorizing as practices of engagement with, and as
part of, the world in which we have our being.
Performativity, properly construed, is not an invitation to tum everything (induding material
bodies) into words; on the contrary, performativity is precisely a contestation of the excessive
power granted to language to determine what is real. Hence, in ironic contrast to the misconcep­
tion that would equate performativity with a form of linguistic monism that takes language to
be the stuff of reality, performativity is actually a contestation of the unexamined habits of mind
that grant language and other forms of representation more power in determining our ontologies
than they deserve. 71

I would extend Barad's characterization of performativity to include the dynamical

change of the material state or configuration of the world. In other words,\ performa­
tivity is thoroughly attuned to the temporal. Since we are concerned with material
change, we should attend to the technologies by which we effect such change, which
in the present epoch is dominated directly or indirectly by computational technology.
In this chapter, I have shifted attention from representational applications to live
performance applications. I do not claim that a particular piece of software or a tech­
nique is either representational or performative, only that one can regard an applica­
tion in representational or performative modes of use. Nor do I simply align the
performative with the body and representational with the sign, since even a physical
action can be carried out representing something other than the physical action.
60 C h a pter 2

Keeping Barad's characterization of performativity in play, we should also add its

positive, generative aspects. After all, the significance of an action, or of any nameable
object or process, becomes evident to the participants in an event in the course of
action. Wittgenstein distinguishes between the descriptive definition of a chess piece
and its use in playing a particular game-the description or the concept of an act
versus the act. 72

Res Publica: Arena of Objects to Arena of Care

Ethico-aesthetics of Technologies of Performance: Play Spaces, Not G a mes

There is also an ethico-aesthetic question: In light of such an understanding of the
performative, can we design technologies of performance that feel rich but not com­
plicated? Yes, of course: consider the ordinary technologies of food and of cloth. The
art collective Sponge built responsive play spaces based on fields of tangible media
evolving according to folk physics that can shape and be shaped under folk knowl­
edge. We have built play spaces in which people can individually and collectively
improvise meaningful gesture. But people can construct things of j oint concern by
movement and gesture without first explicitly agreeing on a common working vocabu­
lary or even a conversational protocol. In a responsive space like the TGarden, players
escape the straitj acket of sequential discrete conversational speech acts. In a formal
model of "explicit" communicative action, we deploy language to represent ourselves
to our fellows, and we take turns making utterances. But in a TGarden, the air currents,
shadows, pieces of words, fabric, or hair that one leaves in one's wake form the mate­
rial conversations in a corona around explicit conversation. Ordinarily such detritus
does not constitute matter of articulated concern, but this is where the technologies
of calligraphic responsive media come into play. Thick media, computationally syn­
thesized or not, provides a substrate in which bodies need not be so dogmatically
identified: it can put into play where your body ends and where the world begins.
(Consider the appeal of a salt bath at body temperature.) Sensors for movement and
physical conditions, together with computational means to modulate media, provide
a substrate whose quasi-physics may be artfully varied. Responsive visual or sonic or
physical media driven by sensors map movement and gesture into dynamical textures
that render the dynamical spaces between our bodies palpable, shareable, and infi­
nitely open.
Although the TGarden was conceived as a play space in which participants could
continuously invent gestures significant to themselves and others, there is a serious
implication. Unlike in "serious games, " there is no explicit rule or goal or didactic
purpose to deaden the event. Such telos can generate games as deadly as what Peter
Brook called Dead Theater.
F rom Technologies of Representation to Tech nologies of Performance 61

In political process, it is the substrate lobbying, researching, writing, and caucusing

processes-punctuated but not constituted by formal parliamentary procedures-in
which individual and collective and political identities emerge. And offering the public
a view (whether via Internet archives or via box seats in the physical assembly cham­
bers) of the parliamentary debate does not make legible the more primordial process
of emergence of political position and relation. Sheathing the Bundesrepublik building
in Berlin with glass symbolically represents this faith in surface politics. So does the
machinery of digitizing and making available on the Internet the texts of the bills
passed in Congress.
In 2004, Peter Weibel and Bruno Latour curated an elaborate exhibition titled
"Making Things Public: Atmospheres of Democracy" at the former's institution, the
Zentrum fur Kunst und Medien. This hybrid between an art exhibition and an aca­
demic conference attempted to materialize Latour's argument for a democracy ori­
ented to things, by which they meant materially, socially constructed things. As Latour
argued, the way to do that is to hybridize politically good representations that faith­
fully gather "the legitimate people around some issue" and are scientifically accurate­
representing "what is the object of concern to the eyes and ears of those who have
been assembled around it. " However strong and clear in concept, the exhibition
seemed to carry a didactic air, the same didacticism that has limited science museum
audiences in North America. Latour himself is sufficiently anthropological to suggest
that not all cultures value the assembly of assemblies that he poses as a political ideal,
naming in fact the Japanese, Chinese, and Papua New Guinea islanders as those to
whom such political practice seems implausible or impractical. However, Latour insists
on the position that, notwithstanding such skepticism, a "parliament of parliaments"
is still a desirable political form. 73
However, a parliament is a political machine in which defined interests take on set
positions and create set battles with procedures closer in spirit to the Euclidean forma­
tions of eighteenth-century military science than to nomadic warfare.
I do not say that stable, even rigid structures ought to be abolished! Indeed it is
extremely useful to have refined, visible structures with respect to which we can
perform tacitly informed, continuous and microtonal improvisations. What is more
important is whether, how, and by whom the conditions of this improvisation can be
modified.
So what about the primordial processes themselves, the dynamics of care prior to
the formal conversation articulated in writing and verbal language? What about the
processes prior to the formal events that in bureaucracies are more loci of ratification
than of emergence? In chapter 4, we will see that a consequence of a plenist (versus
atomist) approach to the world is the possibility for articulation of this primordial
dynamics of care.
62 C h a pter 2

In subsequent chapters we will look at some concrete instances of responsive envi­

ronments and responsive media designed expressly for performative play.

TGarden
The TGarden constituted a metaevent in which the staging of the installation and its
substrate technologies of performance supported the contingent improvisation of
gestures that were of common concern. The substrate technology's latent field of
potential behavior was not at all random, nor was it fixed to a particular decision tree.
Rather, it was shaped by the authors as a particular high-dimensional landscape of
potential tendencies. The consequence was that a TGarden and its descendants did
not presume the objects of a democracy whether wild or toy, but sustained the pro­
cesses by which an assembly of human and nonhuman agencies brought things of
common concern into being, using gestures that themselves were improvised from
contingent conditions as well as habits and tacit knowledge sedimented from prior,
felt experience. We can build technologies of performance to make possible such
metaevents as arenas of care. We will return to this, but need to develop some concepts
and modes of articulation along the way.
3 Performance in Responsive Environments, the Performative Event

One of the principal motivations for this book is to articulate the turn from technolo­
gies of representation to technologies of performance in ethico-aesthetic as well as
technical terms. Slipping the conceptual and operational constraints imposed by sub­
scribing to one representational scheme or another, I cast off from the shores of
representation. I first examine the substrate material in which representations take
shape, the paper and ink substrate of the writing, a less glamorous and more molecular
stratum than Friedrich Kittler's gramophone and typewriter machines. In subsequent
chapters, honoring the processuality (a more apt concept than temporality) of felt
experience, I develop a way to articulate material process as continuous, topological
dynamical processes, of material transformation upon transformation. The pOint is to
see how, without boxing ourselves in a priori through language, or its abstractions to
semiotics or logic, indeed without resorting to the logic of representation, we can open
up technology to intentional and prepared improvisation. This is what technologies of per­
formance enable. This sort of improvisation should be spontaneous but not random,
free but not careless, rigorous and supple. Although I will recognize desiderata conso­
nant with Grotowski's poor theater, by the end of this chapter I will arrive at a carefully
amplified concept of performance. Throughout this book, I have been careful to use
the term "performative art" to distinguish the events constructed in responsive envi­
ronments from the performance art events of the 1960s through the 1 980s, works by
artists such as Yves Klein, Vito Acconci, Hermann Nitsch, Chris Burden, Carolee Schnee­
mann, Yoko Ono, Joseph Beuys, Wolf Vostell, Allan Kaprow, and Gilbert and George.!
By and large those works start by assuming the person and the body of the artist-as­
performer as the central object of regard. For my purposes, what "performance" means
will have to make sense without a priori and arguably transcendentalist assumptions
such as performer, ego, and, even more fundamentally, object and subject. It will, to give
a forecast, entail some notion of articulation in dynamic matter.
To triangulate the field of discussion with some concrete cases, I will first describe
a couple of technical and artistic projects in which one can see how the analysis
becomes richer and easier after relinquishing the logic of representation in favor of
64 C h a pter 3

F i g u re 3 . 1
Geometer's Workbench, a multimodal, high-resolution gesture-oriented display mapping t o sym­
bolic algebra, numerical integration, and differential geometric structures . Sha Xin Wei and
Fran<;ois Guimbretiere, Stanford, 2000. Image courtesy of the author.

performance in an amplified sense. It is necessary to have some "worked examples"

ready to hand because, although technologies of performance can enable nuanced
improvisation, they do not ensure it. I draw on two widely distinct examples from
technology and art to give a strong depth of field.

Examples

Geometer's Workbench: From Notation as Recording to Writing as Performance

Why is it that when two students of differential geometry work together, it is more
natural to turn to a chalkboard than to a keyboard? How do we inscribe and work out
mathematics with marks on paper or on a chalkboard, and how is this different from
typing math in TeX? Is mathematical creativity mediated by freehand writing and
sketching in ways that are not captured by traditional keyboard-and-mouse writing
systems? How does computation,2 or more specifically automated symbol manipula­
tion, augment or degrade a writing system for mathematical work? What forms of
writing, computation, and symbolic encoding help or hinder the practices of invent­
ing new mathematical structures, making new conjectures, or convincing oneself of
the truth of a claim?
The Geometer's Workbench was a wall-sized display that acted like a magic black­
board on which people could draw shapes that could act as figures, diagrams, algebraic
operators, or " meta" gestures like selection or designation.] This investigation was
inspired in part by my past studies in geometry and subsequent work in scientific
computing and mathematical visualization. After preliminary studies in differential
Performance in Respon sive Envi ro n m ents, the Performative Event 65

geometry, particularly in minimal surface theory (variational problems arising from

geometry which called upon methods from geometric measure theory and partial dif­
ferential equations), I turned to a second line of work in the domain of scientific
computation. For the next ten years I created simulations of physics microworlds,
quantum cosmology simulations, pharmacological systems models, multidimensional
data visualization, and various tools for differential geometric investigation. Over this
period, I also followed the development of a few powerful computer systems for the
visualization and construction of topological and geometric structures, notably Geom­
View, Surface Evolver, and Wolfram Research's Mathematica. Using these mathemati­
cal systems and comparing them against many others, I noted how they helped or
hindered creative research in differential geometry.4 I wondered why computers were
by and large so unuseful for mathematical research. This prompted me both to enlarge
the scope of the technology under investigation and to employ more sophisticated
critical-analytic approaches to this practical problem: how to make technology better
serve the creative interests of researchers and students in a few areas of differential
geometry.
At first blush, the problem seemed to factor fairly straightforwardly into a few
technical issues: the speed of computer hardware, the accuracy of numerical computa­
tion (for integration of partial differential equations), and advances in special graphics
hardware and in rendering polygonal computer graphics models. But later on, other
problems and issues became more salient, namely the choice of programming lan­
guage, the representation of geometric structure, and the keyboard-mouse interface.
In fact, it seemed more and more obvious that computation-calculation over
floating-point arithmetic or integers-per se seemed not the right domain in which
to focus energies for the development of mathematical research software. Or at least
such basic computational needs were being adequately met by the state of the art, yet
did not touch the heart of what mathematicians thought of as calculating, as in cal­
culating with exterior differential forms, which is quite different from computer cal­
culation.s Along the way I maintained an interest in questions of language, semiotics,
human-computer interaction, and the technologies of writing and performance. These
studies gradually led me to ask whether numeric or algebraic calculation was in fact
the most salient or effective use of computers in mathematical and scientific work. I
accumulated a set of questions based on experiences from users and developers of
computational tools for geometrical research.6 Of course, economic constraints and
disciplinary biases against "brute force" calculation played significant roles in the
development and adoption of, and resistance to, computational tools for mathemati­
cal research. But there appear to be fundamental obstructions in addition to such
contingent forces. In order to throw some of those obstructions into sharpest relief,
it is useful to consider one of the most active and highly developed areas of contem­
porary mathematiCS, that of differential geometry, and study procedures and entities
66 C h a pter 3

that present some of the toughest challenges to putative allies from computer science.
Examining such concrete challenges motivates a reconceptualization of the philo­
sophical study of mathematical practice.
The continuous or infinite entities described in chapter 6 have posed severe chal­
lenges to philosophy of mathematics and to symbolic, social, and computer technolo­
gies designed along logicist and linguistic principles.7 Canonically, the technical
obstructions include: overly explicit representations;8 the expectation that they be
effective and computable; a reliance on a fixed set of primitive primitives;9 the assump­
tion that sets and operations are finite; and the restriction of structures to finite­
dimensional vector spaces or graphs. Elsewhere, I have described some of the logicist
and linguistic theories of language and meaning, and have offered an alternative
approach to them and to the finitist and fictionalist responses. lO Despite all these
philosophical and technological reservations about infinity, however, contemporary
mathematicians habitually and unproblematically work with continuous or infinite
entities, and seem able to form communities of knowledge practice, shared across
multiple cultural contexts, that are stable over multiple generations of researchers.
How is this possible? How do we work with and gain an intuition about supposedly
abstract things like a Riemannian manifold or a harmonic functionl l via manipulating
material marks? My observation is that mathematicians are able to do this by employ­
ing, with varying degrees of conscious design, extraordinarily sophisticated technolo­
gies of writing. 12 This generalized writing allows them to exteriorize their thoughe3
and then reabsorb it, a process which is essential for mathematical work. What sort
of geometric creation and performance can or cannot be supported in a writing tech­
nology that spans freehand sketching, manipulable graphics, text, symbolic computa­
tion, and simulation? How do we exteriorize thought even when it is about processes
or objects that could not be embedded in a naive or classical space-time?14 How can
we evaluate technologies of writing as they are used in alinguistic ways? I analyzed in
detail what differential geometric work could or could not be performed easily in a
hybrid medium provided by a generalized multimodal writing and sketching technol­
ogy. Focusing on differential geometry through the lens of practice, I examined alin­
guistic semiotic usage, and through the lens of phenomenology I examined how
geometers work with continuous, smooth, or infinite structures and processes.
Broadly speaking, three common ways to get a grip on a concept via representation
are to write text, manipulate equations, or draw sketches and diagrams. But how do
we get intuitions about intangible things by gesturing and working with tangible
marks: ink, chalk, hand gestures? How is it that we can work and develop facility with,
or intuitions about, intangible, imaginary things like a geometric object? Or are imagi­
nary things necessarily intangible? What really makes something tangible anyway?
Could a geometrical entity be palpable or tangible or material in any sense that a
streak of chalk is tangible? I think that answering such questions about mathematical
Performance in Respon sive Enviro n ments, the Performative Event 67

entities yields insight on the analogous questions regarding other creations of imagi­
nation, ranging from songs to digital video and interactive media. It's more striking
when we train our attention on such apparently quintessentially immaterial entities
as the differential geometric objects or processes which are the subject of mathematical
research.
In my study of differential geometers' practice, I analyzed in detail what differential
geometric work could or could not be performed easily in a hybrid medium provided
by a generalized multimodal writing and sketching technology. The critical part of
that project was informed by insights from literary and performance studies as well
as the mathematical sciences. The result of this practical and critical investigation was
a conjecture about how mathematical obj ects emerge as limits of perspectival moves,
consequently as both material and objective, that is, supra individual, entities. Mathe­
maticians can only grip their entities via notation, but this is commonly interpreted
as a consequence of the claim that mathematical entities are immaterial and indeed
transcendental. But everything shifts when the process of writing math-the gestures
of calculation, the inscribing of notation, the sketching, the diagramming, the posing
of conditions and implications, and so forth-itself a corporeal practice, becomes not
merely the transcription or the representation of mysterious and always inaccessible
objects, but the very stuff of mathematics.
This view of writing as the poietic processes of articulating matter that constitute
the very entities that the resulting marks supposedly "represent" has far-reaching
implications. Such an approach to languaging found expression in quite a distinct
context as an art installation, called Hubbub, lS built around public surfaces activated
by speech recognition and speech-inflected dynamic typography.

H ubbub
Imagine that as you chat with a friend in a cafe one afternoon, some of your words
appear proj ected on the tabletop between you, animated according to the sound of
your voice. If you shout, the glyphs appear large and heavy, if you whisper, the glyphs
j itter like leaves. After a while they drift and spill off the edge of the table. You realize
that glyphs tend to circulate around the surfaces of the cafe, bobbing and dancing
around each other until they disintegrate and sink away. You may play games by
dropping some speech into the current of glyphs and seeing whether they get carried
to a distant but intriguing table. The speech recognizer makes mistakes, but this only
adds to the appeal by making a game of phonetic chance. Errors also make the game
safe, since the sense of the glyphs is only indirectly matched to what you say. This is
not surveillance but a living shadow that visually echoes your speech. Speech assumes
some of the fixity of text and text some of the prosody of speech.
Hubbub is an investigation of how accidental and non accidental conversations can
take place in public spaces, by means of speech that appears as glyphs projected on
68 Chapter 3

public surfaces. Each Hubbub installation takes its meaning from the social space in
which it is embedded, so its content and interaction depend on the specifics of each
site. By making public surfaces register speech in this playful manner, we may heighten
the sociality latent in our cities' public places.
In more structured installations, we imagine for example one person speaking a
question that circulates and persists until another person leaves a response that bobs
near the original question. In this way, a Hubbub installation can support chains of
contrapuntal conversations over hours, days, or even weeks in a place sustaining com­
munity memory. Also, by restricting the vocabulary of the speech recognizer, we
simultaneously make it possible for people to walk up and speak freely, and also tune
the mapping of speech to text by sensitizing it to preprepared text such as a story, a
poem, or names and phrases that have symbolic charge for the locale.
Whereas the Geometer's Workbench was oriented toward expert work demanding
special knowledge internal to specialized domains of creative mathematics, Hubbub
was oriented toward public spaces created by public expressions and public lettering.16
Superficially, the Geometer's Workbench and Hubbub seem radically disjoint works­
technical computer-augmented mathematics versus public installation art. However,
they share a common move from computational technology as representation to
computational technology as part of living expression-Le., a technology of perfor­
mance. Their essences had nothing to do with the storage and retrieval of information .

Software Writing as Experience-Forming

As a third example, consider the experience of crafting software. In the professional
world of software engineering, programmers have long been intimately familiar with
the experiential distinction between working with compiled versus interpreted program­
ming languages. Nontrivial programs written in a language that must be compiled
take a long time (minutes or hours) to be processed from the forms of representation
readable by a human programmer into machine code that can be executed by the
electronic hardware. The flow of work is stop and go, with the programmer forced to
take a "coffee break" while waiting for the computer to check, compile, and assemble
his or her code. In an interpreted programming environment, however, every state­
ment as written immediately generates a computed result to which the programmer
can respond. Working with interpreted languages, the programmer intimately and
continuously intercalates his or her actions with the computer.
All these examples show the enabling technology used as part of performing actions
on the fly in what is experienced as a continuous flow of thought and action. The key
common feature to all these instances is that those continuous actions are never to be
repeated, just as Heraclitus said that one could never step twice into the same river. 1?
Performance may still seem like an inchoate notion in our discussion, but we can
note already that performance always occurs in a local (regional) situation, in contin-
Performance in Responsive Enviro n m ents, the Performative Event 69

gent conditions. I do not say "context, " because context connotes a finite, bounded
horizon of interpretation that may well not exist. After Heidegger we know that indu­
bitable, finite contexts of interpretation for a language game do not in general exist.
But given that performance always occurs in some contingent conditions, it is useful
to turn to a concept that is often associated with performance, the event.

Event

For Alain Badiou, the event e is characterized by ex {x E X, ex }/8 but this characteriza­
=

tion bears knotty problems. First, it is not clear what X means, other than as a cipher
for the world, or for all the ways that an event can be qualified, which, since events
can index other events, immediately raises the second problem: formal inconsistency.
Since Russell, we have known that a set that contains itself as an element is an inco­
herent concept. This is in the same class of incoherence as the Liar Paradox: is the liar
who says "I am lying" telling a truth or lying? To more deeply analyze how Badiou
tries to turn this paradox into an instrument of ontology would take us too far into
another proj ect.19 So, borrowing from a subsequent chapter, I will use for now a more
tractable notion of event as a compact occasion. But what are the limits of the per­
formance event? Yes, there are always boundaries, and in fact a performance's dramatic
power may well depend on the existence of perceptual, phenomenal, spatiotemporal
boundaries, as a kettle needs its vessel walls to build up steam to whistle. In a sense,
there is always a boundary, so long as any compact subdomain of the world is chosen.
The only way to avoid a local boundary would be to take the complete topological
space as our ambient set. In any case, we shape and craft thick experiences out of the
very substrate of experience, which we will understand from chapter 4 as the amalgam
of matter and energy and affect.
In The Timeless Way of Building, Christopher Alexander wrote that a room, or at
least a room with a living quality, is not just geometry; it is an event. Its physical
shape constrains the potential actions that humans can take in it, which in turn con­
stitute an event that is the union of shape, constraint, intention, action.
In order to define this quality in buildings and in towns, we must begin by understat: ding that
every place is given its character by certain patterns of events that keep on happening there .
. . . These patterns of events are always interlocked with certain geometric patterns in the
space. Indeed, as we shall see, each building and each town is ultimately made out of these pat­
terns in the space, and out of nothing else.20

The kind of events I address, the kind I am interested in making, are collective, cop res­
ent, embodied, and alinguistic sets of actions in a substrate of processes. These are
situations to which people are invited to be physically together, face to face, in short
copresent. This is a basic condition of theater, too, and distinguishes theater from, for
70 C h a pter 3

example, cinema or photography, in that the performer-actor-artist is in the same

physical place as the spectator-visitor, so that the spectator can in principle get up and
physically lay a hand on the actor to interfere with the action. This potential for
physical contact is a precondition for conducting phenomenological experiments
adequate to collective, embodied experience. By design, these collective situations
should involve thick experiences, with three or more participants, three to destabilize
dyadic pairing, with an eye to lowering the threshold to improvisation of being in
that space. I say "embodied" to emphasize that the fleshy bodies of the participants
essentially move and act together in the coconstruction of the event. Jerzy Grotowski
wrote that theater can be divested of any particular mechanical technology but
requires at the minimum an actor and a spectator:
By gradually eliminating whatever proved superfluous, we found that theatre can exist without
make-up, without autonomic costume and scenography, without a separate performance area
(stage), without lighting and sound effects, etc. It cannot exist without the actor-spectator rela­
tionship of perceptual, direct, "live" communion.21

To understand performative events, however, it helps to decouple the classical actor/

spectator roles from particular fleshy bodies and extend them to actor function and
spectator function. In life, the line between actor and spectator can be crossed at any
moment-anybody may adopt the disposition of an actor as an agent of change in
the event, or a spectator as a witness of the event.
Most critically, how can we make events that are as compelling for the people who
encounter them as theater ever was in the most powerful of events by Bertolt Brecht,
Heiner Muller, Pina Bausch, Sankai Juku, or Dumb Type in their day and for their
audiences? In a sense, this is a technical challenge, in other words a challenge to the
practice and craft of experimental performance, to what Peter Brook called Holy
Theater, as opposed to Dead Theater of typical rote and commercial performance, and
different from the Rough Theater of the street and commedia dell'arte.22 One of the
questions I extract from this very broad challenge is: How can we make a responsive
space and event within which initially accidental, unmarked, unrehearsed, ordinary
gestures can acquire great symbolic charge? These questions are practical concerns of
craft, and can only be answered or explored materially, bodily, in physical built spaces
and peopled events, but the way in which we have explored them in the lab is not
by making commercial shows but by doing performance research. We have made
installation-events that straddle the border between closed studio improvisation exper­
iments with special audiences and open performances with a public. As it turns out,
these questions, though they were forged in a precise context of experimental perfor­
mance research, resonate far outside the world of digital media art and performance.
They are informed by dance, movement, textiles, fabric, musical performance, and
visual art, but they also are impelled by a desire to embed such work in public space
Perfo rmance in Responsive Environments, the Performative Event 71

and everyday space. This is an aspect of the ethico-aesthetic adventure of the work
that greatly appeals to me and fellow experimentalists.
Now the same questions about the event also have a radical, microtextural inflec­
tion. Could technologies like computational media, real-time sound and video (re-)
synthesis, cheap hobbyist sensors, and the like be added to the mise-en-scene of
theater as Antonin Artaud dreamed, to extend the "theater of cruelty" in a way that
is relevant to us today? In his "First Manifesto, " Artaud proposed to expand each
element of theater's arts according to its potential transformative power: stage lighting,
language (intonation, symbolic code), musical instruments, set design, lighting,
costume, stage, propS.Z3 Artaud's "theater of cruelty" would create a theater that would
not drop out of our consciousness as soon as we've finished consuming it but would
transform those who encounter it as utterly as the plague. By "cruelty, " Artaud explic­
itly did not intend the meanness of human hurting human or animal, but the implac­
ability and indifference of matter to our human ego. Stone reSists, and a tree greens,
and software breaks regardless of what we say. If we desire matter to perform differently,
we cannot simply legislate or script it by brandishing a pen alone; we must also manu­
facture a symbolic material substrate that behaves differently from ordinary matter.
In what way could such an aesthetic challenge be met by finite beings, with finite
resources? My response would be: Yes, we have finite resources, but are we finite
beings? Granting humans infinity as Levinas did, the question refines to: How is it,
with finite traces, finite energy, and finite time, that we can make the moves of infin­
ity? There are many answers, one of which is Kierkegaard's assertion that there simply
exist knights of infinity. Actually this question is the same as: How can humans do
mathematics with infinite obj ects and processes using finite traces? Or, how can poets
speak of unbounded passion and history with finite signifiers?
For me, the subclass of events on which to focus attention is that of intentional
events in which improvisation, to be meaningful, is structured improvisation over
prepared and decidedly not random conditions. Near the end of the film The Empty
Space, Peter Brook described how actions, to have expressive power in dramatic event,
must be committed. 24 In addition to this imaginative commitment, some other quali­
fications, by a via negativa different from that which led Grotowski to his poor theater,
are that the participants must be physically copresent, the activities collective and
essentially alinguistic, and the participants' experience continuous. In the wake of
quantum mechaniCS, most non physicists forget or do not know that quantum models
are selected for agreement with their continuum limits, which I would recharacter­
ize as phenomenological limits to draw attention to the heart of the nonclassical
matter. This heart is observation and measurement that inextricably intertwine (via
an inner product in a Hilbert space) the observer and the observed.25 It's necessary
that a quantum theory agree with classical theory in their common scale of macro­
scopic energy.
72 C h a pter 3

Given that people have enormous amounts of sedimented experience with the
physical world, we can leverage such noetic experience by using quasi-physical models.
The ambient environment will be thick with media, filled with thick sound, thick
video, dense physical materials, so that people will live in a dense matter that responds
and evolves in the course of their activity. All of this activity can be conducted alin­
guistically without the need for spoken language. On the other hand, speech is not
prohibited; it's just dislodged from its throne in favor of a plurality of modes of coor­
dination among the bodies and media in the space, again as a way to estrange the
speaking subject, and render more prominent the material dynamics of the lifeworld
on the other side of the veil of the technologies of language.
By "thickness" I refer not only to perceptual thickness-density of video and sound
textures-but also to the rich magma of social, imaginative, and erotic fields within
which people play even in ordinary situations, situations in which we perform without
first analyzing and cutting up our experiences into analytic layers: How did I smile?
How did I rest my feet on the floor? Did my voice carry or resonate well? Did I stand
too close to or too far from other people? Did I interrupt or listen or talk over or under
other speakers? Is the light too bright? Thickness also, and by conventional measures
more consequentially, connotes historical contingency in all its depth and density.
I say "thick" mindful of Clifford Geertz's sociological and anthropological approach
to describing culture in all of its rich social patterns and dynamics without orthogo­
nalizing it a priori into categories and schemata that we would bring to bear on that
culture. The dynamical potential field of experience should be designed in a preor­
thogonalized way by the composers, and enjoyed by the participants without requir­
ing that they make any cognitive model of their world in order to perform in it. Why?
Engineering's power derives from the portability and extensibility of standardized
schemas and methods that apply globally over phenomena and life. Our engineered
systems are already built on taxonomies that must be navigated by grammars and
operated according to rules that discipline our thought and action-the action of
power to discipline humans into docile bodies has radically evolved under the impact
not only of the informatic technology but the epistemic matrix that encases our
imaginary. These taxonomies rest on fundamentalist distinctions such as signal versus
noise, functional versus aesthetic, and syntactical versus nonsyntactical (relative to a
grammar) . It's not enough to side with noise as the opposite of signal, or idleness (the
vacation) as the opposite of wage slavery, because that still leaves in force the distinc­
tion made by the relevant schema in power.
In the last chapter, we will take up the phenomenological implications for this
artistic and technical approach to performance. But now we take an extended tour of
a responsive environment built to sustain events with such aesthetic and experiential
qualities.
Performance in Responsive Enviro n m ents, the Performative Even t 73

TGarden: Toward a Poetics of Performative Space

Vision
In this section, I discuss a series of installation-events called TGardens. These were
tangible environments envisioned as physical spaces filled with computationally aug­
mented video, sound, and luminous material that responded to the improvised gesture
and activity of their inhabitants, called players. I conceived them as phenomenological
experiments in interaction and response, agency, and intention. By definition, Gedanken
experiments exist in thought, but they are performable in principle informed by mate­
rial, corporeal practice. By phenomenological experiment I mean imaginative proposi­
tions made material, designed without presupposing topologically compact subjects
or action sequences defined according to an a priori schema. I describe the architecture
of these performative spaces in enough detail to be able to address certain phenomeno­
logical questions about agency and the continuum of intentional and accidental
gesture in the dynamical substrate of calligraphic media without grammatical
superstructure.
In particular, the return to the performative and the embodied offers an opportu­
nity to reopen questions about the phenomenology of performance and about the
phenomenology and poetics of performative spaces that respond to the activity of
their inhabitants. These questions concern the thresholds of agency, gesture, and
intention without reference to a grammatical or rule-based superstructure. The TGarden
emerged from a conversation among members of an experimental art research group
called Sponge, founded by Laura Farabough, Chris Salter, and myself in 1997. We had
been building experiments exploring what I called deferred (delayed) agency and
quantum performance-performance at the threshold of perception.26 My colleagues
set a challenge to make our discussions about interaction and media tangible rather
than let them remain at the level of verbal theoretical discourse: to materialize some
of these arguments so that other people could encounter them as powerfully as people
have ever encountered theater. In order to understand the TGarden project, one
should bear in mind that it started as a poetiC response to a conversation extending
over several years among artists and theorists affiliated with the Interaction and Media
Group seminar at Stanford University, from which Sponge was formed. 'In 200 1 ,
Sponge realized a series o f TGardens i n collaboration with the FoAM art group in
Belgium and the Netherlands, exhibiting the installation-environments ultimately in
more than ten cities in North America and Europe.27
For the Ars Electronica Festival in Linz (September 2001), we staged one of the
instances of the TGarden-TG200 l-as a miniature theatrical event.28 Before entering
the heart of the installation, a visitor chose a sumptuous garment to wear from a set
of instrumented garments, each with a different strangeness. One billowed in douds
74 C h a pter 3

of fabric so that the wearer grew three times larger but no heavier. Another added an
odd elasticity to the wearer's body so she tended to flop as she walked. The visitor was
led into an antechamber draped in black curtains and was dressed by an attendant.
The attendant belted the pocket computer and battery around her waist and strapped
sensors of acceleration to her arm or chest. It could feel like a medical exam but with
a more erotic charge. The attendant told the visitor little about how to move but sug­
gested that when the visitor dons the costume, she assumes not only a new body but
also a new voice. The attendant told the visitor: listen, move, and attend to what is
happening as she moves. Each of these fantastical costumes served as a phenomeno­
logical experiment, de familiarizing the visitor's body so she could more readily impro­
vise gestures.
When a visitor walked into the installation, he noticed that there were a few other
people costumed unlike him. It was hard to distinguish some of them from the pro­
j ected visual textures sweeping over every part of the floor and the walls. As he moved
he left trails of image and sound behind him. The air was filled with a hubbub of sound.
Everything visual and auditory seemed somehow made by living processes, but he
could identify the entities that made them. The room bore aquatic kinematics, but
there were no identifiable creatures of the sea. (The floor was illuminated with pro­
j ected moving shapes and lines and textures by video projectors mounted 20 feet over­
head.) As he waved his arms he noticed, perhaps immediately, perhaps after a while,
that some aspect of the room's aural texture varied according to his movement. But
depending on the sensitivity of the visitor, it could take a fair amount of play to begin
to understand what was happening.z9 In a responsive environment like the TGarden, a
particular gesture may not always elicit exactly the same sound, and yet the effect is
reproducible. If a visitor can learn how to move to generate some desired effect, then
he can begin to write calligraphically and play as if he were "bowing" through the
medium, much as if he were dragging his fingers or limbs across materials like wool or
metal sheet or rubber.30 He can try to create his own "voice" out of the ambient sound
field as he moves and dances about. He improvises gestures that elicit meaningful
sound or image patterns and develops a personal repertoire of gesture and movement.
In a TGarden, each player was associated with his or her own set of computational
media synthesis processes, and the entire room was associated with its own process
as well. The entire room was treated as just one more player, but a decidedly nonan­
thropomorphic one. At the finest scale, the many streams of sensor signals were
deliberately designed to include both data from physical sensors (such as acceleration
forces) and numerically derived measures (such as energy or period) in the same pro­
cessing ontology, reflecting an agnosticism with respect to the distinction between
putatively internal and external sense data.
As a technical aside, the TGarden hosted a circle of concurrent activity: moving
bodies; camera, sensors, radio; softwear (electronically augmented clothing or acces-
Performance in Responsive Enviro n m ents, the Performative Event 75

sories); software instruments computing statistics, dynamics, visual and sound syn­
thesis; sound processor and interface, speakers, video mixer, projectors.
The visitor noticed that there were no well-defined objects in the room, but as she
played in it minute after minute, or day after day if she were to return, she learned
certain ways of playing that characteristically elicited more or less well defined entities,
whether they were acoustic or visual or socio-psychological obj ects. She could observe
more experienced or expressive players as they invented ways of playing and engaging
the responsive space, and learn from their more deft action and response. Most of this
intertwining could occur without verbal exchange. In the imagined ideal situation, as
one body passed other bodies, it would leave behind material traces of itself: shadow,
hair, echoes, and air currents. Even if one did not explicitly and actively acknowledge
a passerby, one's shadowing matter intertwined with the others' residues, conducting
material conversations in the wake of one's passage.

F i g u re 3 . 2
TGarden/TG200 1 , Ars Electronica, Linz, 200 1 . Nonprofessional visitors, costumes, wireless
sensors, camera tracking, responsive sound and video, speaker array, projector array. Image cour­
tesy of Sponge.
76 C h a pter 3

,,
In one concrete moment, in the TGarden video entitled "hopskip, 31 the rhythmic
beat of the background sound enticed the player to jump. The accelerometer values
mapped the jump to the 3D graphics, which in turn opened and dosed the winglike
pattern projected onto the floor. To the jumper, the pattern projected onto the floor
felt astonishingly like an elastic rubber sheet, even though it was merely light cast
onto an unyielding flat surface. Instead of Simulating physics of elastica, we focused
our engineering effort on optimizing the on-body sensor processing and reducing the
latency to the point where the size of the holes of the projected pattern seemed con­
current with the motion of the jumper's body. Thus, there was no need to simulate
the dynamics of a rubber sheet, because the dynamics came directly from the physics
of fleshy bodies under physical gravity. The player interpreted the dynamics as elastic­
ity that he ascribed to the projected graphics, which encouraged him to leap about
the floor as if it were a trampoline. Our lesson was to minimize the software modeling
to the "semantically shallowest" possible computation, and to reduce the layers of
computational processing to the minimum.

TGarden as a Phenomenological Experiment

One of the key experimental purposes of the TGarden was to explore how we could
make possible a compelling experience without relying on prescripted, linguistically
codable narrative structure.
Having described the aesthetics of the living experience of this performative space,
I pose three phenomenological questions:
(1) How can people coordinate transformative and compelling experiences without relying on
conventional linguistic categories such as verbal narrative? The technical analog to this
is: How can people create sense together in a responsive environment without resort­
ing to grammatical structures? This question may seem a purely technical concern,
but it has extensive ethico-aesthetic implications. For example, it impels us to seek
alternatives to procedural if-then logic and to the locally linear syntax of time-based
scripts and scores, induding patterns found in conventional genres of interactive art
and fiction. The material, one expects, makes a difference.
(2) How could people improvise meaningful gestures collectively or singly in an environment
that is as alive as they are, an environment that itself evolves over time as a function of its
inhabitant life? Interaction modeled on a particularly reduced notion of computation­
ally mediated action and response is a far cry from animism and alchemy. What I
propose to ask is how expressive gestures can be sustained in sensate and animate
matter, some of which may in fact be computationally animated.
(3) How could objects emerge continuously under the continuous action of inhabitants in a
responsive space? This question of novelty itself comes from a larger critique of tech­
nology, which I encapsulate in the motto "a rich but not complicated life, " with a
nod to Clifford Geertz's thick, preanalytic, preorthogonalized descriptions of the life-
Performance in Responsive Enviro n ments, the Performative Event 77

world in all of its nuanced fields and relations and influences.32 But instead of restrict­
ing ourselves to observation, in the studio-laboratory we attempt a potentially complicated
immanent practitioner analysis. So in the Topological Media Lab we have built ele­
ments and techniques of responsive environments that can serve as apparatuses for
exploring phenomenological questions in live experiments.
Humberto Maturana and Francisco Varela observed that a continuously self-reproducing
autopoietic system cannot draw an obj ective distinction or operational boundary
between exterior and interior stimuli. As Maturana and Varela were generalizing from
nervous systems and cellular organisms, it seems that their observation should pertain
to any autopoietic system, which the TGarden was designed to approximate. Therefore
the TGarden's creators and players were by design and in practice themselves
participant-observers of their responsive play spaces.
The significance of these three questions about compelling nonverbal play, impro­
vised meaningful gesture, and the emergence of objects from fields is not confined to
theater or experimental performance alone. Nor are the questions merely technical in
the sense that they only help the professional performer or creator of performance
spaces ply his or her craft. I believe that drawing from performance practice conversely
refines philosophical questions about gesture, agency, and materiality.
In the course of building a TGarden that materialized the phenomenological inves­
tigation, we uncovered a number of technical questions, of which I will discuss three:
How can voices be mixed and a causally individuated voice be foregrounded? How
can multiple player agenCies sum together? And how can the responsive environment
detect the intent of a player? I discuss these questions because they constitute precise,
concrete entries into the phenomenological experiment that a TGarden was intended
to sustain.

M ixing Voices
One of the TGarden's purposes was to explore the erotics of the formation and dis­
solution of bodies from continuous fields of movement, sound, air currents, and video
as textured light. Early on, the creators decided that resynthesized sound, b�ing quint­
essentially temporal, was an ideal medium within which to blend multiple, "voices"
and sonic textures, so that the movements or gestures of a player would tease out
traces in the sound field that the player might associate with his or her own voice.
But since sound is an additive medium and diffuses around obstacles, superposing
sound works only too well-multiple sonic elements blend into a single field of sound.
Similar attempts to match sounds with individual players in a responsive space typi­
cally run aground on the same problem: How can players, the subjects in a dynamic
field of audio that they cocreate with the music synthesis software, distinguish their
own voices in a field of mixed sound? The naive approach would be to assign a pitch
or a rhythm or some basic mechanical musical parameter to each person. But this
78 C h a pter 3

suffers from many problems. For example, fixing a basic musical parameter like pitch
flattens the rich potential melodic trajectory that could be nuanced by a gesture.
Another problem is that, even if fixed, such obvious or "natural" qualities as harmonic
key, pitch sequence, signature melody, or acoustic icon quickly become impossible to
remember or to pick out from a mix of three or more people, unless they are so reduced
as to be boringly simple. The creators designed the TGarden environments for three
or more copresent human players in order to destabilize social dyads and leapfrog
communication theory's dyadic paradigm of atomic sender + (message in channel) +
atomic receiver.33 In any event, the engineered system still must have its sensors prop­
erly tuned34 to local physical conditions in order to parameterize the responsive sound
synthesis instruments that were created by Joel Ryan and Chris Salter. Of course, much
more experimentation remained to be done.3s The creators deliberately avoided wire­
less speakers and microphones worn on the body because, given the constraints on
the budget, labor, and wearability, the available technology was too coarse for our
musical ambitions.36 Poor sound production makes the game of disambiguating voices
from a mixed dynamical sound field that much harder and unrewarding.
To coherently forgo a priori obj ects of all kinds, including prefabricated visual
images and sonic elements, also implies that we should have no predefined narrative
objects like characters or voices or even melodies. Performance has come a long way
from Pirandello's " sei personaggi in cerca d'autore" (six characters in search of an
author) : not only have we displaced the authority of the composer by the distributed
agency of live performers and by software logic; now we have even rearranged and
reconfigured the physical and phenomenological locus of perceiving, sensing, listen­
ing. Where do the patterned sound and light come from? Where are they produced?
How are they produced, and in response to what gesture? In the TGarden a player
fashioned her own sound out of the total sound field rather than selecting a sound
sample that was recorded or synthesized prior to performance. Generally, instead of
triggering prefabricated media objects, a player fashioned her own dynamical media
pattern out of a tissue-a " stoffa" or stuff-that was an amalgam of sound, video,
fabric, and flesh evolving in response to her contingent activity as well as pre composed
autonomous dynamics. But swearing off a priori objects does not mean that no object
can emerge under the impact of the players' activity, because a responsive space sus­
tains the nuanced play of emergent pattern and structure. This stuff of a performative
event is the ontogenetic material substrate proposed in chapters 4 and s .

Summing Agencies
A second basic technical problem can be introduced via this concrete example: Suppose
we project onto the floor, from a single fixed projector, a video texture that is param­
eterized by an individual's actions. Suppose one person is "followed" by a spot of
proj ected red light and a second person is followed by a blue spot. If these two people
Performance in Responsive Enviro n ments, the Performative Even t 79

arrive at the same location doing different things, what color disk should be projected
on the floor "in response" to these two people? That there is a single fixed projector
implies that it is the software logic that must decide what color to synthesize for the
jointly occupied piece of the floor. In other words, one needs a logical model that
accommodates the physical superposition. If the logical model is not constructed to
provide for superposition, then the system will either produce blue or red or some
indeterminate result, which used to be programmers' laconic j argon for crashing the
program. There is no performatively convincing definitive answer to this conundrum.
In this case, a heuristic comes to our rescue: focus on transformations rather than
objects. Favoring transformations means in this case that we apply visual operators
(such as "lenses" that burn in, or "hammers" that optically crack whatever image lies
underfoot), operators that are parameterized by the activities of the people. It is much
more sensible to parameterize visual operators by activities and add operators together
by applying them to a common set of bits in the video stream.
Let me offer a less minimalist example. Suppose one person who has been in the
room only a little while or who has reverted to a "naive" set of gestural activities is
associated to an operator that rubs aside the video to expose a different layer of video
below it wherever the person is standing. Suppose that a second person is able to cause
whatever video is visible under her feet to swirl with a torsion that is proportional to
the bend of her arm. Then when both people are standing close together, they would
see the video in a revealed layer, swirling. Of course, the order of application-expose,
then swirl, versus swirl, then expose-is important, but such logic becomes part of the
composer's art.3 ? But the most significant development in the art of creating responsive
environments is the phenomenological, not epistemic, shift of focus from the aes­
thetic design problem of the legibility of the mark to the intention of the mark
maker.38

Detecting I ntention
Now this algebra of transformations naturally raises the question: How does the system
know what the player wants to do? A pinch of philosopher's skepticism can save a
large amount of engineering. The nub of the problem is that even we humans cannot
uneqUivocally distinguish intentionality from contingency. Moreover, we cannot
unequivocally distinguish lies, quotes, citations, or ironic actions from one another or from
"authentic speech " using formal means. After Derrida's and Wittgenstein's interrogations
of signification and meaning, it is no longer tenable to defend such distinctions even
in principle. Therefore, it seems that we may as well deploy our engineering resources
in less cognitivistically ambitious ways. We expand on these points by placing human­
computer interaction on stage for a moment.
Paradigmatically, with a well-designed interactive system if you push button A and
get response X, then pushing button A again should elicit the same response X or
80 C h a pter 3

some mechanically obvious successor to X (like increasing volume, or switching a

device on and off) . This paradigm of interactive design may be useful for utilitarian
tasks or simple games of habit, but it rapidly grows stale in a performative setting. No
matter how much craft is invested in creating a prefabricated piece of sound (or
image), even if the first time you make a gesture you enjoy the crafted sound, that
response will become boring if you hear essentially the same sound every subsequent
time you make the same gesture. In fact, I would say that such a predictable response
is not making sound but triggering a sound effect. There are multiple ways to fruitfully
complexify the response.39 For example, one can have the software program spawn
complex effects like showing a video or moving a robot arm according to an internal
clock. But that is formally equivalent to the program acting according to a unidimen­
sional script, a generalized timeline. Another canonical technique is to use procedural
rules of the form "If a parameter A satisfies condition C, then do B . " However, such
techniques set us on the slippery slope back to scripted, alphabetically encoded verbal
narrative with all of its commitments.4o Another way would be to randomize the
response to some degree. We could debate whether nature truly is random at heart,
but that misses the point for performance research. We should remember that half a
century ago John Cage and the Oulipo conducted their most interesting experiments
with chance in order to question the locus and role of intentionality in the player,
the spectator, and the system. One of the art collective Sponge's working heuristics
was to set aside the use of the random in its compositional process because, although
randomness may be metaphysically interesting, it does not yield any insight on how
an intentional nonrandom gesture can be artful.
Rather than playing back prefabricated media obj ects triggered according to scripts
or chance, the Ozone media choreography machinery responds like a set of musical
instruments responding to continuous gestures, allowing the player to calligraphically
brush or violinistically bow through the media. Now a substantial technical problem
comes to the fore. Suppose swinging my arm across my thigh pulls a melody out of
thin air. I might do that by accident because my arms swing of their own accord as I
walk. How would the system know to distinguish between me swinging my arm oblivi­
ous to its effect on the sound environment, and me swinging my arm intending to
pull a melody out of the air? In fact, this conundrum challenges not only artificial
intelligence but also generally what was called philosophy of mind. How can a person
or a machine distinguish an aCCidentally made physical movement from the same
movement made intentionally? It seems that we cannot unequivocally make such
distinctions among attitudes and dispositions with identical behavioral data.
Given that neither philosophy nor its would-be inheritor artificial intelligence has
solved the problem of operationally and mechanically distinguishing the intentional
from the accidental, I propose that we design our responSive systems to not make any
semantic model of the user's intent. (By semantic model, I mean a rule-based logical
Performance in Responsive Enviro n m ents, the Performative Event 81

and formal system that is populated using the grammar and syntax and the lexicon
of an ordinary language.) In fact, I propose the heuristic that the software make no
semantic model whatsoever of any "high-level" user state. This means that the com­
putational part of the responsive system should not attempt to model human experi­
ence in terms of cognition, or social experience in terms of information, as is often
done in the engineering of so-called interactive systems. An interactive system predi­
cated on the dyadic paradigm of turn-taking "communication" omits most of the
concurrent density of a live, performative event. For this reason I characterize TGarden
as a responsive environment in which all material patterns costructure each other
concurrently.
But it gets worse. An attempt to model the user with so much semantic, psycholo­
gistic, or cognitive elaboration is an instance of what one could call a "correspondence
error" : claiming that a system of representation corresponds to or is deterministically
coupled to some objective entity "in the real world. " After all, a violin does nothing
like what artificial intelligence experts would like to build. As you draw a bow across
the violin or blow into the clarinet, the instrument does not "decide" or "infer" your
degree of virtuosity and change some part of its structure to write a datum labeling
you as "novice/amateur/virtuoso" or "happy/neutral/sad . " The wood of that violin
vibrates according to the same physics whether you are a beginning student or a
concert virtuoso . And should you draw the bow in the same physical manner as Anne­
Sophie Mutter for some lucky duration, you and the instrument would produce the
same sound as Mutter would (though it likely would not be the same music) . Begin­
ner's luck, they call it. (Indeed, this is how a human performer develops virtuosity in
a TGarden, by playing through computationally augmented physics.) Even if one uses
such models without believing in a correspondence, one reifies entities that progres­
sively bog down or schematize expression as they accumulate transcendental status.
In the TGarden, we set ourselves the challenge of creating quasi-physical systems41
that provided enough richness and depth of response to always make experientially
distinguishable and potentially interesting responses to the player's action and move­
ment, no matter what the player did. No matter how you drag that bow across the
strings, the violin will sound. It was up to the player to make "strokes" and create
gestures, to develop facility through continued practice and to inscribe or ascribe
meaning by shaping material fields.
I have written about the relation between gesture, agency, and materiality else­
where, so let me make only two critical comments here about the consequences of
these relations for gesture in the TGarden.42
Given the tangible, responsive media of the TGarden, we can begin to understand
how gesture conjures the self and how collective gesture conjures the social. One of
the TGarden's principal motivating themes was the dissolution and reformation of
bodies in a field. When this field is a social field, then the act of gesturing becomes a
82 C h a pter 3

way to shape intentional being in the world from a state of nonintentional distraction.
At a larger scale, since our gesture is conditioned by birth, habit, and culture, gesture
entangles social history with the body in action. Not only our own personal histories
but also the habits of generations sediment into our own bodies as disCiplines that
fluidly scaffold our gestural expectations, anticipations, and intentions. The technol­
ogy of performance allows us to play most tangibly with such processes of
individuation.
One of the technical interests of the TGarden was to study how people could
improvise gestures meaningfully in a media-rich space that evolved continuously in
response to their activity. We built a performance space with instruments that impose
no syntax on the player's gesture. Without syntactic constraints, there are no wrong
movements, and every movement "does something. " In place of syntax and grammar,
we built a responsive environment that tangibly connected people's gestures and
movement to one another and to the environment: every glide, every stroke, every
slip and slide stirs media processes in concert with the physical material world. In a
deep sense, it is the topological continuity and density of the TGarden's material
media processes that enabled improvisation and performatively rich nuanceY
This continuity has strong phenomenological consequences. Continuity is a leit­
motiv of topological media and the heuristic lens into the full, thick dynamics of our
embodied experience. As you sweep your arm, it moves continuously through the air.
As you walk to your friend to greet her, your consciousness has no gaps. In everyday
experience, your existence appears to have no gaps. As human experience is dense and
continuous, a TGarden environment sustains playfully intensified experiences that are not
complicated but rich.
Since we TGarden composers wanted to sustain such phenomenal density in our
own play space, we made software media engines that synthesize time-based video
and audio. These engines, especially the sound instruments, allowed players to dis­
solve, reconstitute, and shape perceptual entities under the impact of their individual
and collective activity. Making a media engine that evolves continuously also radically
reduces the complexity of the media elements that need to be assembled for produc­
tion, because media can be synthesized afresh in response to the activity during an
event. In fact, we prefabricated relatively few media objects (Le., video or sound files)
for TG2001 , because we only needed them as initial textural material to seed the
processes that resynthesized dynamic fields of sound and image in real-time
performance.

Applications

I have described this approach to the event dynamics in detail because it represents
a strong alternative to the usual computational augmentations of conceiving and
Performance in Responsive Enviro n m ents, the Performative Even t 83

conditioning events in a responsive environment. To recapitulate, conventional

mapping logics include ( 1 ) fixed timelines, such as cue-based systems, which impose
a fixed, predesigned sequential order on media objects and events that can only be
repeated; (2) if-then logic, which tends to be brittle and hard to revise on the fly; (3)
stochastic methods, which provide a degree of unpredictable variation, but with no
ethico-aesthetic sense to the distribution-or rather a very particular one: that of the
random; (4) written scripts, which permit a great deal of interpretation by persons
inhabiting the event, but not by the costructuring software or material.
Beyond the motivating TG200 1 and txOom applications, we have exercised differ­
ent components of the Ozone system in the following installations and events: Meteor
Shower, 2006 (sound, video from particle systems driven by video-based motion track­
ing); Cosmicomics, 2007 (public installations with responsive sound and video, and
linear states); Remedios Terrarium, 2008 (public exhibit of complex self-interacting
sound, video tracking, and state); and Frankenstein's Ghosts, 2008-201 0, a perfor­
mance research proj ect augmenting the Blue Riders chamber music ensemble and
dancers' movements with responsive sound, and the IL Y A video membrane.44
The Ouija experiments centrally inform our later applications of this continuous,
state-based approach. Ouija is simultaneously a phenomenological experiment and a
movement exercise about gesture, intention, and collective versus individual agency.
The two questions we ask are:
( 1 ) When is a gesture intentional versus accidental? How can the system-the people
as well as the media-evolve accordingly?
(2) When is a gesture individual to a particular body and when is it collective? And
how can the system evolve accordingly?
Context, history, expectation, and intent all condition these questions, so the answer
does not lie solely in the sensor data's instantaneous values. All the agencies in an
event-the human actors and spectators as well as the computational media (the real­
time sound, video, and light field, etc.)-respond to each other concurrently. As with
other Ozone applications, Ouija focuses primarily on the first-person experience of
the mover, rather than third-person spectator experience.
In 2007, choreographer Michael Montanaro45 designed a series of movement sce­
narios that explore these questions.46 In one experiment, called Entrainment, eight
visitors and actors (rehearsed with the responsive media effects) gathered around a
large table set for tea, to make conversation. The visitors were not told who were the
actors. After some warm-up, we asked them to stop using verbal language but continue
the conversation gesturally, which encouraged improvisation. We instructed the actors
to gradually amplify their gestures by making them more and more "acted" using
prepared repertoire. Over the same period of time, we introduced sound responding
to data from a contact microphone attached as a sensor under the table. Later, we
84 C h a pter 3

projected video fr om overhead onto the table that slowly transformed from homoge­
neous illumination to chiaroscuro textures that registered the passage of limbs over
the tabletop. Under these conditions, we witnessed the construction of an entirely
nonverbal creole that seemed to make sense to the participants but not to the
observers.

I n sertions and Eversions of the Responsive Environment's Technical Objects i nto

Less-Controlled Environments

The TGarden genre of responsive spaces presented entire environments staged in a

" sacred" space like a theatrical black box. But one of the ongoing challenges has always
been to see whether similar enlivening effects can be achieved when elements of
responsive media are embedded in the everyday built environment.

Sauna
One way to do this is to evert the environment, as was done in the course of Sponge's
Sauna experiments on immersivity. The phenomenological questions included: What
are the essential elements of immersive experience, and what could experiential varia­
tions of those elements in different ambient conditions yield toward experiences
hovering on the phenomenological knife edge between Geworfenheit (thrownness) and
reflection? For the first version we built a "cargo cult" version of an immersive
chamber, stripped down to a cylinder for a single person wearing headphones. For the
latest version, we created cylinders acting symbolically-materially as media ramjets
that processed and transmuted the fields of sound and light from the ambient urban
environment. We suspended three of these large open tubes from the second-story
exterior walls of a building (The Lab) in San Francisco where they functioned as reproc­
essors of ambient activity, projecting the transmuted media onto the sidewalks below.
Virtual reality systems have striven to replace every single atom of a human's percep­
tual field by synthetically created media. Instead, we explored whether it was possible
to create immersive experience without isolation and containment, by insertion into
an existing urban fabric, with all the attendant distractions, accident, noise of everyday
urban activity. People were not physically contained in a sacred chamber but could
walk freely down the Sidewalk, subj ect to all the other stimuli of the world.
In sum, I would say that while the "cargo cult" immersive tank achieved the desired
effect of suspension, the exterior, everted version was not as successful, and for inter­
estingly mundane reasons. For security and safety, we had to suspend the cylinders
out of reach of people walking on the Sidewalk, which resulted in cones of light that
were quite weak relative to all the other lights on the street. While the strobe lamps
created an effectively intense luminous region in an indoor study,47 outdoors they
were too far above the sidewalk to adequately "heat" the areas of the sidewalk onto
Performance in Responsive Enviro n m ents, the Performative Event 8S

Front elevatio n S i d e elevatio n

F i g u re 3 . 3
Diagram o f a n IL Y A membrane installation: double-sided screen, two live video streams,
1 2-channel sound, responsive video and sound, state engine. Illustration by Harry Smoak.

which they projected. The sound, however, seemed to work much better, as reproc­
essed and subtly reinforced spectra from the ambient street sounds.

Membranes, IL Y A
Another, related technique for deploying media choreography techniques into the
everyday environment is to insert media filters. Think of a filter not as the semiotic
object (Le., as a visual or sonic representation) but as an operator transforming what
material-light, sound, pressure, fluid, etc.-flows through it. An ordinary pair of
glasses does this, as does a stained glass window.
A membrane is a double-sided video display that transmutes what people on one
side see of the other. A membrane is not a mirror but a lens that can transform people's
views of each other and of their surroundings according to their own movement. The
computer processes live camera input into video which is then proj ected back to the
other side. A spatialized sound field computed from present and past move� ent gives
palpable weight and friction to the action. Unlike most media art obj ects, a �embrane
does not exist until it is observed simultaneously from two distinct perspectives. By
deSign, a membrane does not draw attention to itself, but through itself to whatever
is on the other side. In this way a membrane entangles people with each other, a
particular material, embodied, continuous mode of making and unmaking things of
common concern in an open, collective space.
IL Y A48 was a particular series of membranes created to explore the emergence of
social density in a collective space, and to celebrate the indirect and unknown conse­
quences of the other's action-unknown but not unfelt (in Eugene Gendlin's sense of
felt meaning) . Your movement distends what you see of the other side like smoke or
86 C h a pter 3

Figure 3.4
IL Y A membrane installation: "glass" state. Photograph courtesy of Jerome Delapierre.

other pseudo-physical matter stirred by the viewers' own movements. The effect was
symmetrical-any movement by the other reshapes your image as well.
When a person vacated one side of the membrane, IL Y A substituted historical
footage of people from the past. These historical ghosts' movements affected the video
just as the movement in the live video. IL Y A also was symmetrical between the past
and the present: as you became still, the figures of the dead would reappear and rein­
habit the present. In fact, their movements and gestures would drag and perturb your
image as well. Moving bodies from the past could act on your image just as you could
act on the bodies of past or present others. Since the effect was symmetrical, people
past and present intertwined and could transmute each other's bodies with dynami­
cally shifting but equal agency. When no one at all was in the room, the membrane
displayed only historical documentary footage of the populated site, before living
memory of that place. Using historical footage of activity local to the site, IL Y A acted
as a lens into the past as well as the present of the given site.
IL Y A could be installed in a gallery as well as an industrial or public site, localized
with images from the site's historical past. By taking this anthropological and ethical
stance, IL Y A constituted an instrument concerning the material and architectural
substrates to sociality. Precisely, it permitted historical past bodies to play with present
living bodies, and permitted living bodies to play with each other. This made the
Performance in Responsive Enviro n m ents, the Performative Event 87

dead's activity more symmetrical with the living via real-time software calligraphic
effects on the streams of live and recorded video. In the Van Nelle Fabriek Membrane
study exhibited at the Dutch Electronic Art Festival in Rotterdam in 2004,49 the state
evolution software rocked the agency of the effect back and forth between the two
sides of the membrane according to the relative activity on either side. We can think
of the IL Y A installation as a time lens that allowed us to gesture our way into deeper
and deeper layers of historical time. I say historical time, because these IL Y A instal­
lations let the visitor fall from deep past into the present, and back again. In this way,
IL Y A intertwined corporeal activities that occurred in the ambient space before living
memory.
Moving past the critiques of models and categories, IL Y A helped bodies and
persons entangle and reenchant the world, our world. By inserting membranes into a
space flowing with people from different eras, we explored how by changing the optics
a social space might catalyze our playful care for each other. IL Y A and related respon­
sive video membranes were predicated on a radical notion of the other, divergent from
a psychoanalytic ego or a transcendental subject, a radically lower-cased "other" that
is constituent and immanent in the value-producing substrate magma of the world. so
The common theme throughout this chapter has been how performative activity
can be regarded as the articulation of matter in dynamical processes of sense making.
This shaping of matter as physical, affective, symbolic material in a rich magma of
process constitutes events. In order to understand this we conSider, in the next chapter,
a material field-oriented approach to stuff and transformations of stuff.
4 Substrate

9w.,ucrcru OtUXEE'tat Kat Il£'tPEE'tat Et� 'tOY umov AOYOV OKOtO� npocr9Ev TJV v '/EVE<19at 'Y'l .
The earth melts into the sea a s the sea sinks into the earth.
-Heraclitus, fragment 23

In chapter 2, we reprised salient concerns about the limits and powers of representa­
tion. I advocated shifting our technological perspective from the technologies and
sciences of representation to those of performance, by which I intend improvising and
designing continuously in continua of symbolic, embodying matter. To elaborate what
that means will require the content of chapter 6 to flesh out what we mean by the
continuous. For now, let us use continuity and continua in their intuitive senses,
seeing how we might make sense of matter as living continua versus matter chunked
as living and inert objects. I use "living" as a compact term linking Maturana and
Varela's autopoiesis with Whitehead's processual notion of concrescence. More fully
justifying the link will have to wait till chapter 5, but for now I will emphasize that
we should always think of our continua as being in continuous variation.
My aim is not to "explain" what the world is made of, or how the world works, or
how humans work. For that we should go to metaphysics, physics, cosmology, psy­
chology, economics. Nor is my aim to explain to you what you must do to be good
in the eyes of the gods. For that we could go to a church. Consider an analogous
distinction between Taoism and a church-based religion as social and epistehIic forms.
Zhuangzi's parables ripple the smooth surface of thought, and discontent , rational
judgments but do not supplant them with an alternative rational judgment. The verses
of the Dao De ling do not work as dogma or scripture or moral stricture; their power
is that of poetry. And although people appear as pedagogical devices in these stories,
Homo sapiens is not the poetic subject. So although I have drawn much from phenom­
enological thinking, it should be clear that I do not pursue the Cartesian thread that
is shared by Husserl, Heidegger, and Merleau-Ponty, ending in the present day with
the cognitivists who have solved philosophy. My ambition here is not to settle old
scores in metaphysics but to suggest an expressive mode of articulation that will
90 C h a pter 4

0)
Figure 4. 1
Field of superposable vortices, after David Bohm. Diagram by author.

accommodate poiesis and enchantment, by which I mean the transmutation (not

transubstantiation) of material. Let me underline that I am not offering a new truth
overturning yesterday's truth. Instead, I am poetically articulating the world as dynam­
ical plenum, a conception that is as ancient as any other.
By the way, I do not object to obj ects or deny that they exist. Briefly put, my purpose
is not to classify obj ects or their dual subj ects, but rather to offer an approach to
articulating objects as they come into being, as they emerge from continuous and con­
tinuously varying fields of media-material and then dissolve again into those fields.!
Such an approach needs to move carefully to avoid appealing to a priori objects, a
challenge even more difficult than inventing a model-free "learning" algorithm, which
is an epistemological project rather than an ethico-aesthetic and material proj ect.2 This
articulation should accommodate suppleness, nuance, fresh 3 expression.
In this chapter, and throughout this book I will use media and material interchange­
ably to lexically remind us of the materialist and topological thrust of this project:
that media is material as much as material is media. To reiterate: material for me is
an amalgam of matter + energy + affect. (This is a creative axiom like what Stengers
meant by "propositional. " ) The middle term, energy, is more than the energy of
modern (nineteenth-century) physics, since it can be constituted of money in flux or
force x time or other types of energy, depending on how we are attuned. Media for
my purposes is the mode in which language resonates materially and corporeally, but
this asignifying mode, to use Guattari's term, works below the level of meaning. If we
take an interest in materiality via media, then it makes sense to look for a variety of
ways in which media acts materially. This motivates creating computational media,
because the computational affords boundless and intricate ways to construct media with
experimentally different sorts of behavior than what one expects from noncomputational
media like water, wood, tissue, and sinew.
For my purposes, an essential aspect of materiality is its temporality. Consequently,
understanding what we can do with time-based computational media should afford
more ample insight about dynamical matter, or material dynamics, in general. One
strategy I take is to transmute questions about things, or more precisely monads, into
questions about stuff, or plenum.4 In particular, instead of puzzling about the tempo-
S u bstrate 91

rality of objects, I would ask what makes material material. Adapting Whitehead's
observation "actual entities perish"s to a plenist ontology, then, we should say "materi­
als" perish, which should seem no more and no less mysterious than saying that
yogurt has a shelf life.
In the next section, I present what is at stake, followed by a section summarizing
some problems with atomistic, object-oriented ontologies. Subsequently, I describe a
Heraclitean view of a continuum, a plenist world, and close with some implications.
Throughout I use figurative language to lay out tropes, lures for feeling, motivating
the much more precise modes of articulation that I will build in chapter 6.
Finally, if we want to answer the question What is an object?, we can readily set up
speculative experiments using computational media or computationally thickened
and nuanced physical events.

The Stakes

Before diving into the nature of this plenist orientation, let's take a moment to review
what is at stake with this alternative. Akeel Bilgrami laid out the stakes in an eloquent
passage that merits citing in full:
In the dissenting tradition-which was a scientific tradition, for there was in fact no disagreement
between it and Newton and Boyle on any serious detail of the scientific laws, and all the funda­
mental notions such as gravity, for instance, were perfectly in place, though given a somewhat
different metaphysical interpretation-matter was not brute and inert but rather was shot
through with an inner source of dynamism that was itself divine. God and nature were not sepa­
rable as in the official metaphysical picture that was growing around the new science . . . .
The link between Gandhi and the dissenters is vivid and explicit. One absolutely central claim
of the freethinkers of this period in the seventeenth century was about the political and cultural
significance of their disagreements with the fast developing metaphysical orthodoxy of the
"Newtonians . " Just as Gandhi did, they argued that it is only because one takes matter to be
brute and stupid, to use Newton's own terms, that one would find it appropriate to conquer it
with the most destructive of technologies with nothing but profit and material wealth as ends
and thereby destroy it both as a natural and a humanitarian environment for one's habitation.
In today's terms, one might think that this point was a seventeenth-century predece�sor to our
ecological concerns, but, though there certainly was an early instinct of that kind, it �as embed­
ded in a much more general point (as it was with Gandhi, too), a point really about how nature
in an ancient and spiritually flourishing sense was being threatened. Today, the most thoroughly
and self-consciously secular sensibilities may recoil from the term spiritually, though I must
confess to finding myself feeling no such self-consciousness despite being a secularist, indeed,
an atheist. The real point has nothing to do with these rhetorical niceties. If one had no use for
the word, if one insisted on having the point made with words that we today can summon with
confidence and accept without qualm, it would do no great violence to the core of one's thinking
to say this: the dissenters thought of the world not as brute but as suffused with value. That they
 flows

F i g u re 4.2
Real-time motion analysis with cv. j it Max/Jitter library: (A) live video of dancer, (E) optical flow
as gradient (difference) density, (C) optical flow as vector field. Note that in (B) hoth the body's
and proj ected image's movements produce optical flow.
S u bstrate 93

happened to think the source of such value was divine ought not to be the deepest point of
interest for us. The point rather is that if it were laden with value, it would make normative
(ethical and social) demands on one, whether one was religious or not, normative demands
therefore that did not come merely from our own instrumentalities and subjective utilities. And
it is this sense of forming commitments by taking in, in our perceptions, an evaluatively
enchanted world, which-being enchanted in this way-therefore moved us to normatively
constrained engagement with it, that the dissenters contrasted with the outlook that was being
offered by the ideologues of the new science. A brute and disenchanted world could not move
us to any such engagement since any perception of it, given the sort of thing it was, would
necessarily be a detached form of observation; and if one ever came out of this detachment, if
there was ever any engagement with a world so distantly conceived, so external to our own
senSibility, it could only take the form of mastery and control of something alien, with a view
to satisfying the only source of value allowed by this outlook-our own utilities and gain.6

In the Nature of Order, architect and prophet Christopher Alexander called for
exactly this sort of physics fusing matter and valu e a la Spinoza, rather than matter
formed only by geometry (Einstein) or number (Pythagoras) . My concern is indeed to
explore the qualities of matter construed this way-as laden with value. I transmute
Whitehead's axiom of process philosophy, "How an entity becomes constitutes what the
entity is, "7 to move from a concern about values of obj ects to concerns about value­
generating or value-signifying processes. Classical metaphysics oscillates between pre­
constituted subjects perceiving, reasoning about, and acting on pre constituted objects.
Sidestepping both realist and idealist theories, in this chapter I consider objects, sub­
j ects, values, and relations all coconstituting each other in the dynamic of the stuffs
of which they are made. One key feature of this account is plurality: there can be
boundlessly many fields of potential. Another is dynamism-perceived as pOiesis. We
will see how value can arise out of the superposition of dynamic fields without requir­
ing us to preconstitute particular subjects, or follow a totalizing telos. This relies on a
triple conceptual transmutation: ( 1 ) shifting from objects to material fields ("stuff"),
(2) shifting from obj ects to processes, (3) shifting from values as predicates to processes
that produce value. The three aspects of this transmutation will take us through our
discussions of the phenomenology of performance, substrate, and ontogenesis; this
chapter concerns substrate. Before we launch into the chapter, let me stat� the key
propositions succinctly. Until I tell a story in which these propositions tak� on life,
we should not expect their significance or relationships to one another to be self­
evident, but nonetheless I state them here as a map of what is to come. First, let me
adopt a notation: conventionally in mathematics, the sign "=/' means "implies. " Let
me read this sign to mean permits, allows, sustains, articulates.
( 1 ) A connected8 plenum is a condition of possibility for ethics.
(2) Field => potential dynamics. Field => multiplicity.
(3) Affective intensity constitutes value, and so is primordial (as substrate) to ethical action.
94 C h a pter 4

(4) Care, an affective field, nuanced by attunement and pathic sensitivity, is primordial
to politics, an ever-dynamical system.
(5) Continuity (in the space of variations of plenum) =:} nuance and poiesis: the textural
natality of material.

Atomism: The Problems of Complexity and I ntersubjectivity

There are formal and ethical problems with atomism. The formal problem is that an
atomic concept of the world sooner or later must come to terms with combinatorial
complexity. The actual world, with its perverse habit of constantly producing more
and more things over time, tends to outgrow every static categorization into a finite
number of finite categories, every synchronic schema no matter how carefully rea­
soned. From modern science, the periodic table and the taxonomy of animals and
plants epitomize this categorical approach. But why can the chemists and physicists
keep their periodic table so tidy when the biologists have had to revise their classifica­
tions ever since Linnaeus? The classification of life forms has proliferated whole king­
doms, even to the point of blurring boundaries between them and between life and
nonlife, a liminal region occupied by prions and organic molecules weighing six-figure
daltons. Of course, physicists have the lUxury of scoping their discipline to exclude
much of the richness of the material world, leaving for example the messy work of
rationalizing alchemy to the modern day chemist and astrologer. In fact, under the
impact of successive waves of industrially powered material experimentation and
innovation, we see a proliferation of categories of matter in modern chemistry: plas­
tics, pharmaceuticals, and now some nanomaterials being absorbed into this second
oldest of technologies (the oldest being cooking) . All these taxonomies, especially
those that maintain an ambition to categorize the entire unfolding world, share a
common strategy, which is to partition their categories into hierarchies. A hierarchy
contains towers of categories, where an element of a given category contains elements
from a subcategory. But this does no more than sweep the problems under the rug,
because in a given category we still face the same formal structure.
A word about discrete sets in computer science. Faith in the generality of discrete
atomism has been canonized in software programming languages from LISP to Java
in which a " set" is always defined as an unordered, finite list of discrete elements.
(ML, a rare exception, represents categories as first-order entities in the language, and
so can define a set without any concrete representative of that type.) A " set" character­
ized as an unordered list may seem like the height of unconditioned generality to a
programmer, but it can in no way encode even the unit interval [0, 1] of all real
numbers between 0 and 1 (inclusive) , much less the transfinitely larger set of all
measurable functions mapping the interval I to itself. (Chapter 6 will develop the
concepts needed to understand this more clearly.) "Arbitrary and finite" means that
although the formal structure does not impose any limit on the number of items, any
S u bstrate 95

instance of a "set" has only a finite number of items. "Arbitrary" just means that the
cardinality of the set is not a preordained constant in the formal scheme. An arbitrary
set can be finite or infinite. In a programming language, it is always finite, of course,
and in practice limited by machine memory. What breaks this formal finiteness in a
programming environment is the data " stream. " A stream of data is a pipe to the world
outside the program through which an arbitrary and boundless number of packets of
data can enter over time. A video channel that takes data of unspecified size from a
server can be represented as a stream in the program. Another example is a stream
kept open to pass data from a sensor. In both cases, the program opens itself to the
temporal: it enters the temporal world and waits for obj ects (packets) to be sent from
the world. So it is the temporality of the world, its ever-changing state, that breaks
the assumption of a formal, bounded, discrete, and hence finite data storage structure.
(We will return to this in chapter S .) Now, as everyone has been told for the better
part of a century, the computer's canonical atom has been the bit, which can take on
only one of two values: 0 or 1 . We can regard this as a special case of the generally
atomistic view of the world.
So, with this canonical example from the digital epoch in hand, let's turn to the
formal problem of atomism. Our world contains not only things but also relations of
things, so this induces a combinatorial complexity. Such complexity has often been
valorized as yielding phenomena emerging from large collections of discrete entities
in networks of relations modeled on graphs, phenomena that one does not observe
in an individual entity. However, as I have said before, combinatorial complexity does
not equal richness. Indeed complexity inevitably tends to overwhelm sense and value.
For the sake of completeness and clarity, let's consider the following exercise.
Suppose a discrete set S contains exactly N elements. One says that the size of S is N.
The set of all subsets of S, called the powerset of S, generally has larger size than S. In
fact, if S has cardinality N, then its powerset has size 2N, a much larger number than
N. If a set has ten elements, then its powerset has about a thousand subsets. If S has
twenty elements, then its powerset has more than a million subsets. In other words
the powerset of a set S is exponentially bigger in cardinality than S itself. Generally,
discrete structures exhibit this sort of combinatorial, exponentially explosive. complex­
ity as you add more elements, components, or dimensions to the structure. The same
is true of networks of discrete nodes and arcs. As these networks grow larger, we can
attempt to salvage the situation by aggregating subgraphs into nodes, but that merely
defers the explosion by one step. Eventually combinatorial complexity overwhelms
us. On the other hand, if we believe that human experience is continuous, dense and
rich but not combinatorially complex, then it should be a healthy challenge to
try to make our performance technologies themselves topological rather than
combinatorial.
So to account for relations among a universe of obj ects requires combinatorially
and exponentially complex structures. And one lesson we can easily verify from
96 C h a pter 4

collective as well as individual life is that exponential growth outpaces any finite
bound, in a shockingly rapid way.
Exercise: To get a feel for exponential growth, take a sheet of paper, of any size.
Fold it in half. Take this halved piece of paper and fold it in half again. Repeat. How
many times can you do this? How does this depend on the width of the sheet? Try a
larger sheet of paper, of any size. This is an exercise that you must do with your own
hands on a physical sheet of paper in order to appreciate. It is not a Gedanken exercise
but a material, corporeal one. Each time you fold, you are doubling the thickness of
the folded paper, which is an exponential increase in thickness: 2N x original thickness
of the paper.
So much for technical, practical problems with discrete sets and the combinatorial
complexity of atomistic representations. But more fundamental, conceptual problems
with an atomistic, obj ect-oriented ontology abound. For instance, it makes it hard to
account for change. Indeed, Whitehead's entities are "changeless, " so he needs to jump
through some hoops in order to accommodate the dynamic.
In fact, both difficulties are artifacts of the atomistic object-oriented ontology. They
go away under others.
There is a tendency to transcendentalize-to treat material as if it were abstract and
to dematerialize any concept from its material field. This includes a transcendentaliza­
tion of obj ects, classes of objects, and reasoning based on objects.
Another problem is the reification error: just because you provide a name or label
does not necessarily imply that the named or labeled thing exists. (" Let X Four-sided
=

triangle. ")
Still another problem is the commodification of artistic process and its products.
On a macroscopic scale, this is related to the formation of corporations as legal
persons. And that in turn is part of a general well-rehearsed critique of the metaphYSiCS
of presence and totalizing narratives. In light of all these difficulties, it is hard for an
atomistic theory to account for intersubjectivity or intersubjective experience.
Another and to my mind the most critical problem with atomism, as intractable­
perhaps more so-as its formal complexity, is its ethico-aesthetic inadequacy: how
could a set of isolated, atomic egos ever come to share a common experience of the
world? This basic problem of intersubjectivity, has plagued philosophy throughout its
history, but assumes a peculiar intensity as a problem for Husserl and Heidegger.
Moreover, how would these atomic subjects come to care about one another if it is
not clear how one consciousness can even know what another consciousness knows?
The first problem is perceptual and epistemolOgical. But the second is ethical, political,
and phenomenological. Pace zombie theorists, who deny interior experience, and
solipsists, my question is not whether we can act as if we know and feel another, but
how. It is a pragmatic ethico-aesthetic question whose implications depend on one's
attitude toward representational schemas.
S u b strate 97

Let me add a comment about hierarchies. It may seem odd that cyberlibertarian
engineers and allied theorists who inveigh against political hierarchy extol the formal
virtues of hierarchical (modularity and nested grouping) representation as a solution
to the world's profligate and exponential behavior. But this becomes more compre­
hensible when we understand that their allegiance is oriented not so much to
hierarchies but to objects in themselves, singular and autonomous. One could read
this as a wishful projection of Homo sapiens's willful freedom into the ontology of
the world.
To be fair, discrete sets can admit far more complex structures than what I have
enumerated here. We have barely reached the shores of algebra, and will not be able
to do much more in chapter 6 than name the simplest of these: groups, rings, fields,
and structures upon structures such as exact sequences and homological algebras. But
the technical and, most critically, the ethico-aesthetic problems remain in force for
any commitment to an atomistic ontology.
But enough negative critique, at least for the moment! What if we view the world
not as a vacuum raisined with corpuscles but as a plenum instead? What if we construe
and construct our world as a single medium varying through boundlessly many modes
of articulation, continually exfoliating in a value-creating magma of experience? What
this conception affords us will be the subj ect of the rest of this book.
It's an old alternative, of course, one that courses in the West from Heraclitus
through Spinoza, Leibniz, Serres, and Deleuze, and in the East from Laozi to the
present. Referencing Laozi prompts an amateur comment about Chinese brush paint­
ing. One of the striking characteristics of Chinese painting is that the repertoire of
brush techniques continuously span what in the West are distinct forms of graphical
expression: characters of poetry in calligraphy, the human figure, landscape, and
details of plants or animals. All these very different entities emerge from differential
intensities and local contexts from a common substrate impregnated with water inks.
We will return to this later.

S ubstrate, Fire, Water, Field

Forms interact not with forms but with their background, . . . the reservoir of the tendencies of
all forms even before they had separate existence or constituted an explicit system.9
-Gilbert Simondon

Substrate
What is a watermark? Of what is a watermark made? We see the watermark as imma­
nent in the substance of the paper-it is not made of some physical material other
than the paper, such as a glyph written in ink, and yet it has form. Moreover, its form
does not obscure, and cannot be eliminated by what is inked over it. Physically
98 C h a pter 4

reshaping the paper necessarily reshapes the embedded watermark. So the paper is
the substrate in which the watermark takes form.
So substrate means the stuff of the world, the material coextensive with all the actual
entities made of it. If this hyletic substrate is always and everywhere in dynamical
transmutation, to what poetic figures can we appeal in order to articulate matter +
energy + affect? l O

Running Water
Some leaves fall into a river and swirl away in the flow of the water. Some leaves twirl
into an inlet by the shore and spiral round and round with the coiled water, while
others float downstream. The leaves j ostle one another flowing along their own tra­
j ectories. By the shore, someone downstream can see those leaves coming from unseen
origins and going around the bend to unseen fates. The leaves are not the water in
flux but they make visible the movement of the water's current, at least at the surface.
We use this humble example to guide our foray into a qualitative and later topological
approach to dynamical systems. l l
Similarly, a wave i n the ocean is not some dust o r leaves laid o n top o f the water nor
even the water as substance, but a shape that moves through the water. Moreover this
shape is constantly in motion with respect to the constituent fluid: a particular mole­
cule of water will be in a given wave, and then pass out of the wave. Even in the striking
example of a water wave standing still with respect to the banks of a river, the water
itself is flowing downstream so the constituent water actually is progressively displaced
with respect to the standing wave. Therefore by symmetry the wave is in motion with
respect to the water. It is in just this sense that a wave takes form in its substrate. 12

Water M usic
Exercise: Dip a finger into a basin of water and make ripples continuously. See how the
ripples lap against the edges of the basin. Dip a second finger somewhere else in the
water and make a second set of ripples continuously. See how the first ripples continue
to lap against the walls of the basin as before, even though they superpose with the
second set of ripples as they pass through one another across the water's surface.
Imagine replacing the water by a sheet of wood. Drumming your fingers on the
wood makes ripples as well, but the wood vibrates at such high frequencies-frequencies
that are a function of the stiffness and density of the material medium-that you
cannot see them. But you hear the vibrations as timbre, and, in the refined shapes of
musical instruments played tonally, you hear them as pitch.

M usic Sound
Music provides, as always, one of the richest substrates articulating matter + energy +
affect, but since Schonberg we have come a long way from conventional discrete pitch
S ubstrate 99

scales, harmonic structures, and time signatures. As Deleuze and Guattari wrote in
their chapter on linguistics:
But when chromaticism is unleashed, becomes a generalized chromaticism, turns back against
temperament, affecting not only pitches but all sound components-durations, intensities,
timbre, attacks-it becomes impossible to speak of a sound form organizing matter; it is no longer
even possible to speak of a continuous development of form. Rather, it is a question of a highly
complex and elaborate material making audible nonsonorous forces. The couple matter-form is
replaced by the coupling material-forcesY

Deleuze and Guattari's embedding of music into its continuous substrate also holds
for speech:
There are many procedures for placing the voice in variation, not only Sprechgesang (speech-song),
which constantly leaves pitch behind by descent or ascent, but also circular breathing techniques
and zones of resonance in which several voices seem to i�sue from the same mouth . . . .
[Elthnomusicologists have found . . . cases . . . where a first, diatonic, vocal part is superseded
by a chromatic descent into a secret language that slips from one sound to the next in a continu­
ous fashion, modulating a sound continuum into smaller and smaller intervals until it becomes
a "padando" all of the intervals of which blur together-and then the diatonic part is itself
transposed according to the chromatic levels of a terraced architecture, the song sometimes
interrupted by a parlando, by a simple conversation lacking definite pitch. 14

Following Deleuze and Guattari's lead, let us return from sound to speech, holding
on to acoustic density and contingencies of sound in matter and flesh.
The Hubbub installation based on speech recognition has multiple parentage. Aside
from the obvious concerns with public speech and lettering in public space, 15 the
installation populates the theoretical gap between speech and text by materializing
glyphs that dance according to the prosody of live speech but persist with text's iter­
able durability.
But deeper concerns inform Hubbub. United States law recognizes "fighting
words"-utterances that when spoken under certain conditions have the same legal
impact as a physical blow, so that you can sue for such speech as if you had been
physically attacked.16 Why? Beyond the semantic content of the words, speech is
\
sonic; its sonic field copermeates both the body of the one who speaks anq the one
who hears, rendering them acoustically coincident in their tissue and blood. Ethically
one holds responsibility for what one does corporeally with respect to another body.
Therefore the sonic field, since it permeates all bodies present, constitutes an ethical
medium by its very transcorporeal extension.
When a doctor lays her hand on you, her patient, she performs the most ancient
medical technique: palpation. Palpation-the laying of hands on a body-does two
things. It is an act of finding, of determining the situation of the patient, and as such
it can be regarded as an analytic act. But it is also an ethical act: by laying a hand on
1 00 C h a pter 4

the body of the patient, the doctor effectively and affectively affirms: "I am taking
you into my care, I am now responsible for your well-being. " She is literally and cor­
poreally reenacting her oath.
What is essential in this example as in the previous is the superposition of two
fields of living material, breathing and pulsing together for a duration. To carry out
palpation, it would be a contradiction to separate the physician and her patient into
two diSjoint bubbles of the world. This epitomizes an essential quality of any ethical
medium, that is, its inseparability, or, to anticipate a more precise concept: its (topo­
logical) connectedness. And so we arrive at the first proposition:

(1) A connected plenum is a condition of possibility for ethics.

Anticipating the discussion of topology in chapter 6, separation or connectedness
do not necessarily refer to what can be marked off by ordinary Euclidean geometry.
Topology articulates proximity without relying on metric distance or geometry. By a
topology induced by the flow of matter in the universe over the scale of eons, some
of the atoms in the hair of the doctor and the patient and some atoms in the fabric
of the chair may be regarded as being in the same "open set, " because in the distant
future they will be whirling in common orbits about the same compact neutron star
that our sun will one day become. However, other atoms of the doctor and her patient
may wind up floating into relatively un correlated paths (during the epoch in which
a classical physics perspective is adequate), and therefore regarded as not so proximate
to one another. It's for such situations that the topological concepts of an open set
and connectedness seem well adapted.
Or the topology could be a very different one, induced by the set of touchings, of
all the ways that a touching-which is simultaneous with being touched-can be felt
to resemble or recall other touchings in any event that the doctor and her patient
have experienced. You can imagine that there is no reason at all for such a set to have
anything like the structure of a Euclidean space Rn. And the notion of proximity in
the set of touchings (which is not a geometrical space) may have to do for example
with all the emotional intensities that a touching evokes as well as the social codes
deriving from such intensities. People speak of feeling disconnected, or that someone
seems disconnected. This disconnection may not have to do with physical contiguity
but, as in this case, with a separation between the touchings that the doctor feels
appropriate to carrying out her medical responsibilities and the touchings that the
patient feels appropriate to being the subject of medical attention. Although irreduc­
ible to data and utterly contingent, the separation can be quite marked and even stable
in the sense that no sufficiently small perturbation of a careful touching turns it into
a careless touching. l? This gives an example of where a concept of topological separa­
bility can serve in very rich situations that cannot and should not be reduced to
geometric schema like boxes or grids or any rule-governed stratification.
S u bstrate 101

This example o f touching brings out a n essential aspect t o which I would like to
direct attention, away from classification of actions and to the dynamical substrate
within which any action takes place and shape. Noticing that Deleuze and Guattari's
turn of phrase "coupling material-forces" makes explicit the dynamical aspect of mate­
rial, let's proceed from the notion of substrate to the heart of a field-oriented approach
to substrate: its dynamism.

Heraclitus's Fire
At some point in a child's life, he or she wonders at fire. This wonder may or may not
be articulated in words, but nonetheless the child marvels and wonders: What is fire?
Is it fuel or is it light? Is it matter or is it image? How is it that it consumes its fuel
but does not consume itself? In an alchemical, poetic, and cultural meditation in a
design course about materials, Yoichiro Serita once observed that fire has this peculiar
material behavior: as a substance, it tends to expand without limit; it does not observe
the conservation law obeyed by ordinary matter. So thinking of fire amplifies our
notion of matter.
Process philosophy in the West found one of its greatest early poetico-cosmological
exponents in Heraclitus. Immediately before him came the first natural philosophers,
the Milesian physikoi of the sixth century BeE: Anaximander, Anaximenes, Xeno­
phanes, who created ways of looking at the world in terms of its physical contents
and dynamics rather than in lyric accounts of the actions of heroes and gods.
By no means, however, did the natural philosophers exclude the superhuman from
their cosmologies. Xenophanes proposed, primordial to Homer's all-too-human pan­
,,
theon, a cosmic god " similar to mortals neither in body nor in thought, I S opening
the way for a power that was not anthropomorphic but a principle of the dynamical
universe. Pythagoras constructed geometric order from which one could derive both
eternal and material, mortal patterns. In addition to, informed by but not limited to,
the mundane physics and geometry of the physikoi, Heraclitus created a poetic, mul­
tivalent complex of a dynamical plenum. The literary context of Heraclitus's time and
the syntax of his language suggest that his figures, such as the most famous ones about
fire and the river, speak to far more than what we typically consider the purview of
modern phYSiCS, binding cosmic pattern with the material and with mortal and non­
mortal experience.
Milesian and Ionian cosmology interpreted "physical change as a conflict of ele­
mental powers within a periodic order of reciprocity and symmetry recognized as
just"-dike.19 But departing from the natural philosophers, Heraclitus's poetic con­
struction took on quite a different order, in which the cycling of matter was fused
with a human and even cosmic order of justice in reCiprocity. This interpretation
placed the mortal notions of justice in a much vaster yet immanent frame of the
prinCiple of turning-into-the-opposite.
1 02 C h a pter 4

Table 4. 1

KOcrllOV 'tOY UU'tOV unuv'tov UU'tE
'tt� 9EOlV OmE UV9pronOlV
E1totTlcrEV, aAATlV aEt Kat EV'ttV Kat
Ecr'tat 1t'\lp aEt�OlOV, un't0llEvov
llE'tnu Kat unocr�EVvullEVOV IlE'tpU.
nupo� 'tpOnat npOl'tOV 9aAucrcru,
9UAUcr<JTl� OE 'to IlEV Tllltcr'U 'YIl, 'to
OE Tl lltcr'U npTl'tTl p ·
8aAucrcru OtaXEE'tat Kat IlE'tPEE'tat
Et� 'tOY UU'tOV AfY'{OV OKOto�
npocr9EV TlV Tl )'EVE9at 'YIl .
nupo� UV'tUllot�Tl 'tU nuv'tu Kat
nup UnUnOlV OKrocr1tEP �pucrov
KPTlIlU'tU Kat KPTlIlU'tOlV KPUcrO�.
The ordering, the same for all, no god or man has
made, but it ever was and is and will be: fire
everliving, kindled in measures and in measures
going out. (Heraclitus, fragment 3 7, Kahn pp.
44-45.)
The reversals of fire: first sea; but of sea half is
earth, half lightning storm. (Heraclitus,
fragment 38, Kahn pp. 46-4 7.)
Sea pours out <from earth>, and it measures up
to the same amount it was before becoming earth.
(Heraclitus, fragment 39, Kahn pp. 46-47.)
All things are requital for fire, and fire for all
things, as goods for gold and gold for goods.
(Heraclitus, fragment 40, Kahn pp. 46-4 7.)

From The Art and Thought of Heraclitus: An Edition of the Fragments with Translation and Commen­
tary, ed. and trans. Charles H. Kahn (Cambridge: Cambridge University Press, 1 9 79).

It is against this cosmological frame that we can interpret four fr agments of Hera­
clitus concerning fire.
It is crucial to understand that Heraclitus's fire is not just one of the four terrestrial
elements codified by Aristotle three centuries later, but the universal agent of change
of state, and the mediator of exchange analogous to gold's function for goods. Hera­
clitus's fire acts simultaneously upon the cosmos, the hearth, and the fu rnace, in other
words universe, home, and technology. Fragments 3 7 and 40 state that the Heraclitean
"fire" is the everlasting principle vitalizing the entire universe, KocrlLov (cosman, order­
ing); in other words, his is a monist ontology, but a thoroughly dynamic and intimate
one. As Kahn writes,
Fire is indeed a mysterious symbol of life, of superhuman life-despite or because of the fact that
it is the one element in which no animal can live, and a power that . . . often served to receive
human bodies at death. Thus representing life and creativity it also represents death and
destruction . . . .
[FireJ is not itself a kind of matter, not a body at all, but a process of transition from one state
to another . . . [emphasis added] .20

So what are the dynamics of Heraclitean fire? We get a hint when we note that a
key term in 38 is 'tpomx,t (tropai)-reversal, connoting the rout of an army as well as
the inflection of path of the sun at the two solstices of the year. And in fragment 49,
Heraclitus simultaneously describes physical dynamics and lived experience in terms
of a condition becoming its opposite:
XLIX (D. 1 26) Cold warms up, warm cools off, moist parches, dry dampens.21
S u b strate 1 03

This transformative principle of C into not C, combined with the reflexive principle
(not not C) C, yields an eternal cycle which anticipates the fundamental role of the
=

harmonic oscillator as a model of physics from Newton to the present day.

Moreover, we can regard Heraclitus's "measures up to the same amount" (Il£'tp££'tat)
in fragment 39, and the balanced measure (Il£'tpa) of fire kindling and extinguishing
in 3 7, as stating a principle of conservation of matter-energy. Obviously, this state­
ment referencing measure concerns not numerical metric but proportionality and,
in fact, symmetry or reciprocity. In modern times, this reminds us of one of the more
profound foundational theorems of the calculus of variations, by Emmy Noether.
Colloquially phrased: to every symmetry of a variational system corresponds a con­
served quantity. We do not have modes of articulation adequate to describe Noether's
theorem more precisely and meaningfully, but let me immediately note that in the
twentieth century we came to question the ubiquity of this theorem.22 What that
means is that in some situations either matter + energy + affect is not conserved or
there is no invariance of the dynamical system under any ("infinitesimal variation
of the parameters") local variation field. Now, what this has to say about care poten­
tially as a substrate for ethics and politics remains to be worked out in this chapter
and in work beyond the scope of this book. But to start the work, consider the
Golden Rule, do unto others as you would have them do unto you. It assumes, to adapt
a notion more rigorously defined later in this chapter, isomorphic subjects. In other
words, you and your others are the " same" as ethical subjects. You and others have
the same interests and desires. It also assumes that the only relevant forces are
humans, anthropic subjects. But if we are to try to avoid making Man the center of
the eco-ethical universe, to come up with some modes of articulation that are not
so tautologically or solipsistically anthropocentric, it should be relevant to understand
how to relax the anthropocentric notion of symmetry underneath the Golden Rule
to a nonanthropocentric understanding of the relation between symmetry and the
dynamics of matter, energy, and affect that we have begun with Heraclitus. In our
context, these modes of articulation generalize ethico-aesthetic dynamics that in
human terms could be understood as the dynamiCS of caritas or care to a cosmologi­
cal setting.
These three principles of reversal, of conservation under transmutation, and of sym­
metry do not completely articulate ontogenesis, but at least they can serve as ingredi­
ents of a dynamical ontology. And we can discern some patterns in this ceaseless flow
of change that Heraclitus describes in his most celebrated statements, fragments 50
and 5 1 :
L (D. 12) A s they step into the same rivers, other and still other waters flow upon them.
LI (D. 91) Plutarch: [According to Heraclitus one cannot step twice into the same river, nor
can one grasp any mortal substance in a stable condition, but by the intensity and the rapidity
1 04 C h a pter 4

of change it scatters and again gathers. Or rather, not again nor later but at the same time it
forms and dissolves, and approaches and departs.j23

In Kahn's view, Heraclitus's statements about the dynamics of the world do not claim
that identities are unstable-indeed lithe river" is a stable obj ect-but rather that the
state of the river, and one's experience of the river, are never statiC, and therefore not
necessarily identically constant. This observation bears remembering. It will serve us
well as the core of the performative. It anticipates quantum mechanics' inextricable
intertwining of the observer and the observed (precisely scaled by Planck's constant),
which may offer deep insights for the theatrical duality between spectator and actor.
Given this Heraclitean prinCiple, which works as the potential of change, let's turn to
consider the qualitative dynamics of matter.
It may be richer and more fruitful to imagine material dynamics born of Heraclitean
fire rather than Aristotelian substances, each with their own fixed formal and material
dynamics. Building on Heraclitus's conception, we can posit manifold bundles of
potential fields, as many as there are spectra of flames. We can posit fields of potential
force extending throughout the material manifold, i.e., the world. Since there can be
multiple passions in play at the same time, we posit multiple fields for their superpos­
ability (to be made precise in chapter 6) . These fields may act at multiple scales, may
be restricted to or concentrated in various parts of the material manifold, or may even
be functions not of the base manifold but of its spatial derivatives, in other words of
the variation of densities of the material manifold, rather than the densities (objects)
themselves.
We can think of a field, roughly, as a continuous distribution of potential with
respect to the actual world. Even for a fixed actuality, were that possible, there are
uncountably many fields of potential. And the structure of the potential can vary
uncountably as well. We can be more precise in chapter 6, but for now imagine for
example the fields of repulsions and attractions that condition how lava, capital, or
individuals flow within their planes of immanence.
I propose to adopt a field-theoretic attitude (and after chapter 5, a topological
articulation of a field-theoretic attitude) and see what becomes of the world and of
our inhabitation in the world under such a perspective. What happens to language
and languaging, to computational media and computational process, to subjects and
objects under a field-theoretic approach? To summarize this compactly as our second
proposition:
(2) Field => potential dynamics. Fields => multiplicity.

Affective I ntensity
If affective intensity is a scalar density, its gradient with respect to the material
manifold-how it varies along with the extensive distribution of the material
S u bstrate 1 05

manifold-is directed. This variation with respect to a given direction has an intensity,
so it is vectorial, with direction and intensity. (To anticipate chapter 5, this is a precise
formulation of Whitehead's vectoriality of prehension.)24 In fact, any differentiable
distribution-a scalar intensity-gives rise to a vectorial gradient. But there can be
vectorial fields that are not gradients of a scalar intensity: namely those that have
sources and sinks, singularities. In any case, an affective field yields a directedness with
respect to which one can articulate dynamics and concentrations of relations and
subjects, organisms, collectives, nexus. That these are affective fields implies by defini­
tion that they induce value, or to be more precise, they induce processes that articulate
value. That these concentrations or densities, particularly compact ones (again antici­
pating chapter 6 where I will give a precise notion of compact), can be identified as
subj ects in affective relations with respect to one another makes them ethical subjects.
In other words, it is with respect to the vectorial fields of value that actions arise and can
assume ethical force.
We can gain some insight from this way of articulating ethical dynamics from
affective intensity. To take one consequence: we do not have to preidentify or preposi­
tion ethical subjects, any more than we need to preidentify a charged particle to have
an electromagnetic field. Indeed we can borrow the notion of a test particle. In order
to understand how a potential, vectorial field can induce actual dynamics, one can
experimentally place a "test charge!! and see how it moves under the influence of that
field. Or one can place an actual charged body and witness how it moves. But the
field extends through the world, and there is no place where it does not "exist, !!
though it may be attenuated or even zero in value. There is an essential difference
between saying that the field does not exist at some place on the manifold and saying
that it exists but is of value zero. To take the example of electromagnetism, a charge
placed at a location where the electromagnetic field is zero does not move. But to say
that the electromagnetic field is undefined at some point would mean that one cannot
make any statement at all about what a charge placed at that location would do.
To more satisfactorily explore consequences of this would take another proj ect, but
let me make this initial observation. Recall that, as I said in the chapter I, this project
has been motivated by whether it makes sense to hyphenate ethico-aesthetics, and
how it makes sense to improvise ethico-aesthetically. Recall also that we set aside
appeals to transcendental, a priori, and teleological schemes on one hand and random
schemes on the other, the former because they are rigid or brittle, the latter because
they are boring by definition, and both because we are after the vast region of artful
gesture in between. What I suggest is a more nuanced account of material articulation
cosh aping ethico-aesthetic potential that is neither overdetermined by originary or
teleological schemes nor random.
Now it is fair to wonder how much richness of potential action this mode of articu­
lation could sustain. Since there can be multiple affective intensities, indeed as many
1 06 C h a pter 4

as can be imagined, there can be a multiplicity of fields of value. In fact, in principle

the space of possible fields of value can be boundless or infinitely dense in topological
and measure-theoretic senses.2S
The first proposition was that a connected plenum was a necessary condition for
ethics. The second proposition was that fields enable a boundless multiplicity of
potential dynamics. Now we can add a third proposition:
(3) Affective intensity constitutes value, and so is primordial (as substrate) to
ethical action.

Care and Politics

Years ago, one of the students in my Interactive Design program wanted to create a
web-based information system that would archive congressional records and make
them available to the public as legislation was being proposed and shaped in commit­
tees. She reasoned that this would encourage public investment, intertwining public
discourse with legislative discourse, and ultimately be a tool helping to reconstitute a
more democratic res politica. This was predicated, like many great civil rights move­
ments of the latter part of the twentieth century in the United States, on a certain
assumption of communicative rationality, the Habermasian contention that human
rationality is a necessary outcome of successful communication. The identity-based
movements of the 1 980s inherited the political logic of the successful civil rights
movement a generation earlier and even managed to partially institutionalize recogni­
tion and power for certain disadvantaged identity groups. I have much sympathy with
these movements. And I do not wish to debate the possibility and rational efficacy of
"transparent" communication, because my concern lies at quite a different level. Even
if I were able to gain a proper political place as a Chinese-American in the heteroge­
neous society, even if the glass ceilings were broken for women in the workplace, even
if every harm against the Japanese-Americans in the United States during World War
II were fully recognized and compensated, even if African-Americans were able to come
into their own after slavery and its aftermath, even if the peoples of First Nations were
able to live in the lands in the manner that they would wish, why should a member
of one kind of people feel any solidarity with a member of another? Why should I
care about you? Care is what I call the affective field primordial to collective politics
(what used to be called solidarity), or more precisely, primordial to the vector fields
of collective political power. (Here I use this word "field" particularly for its connota­
tions in mathematics and physics as well as its colloquial sense of continuous variation
in time and space, and its latent potential for powering movement and material
change. Precise definitions will come in chapter 6.) It has nothing to do with charity,
Christian or otherwise, because it exists prior to the institutions of power, whether
secular or churchly. Nor is it identical with Heidegger's care as an existential attune-
S u bstrate 1 07

ment of Dasein to the world. It is primordial in the way that Foucault's governmen­
tality is primordial to the institution of the modern state.26
Care ¢:} recognition of fellowship.
However, this raises its own question: What underlies this implication?
Recall the example of palpation. Palpation has ethical and informative power
because of a textural resonance between the physician and the patient. The textural refers
not to pushing buttons or measuring predicates on a patient, but to the warmth, sure­
ness, and continuity of movement, vocal as well as gestural. (I will elaborate on the
textural later on.) Resonance is the concept bridging natality and care. It does not
require isomorphism or telementationalise7 transmission of objects, or the supposed
transmission of abstractions such as "data" or "information. " (In a deep sense, that is
why it takes relatively negligible force or energy to make a large effect when rocking a
car cupped in snow, if the force applied is synchronized to the cycle of the car's motion.)
Arendt's caritas =} natality
To paraphrase Maurizio d'Entreves, natality is the fact of having been born, introducing
a new beginning in the world. The later Arendt's description of natality substantially
deepens her dissertation's approach to the problem of Augustinian care-"caritas"­
against Heidegger's existential and thanatopic characterization of care.
Care, in my use of the term, is the pathic recognition of natality. This is consider­
ably more special than the Heideggerian concept of care (Sorge) as attunement.28
(4) Care, an affective field, nuanced by attunement and pathic sensitivity, is
primordial to politics.

Freedom and Poiesis

So, natality underwrites care as an affective field primordial to politiCS. Natality is also
intricately related to freedom.
By freedom, I do not mean being able to make an unconditioned choice among
options (think of the coffee shop example) . Maurizio Passerin d'Entreves writes:
[Bly freedom Arendt means the capacity t o begin, to start something new, t o d o the uQ.expected,
with which all human beings are endowed by virtue of being born. Action as the realization of
freedom is therefore rooted in natality, in the fact that each birth represents a new beginning
and the introduction of novelty in the world.
To be sure, Arendt recognizes that all activities are in some way related to the phenomenon
of natality, since both labor and work are necessary to create and preserve a world into which
new human beings are constantly born. However, of the three activities, action is the one most
closely connected with natality, because by acting individuals re-enact the miracle of beginning
inherent in their birth. For Arendt, the beginning that each of us represents by virtue of being
born is actualized every time we act, that is, every time we begin something new. As she puts it:
1 08 C h a pter 4

"the new beginning inherent in birth can make itself felt in the world only because the newcomer
possesses the capacity of beginning something anew, that is, of acting."z9

Freedom entails Kantian autonomy, Le., independence from being determined by

a rule-governed system. But Arendt extends this notion of freedom, in Miguel Vatter's
interpretation:
Freedom requires the automatism of life while remaining counter-natural, where "natural" means
subsumed under a law-like process. Natality is the only category that satisfies both of these,
apparently contradictory, conditions : belonging to life yet not subsumable under a rule or law
,,
(and hence "miraculous ) .3o

In an essay on Arendt and natality, Vatter remarks that for Arendt, freedom has three
essential features. First, it has a sense of the automatic, Le., of not being subject to
determination in a rule-governed system. Second, unlike Kant's noumenal conception
of freedom, Arendt's freedom is thoroughly conditioned, phenomenalized, and there­
fore particularized to the individual life.31 Third, and most importantly, he finds that
Arendt links freedom not to Heidegger's being-toward-death but to natality: "Natality
is the key category for a politics that is to come after the end of the nation-state, . . .
after all attempts to [relate the] political . . . to the familial, and . . . after . . . all politi­
cal form or organization as such."32
With regard to an Aristotelian distinction between zoe and bios (the category of
living things versus the category of the human), one could align Arendt with Heidegger
as being concerned primarily with bios. However, one need not accept this distinction
so categorically, nor identify Arendt so hastily with Heidegger. As Vatter writes:
Arendt's conception of natality does not presuppose the Aristotelian dualism between zoe and
bios . . . . [N]atality is not only what "inserts" life into a pre-given world, but . . . is also what
"daily renews" this world itself. . . . [N]atality is irreducible to the bios politikos . . . [and] a politi-
cization of zoe, not of an always already "political" bios. This politicization of life goes in the
opposite direction of . . . "the cycle of ghenos (descent, race) . '>33

I suggest that we can no longer afford to act insisting on this distinction between
zoe and bios (as if we ever could) . My point is somewhat stronger than where Vatter,
and Arendt, may have been willing to go: if we are to hope for any understanding of
how to live adequately to the world, we need to lift the political implications of natal­
ity from its anthropocentric cradle to include the energetic labor of zoe as well. And
in so doing, to avoid the anthropocentric commitments in the concept of freedom, I
prefer to think in terms not of freedom but of its textural analog, poiesis.

Texture
A parable: If you lay out tiles of glass on the ground in the cold, edges tightly butted
together, over time, with the heat of the sun, they will expand and, because they have
nowhere to go in their plane, will buckle and crack. So the next year, you learn to lay
Su bstrate 1 09

them with gaps in between . If you nonetheless wish to make a continuous, smooth
surface, you could fill the gaps with silicone or some other elastic material that bonds
to the edges of the tiles, but gives and stretches under the expansion and contraction
of the glass.
Another parable: The difference between a violin and a piano is nuance. A violinist
can bend the pitch of a singing line of sound by varying the tension of the fingers on
the strings and the bow at the same time. The infinite degree of variation possible
yields nuance not only as pitch bend, but as vibrato, fatness, and any number of quali­
ties. In fact there is no a priori set of mechanically distinct qualities, although by
convention over the years violinists have developed an expert vocabulary for talking
about how they nuance their musical sound. But the essential point is the possibility
of continuous variation-nuance.
What nuance (the continuous dense variation of corporeal gesture in material sub­
strate) affords is an infinite possibility of the fresh or singular, even at the textural level.
In fact, it is the indefinitely fine that is the texture of freedom, which I will call, from
now on, poiesis. This poiesis, like Arendt's freedom, is fully conditioned-textured,
colored-by all the affective fields in play, fields that are in turn sensitive to the mac­
rodynamics of history and politics, as well as mesodynamics of sexuality and power.
However, the poiesis I propose differs radically from freedom in two respects: ( 1 ) it does
not presume an anthropocentric subj ect, (2) it is textural, not oriented toward obj ects.
We can ask what "coloring" implies in such a context. In the case of speech and lan­
guaging, Deleuze and Guattari write in A Thousand Plateaus: " [This chromaticization]
places the public language's system of variables in a state of variation [emphasis original] .
This is what we are getting at: a generalized chromaticism. Placing elements of any
nature in continuous variation is an operation that will perhaps give rise to new distinc­
tions. "34 We can understand chromaticization in fact as a mode of continuous variation
through the dense substrate. Regarded from the perspective of the textural rather than
the anthropocentric, chromaticization is a mode of the world in poietic ontogenesis,
the subject of chapter S. What is natality then, in the nonatomistic sense of substrate,
but the fact of fresh, textural singularity in the material exfoliation of the world, the textural
condition of poiesis? Textural means that it is an everywhere-dense subset of t� e world,
in a sense that will be made precise in chapter 6. The subtle point here is that a plenist
texture has a different order of continuity than a union of isolate points-a bag of dust
versus a bag of water. The twentieth-century characterization of the density of the real
line, the Dedekind cut-"between any two real numbers a < b there exists a third real
number c such that a < c < b"-is a test that applies to the continuum R (the so-called
real line) . But this test uses the peculiar unidimensionality of R and does not work
in more general sets. For that we appeal to general point set topology and the notion
of open set and open cover, to be developed in chapter 6.
I gather these thoughts into a fifth proposition:
1 10 C h a pter 4

(5) Continuity (in the space of variations of plenum) affords nuance and poiesis:
the textural natality of material.
In my usage, to afford means to constitute the material conditions of possibility.

Some I m p lications

Let me trace these themes in three domains of expression: software programming,

video, and bodies in movement. Why go to such a level of technical detail? One of the
guiding challenges for this book is the motto " art all the way down . " If we are going to
experiment with how obj ects, tools, instruments, even gestures and techniques come
into being, dissolve, and reform, then we need to pay attention to the materials from
which they are made. So the microtexture, quasi-chemistry, and quasi-physics of the
material itself as well as the macrosocial are all potential media for artfu l investigation,
warranting the artful synthesis of visual, sonic, textile, and other material regarded as
temporal media. Of course opening up the technical substrate for artful investigation
may simply displace the schematizing power from one stratum of technology to
another. To gain fine technical control of computational video, for example, requires a
bubble of a lab funded to do that sort of work (chapter 7) and technical expertise in the
lattices of knowledge (e.g., computational fluid dynamics and low-level coding for
parallel machine architectures) significantly more disciplined than the craft typically
available to DIY artists and hackers. However, the very same epistemic and technical
disciplines that enable such powerful expressions equally powerfully hobble them.

Example 1 : From Object-Oriented Programming to Real-Time Signal Processin g

More than twenty years ago, a group of Xerox PARC researchers released the grandfa­
ther of obj ect-oriented languages: SmallTalk.35 Promoters of object-oriented program­
ming (OOP) urge programmers to think about their world in terms of classes of
" concrete" objects and relations between these classes. Of course these classes of
objects, being software, are already stripped of most of the qualities of ordinary experi­
ence, yet the pedagogical examples often use comforting analogies to everyday catego­
ries like animals: a dolphin is a mammal is an animal; a giraffe is also a mammal. A
giraffe is both a mammal and a land creature, therefore it inherits properties of both.
A dolphin, on the other hand, only inherits properties of mammals. Each class of
objects has certain functions (called methods) that allow other software objects to
inspect or set its properties or invoke some action. For example, "eat" is a method
(action) particularized to very different actual actions depending on the specific class
of animal, but the method (action) eat could be performed by any member of a sub­
class of the class of animals. Also, the method "eat" is polymorphic: what dolphins eat
(and how they eat it) differs in type from what cats eat. OOP, while comforting in its
promise to allow programmers to write software isomorphic to any taxonomy of a
subset of the world, also quickly developed as much complexity as the world itself,
S u bstrate 111

and worse. The world comes to us ordered by generations of stories and myth, folk
knowledge, and many other machines of memory, economy, and desire. But OOP
taxonomies, being invented naively and instantaneously with respect to historical
time, do not inherit such sense-making order. The community of OOP software engi­
neers has compensated by inventing patterns36 and frameworks that bundle program­
mers' folk knowledge about a coherent set of functionalities, usually quite mundane
but intricate functions like printing or saving a file. As I pOinted out, hierarchical
structuring only defers the problem of complexity by one structural stratum. The
contradiction is that to be useful in practical, commercial situations, OOP frameworks
have to be as large and complex as the world they model, and they grow ever more
complex over the years. But large frameworks typically become intelligible only to
their authors. Any large OOP framework tends to challenge a novice programmer's
ability to read and comprehend a complex hierarchy of interrelated classes. In fact,
SmallTalk's authors elevated this to a virtue, saying that a good SmallTalk programmer
reads more than he writes, a particularly dry simulacrum of scholasticism.

An Aside about Categories, Objects, and Morphisms

What lay beyond the Babel of OOP was the shimmer of category theory, which emerged
in the 1 940s, about twenty-five years before the emergence of SmallTalk and then the
more statically schematized object-oriented language C++. The next pages describe
category theory in some detail precisely so we may know something of what this book
attempts to set aside.
A category is a collection of entities with some internal structure, with associations
relating pairs of entities called morphisms that preserve structure. A basic axiom of
categories is that morphisms can be composed: given a morphism { from object X to
object Y, and a morphism g from object Y to obj ect Z, there is a morphism called ga{
from object X to object Z. Usually, where the morphisms can be defined as mappings
on elements of objects, this means that for every r in X, the mapping gaf. X -7 Z is
defined by g[f[r] ] , that is, apply { and then apply g (see figure 4.3).
Given two categories C, D of mathematical objects, and mappings between them,
category theory defines a functor as a map carrying objects in C to obj ect s in D, and

f
x ' y

� lg z

F i g u re 4.3
A composition of maps f and g among three objects X, Y, Z in a category. Diagram courtesy of author.
1 12 C h a pter 4

morphisms in C to morphisms in D. More precisely, if H is a functor from C to D,

then for every morphism f. X -7 Y in C, H[fl is a morphism H[X] -7 H[Y] in D . The
two axioms that a functor must satisfy are that it respects identity morphisms and
composition. More precisely:

( 1 ) H[idx] H[idH[xJl for every obj ect X in C, where an identity morphism on an object
=

X maps every element of X to itself: idx [s] s for all s in X.

=

(2) H[f>g] H[f]oHIg] for any composition of two morphisms f. X -7 Y, and g: Y -7 Z in C.

=

A student of mathematics first encountering object-oriented programming (OOP) with

its emphasis on classes may be reminded of category theory, but it soon becomes clear
that almost no functorial relations obtain. Although it may be theoretically interesting
to make a formal category out of software frameworks, most working-class OOP engi­
neers outside the largest engineering combines such as Hewlett-Packard and IBM do
,,
not try to make such structural-propositional homologies across OOP "ontologies. 37
At the beginning of this chapter I said that my purpose is not to classify obj ects or
their dual subjects, but rather to offer an approach to articulating objects as they come
into being, as they emerge from continuous fields of media-material and then dissolve
again into those fields. Category theory does not satisfy this in two fundamental ways.
First, category theory is a description, not a mode of articulation of material. Indeed
category theory is not a theory about a set of mathematical objects like manifolds or
paths in a configuration space, but a theory about theories: a doubly abstract (in a
logical and not Deleuzian sense of abstract) description about theories about families
of mathematical obj ects. Second, category theory says nothing about the dynamics of
physical, or living, affective material. For this, as we will see in chapter 6, we go to
particular mathematics adapted to our purpose here, the dynamical articulation of
material plena of matter, energy, and affect. Despite the enormous power afforded by
category-theoretic methods to extend whole theories across vastly different domains of
objects and their structurally indigenous mappings, mathematicians who favored more
concrete ontologies nicknamed category theory " arrow theory, " or "abstract nonsense. "
One may try t o justify a n atomistic, obj ect-oriented ontology b y claiming a homo­
morphism of sorts between the " real world" of things and the objects in an object­
oriented programming language via the formalisms of category theory. However, the
existence of such a framework does not imply that there is a natural functor between
phenomenon and representation.

Example 2: Drawing, 00 G raphics, Video as Picture, Video as light

An object-oriented drawing application like the canonical MacDraw or Adobe's Illus­
trator provides a small set of atomic geometric shapes that can be combined or modi­
fied as algebraically independent objects. "Algebraic" in this case means there is a
finite discrete set of primitive objects, for example {circle, square, line segment, triangle};
 7

1 2 3 4 5

I n [ l j := Vertex 1 = {3,3}; Vertex2 = {S,7}; Vertex3 = { 1 A};

I n [4j := Tri a n g l e := {Vertex l , Vertex2, Vertex3, Vertex 1 };
I n [Sj:= Show[
G ra ph ics [UTh ickness [0.0 l j,Li ne[Tria ng Ie]},{RG BCo l or[O,0,Oj,Poi ntSize [0.04j,
\Point[Vertex l ], Poi n t[Ve rtex2j,Point[Ve rtex3]}}j,
Axes\[Ru lej True, AxesOr i g i n\[Rule]{0,2}, PlotRa nge\[Ru l e]{{-0.5,S .S},{ 1 .S,7.5}},
AspectRatio\[Ru l ejAutomatic

Hrgb[ l , l , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ], rgb[ l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ],
rg b[l , l , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rgb[ l , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ],
rg b[l , l , l ], rg b[l , l , l ], rg b[l , l , l ], rgb[l , l , l ], rg b[l , l , l ]' rg b[l , l , l ]' rg b[ l , l , l ], rgb[l , l , l ], rgb[ l , l , l ], rg b[l , l , l ], rgb[l , l , 1 ],
rgb[l , l , l ], rgb[ l , l , l ], rg b[l , l , l ], rgb[ l , l , 1 ], rg b[l , l , l ]' rg b[l , l , l ], rgb[ l , l , l ], rgb[l , l , l ], rg b[l , l , l ], rg b[l , l , l ], rgb[l , l , 1 ],
rgb[ l , l , l ], rg b[l , l , l ], rg b[l , l , 1 ], rgb[l , l , l ], rgb[ l , l , l ], rg b[l , l , l ]},

{rgb[l ,1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ], rgb[l , 1 , 1 ], rgb[l , 1 , 1 ], rgb[l , 1 , 1 ], rgb [ l , 1 , 1 ], rgb[l , 1 , 1 ], rgb[1 , 1 , 1 ],
rgb[ l , l , 1 ], rgb[l , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b [ l , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ],
rg b[l , l , 1 ], rgb[l , 1 , 1 ], rgb[O.870588,O.870588,O.870588], rgb[O.266667,O.266667,O.266667]. "
rg b[O.85098,O.85098,O.85098]' rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ];rg b[l , 1 , 1 ],
rgb[ l , l , 1 ], rgb [ l , 1 , 1 ], rgb[ l , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , H rgb[l , 1 , 1 ],
rg b[l , l , 1 ], rgb[l , l , l ], rgb[ l , l , 1 ], rgb[l , l , l ]},

{rgb[ l , l , l ], rg b[l , l , l ], rg b[l , l , l ], rgb[l , l , l ], rg b[l , l , l ], rg b[l , l , l ], rgb [ l , l , l ], rgb[ l , l , l ], rg b[l , l , l ], rg b[l , 1 , 1 ], rgb[l , l , l ],
rg b[l , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[ 1 , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ],
rg b[l , 1 , 1 ],\ rg b[l , 1 , 1 ], rgb[ l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ],
rg b[ l , l , l ], rg b[l , l , l ], rgb[ l , l , 1 ], rg b[l , l , l ], rgb[l , l , l ], rg b[l , l , l ]' rgb[ l , l , l ], rgb[l , l , 1 ], rg b[l , l , l ], rg b[l , l , l ], rgb[l , l , 1 ],
rg b[ l , l , 1 ], rg b[l , 1 , 1 ], rgb[l , 1 , 1 ]' rg b[l , 1 , 1 ], rgb[O.890 1 96,O.890 1 96,O.890 1 96], rgb[0.768627,0.768627,O.768627]}}

F i g u re 4.4
Three representations of a triangle: a relatively semantically rich description as geometric primi­
tives, and a semantically shallow bitmap.
1 14 C h a pter 4

every obj ect comes with a notion of inverse: for example, create-a-square, delete-a­
square; and objects may be combined with some uniform operation: for example
place-in-view-at-cursor. By contrast, a bitmap drawing program, such as MacPaint or
Photoshop, has no primitive set of macroscopic objects, and therefore makes no
"object-oriented" restrictions on what the user can draw. This freedom for sketching
comes at the cost of having the program not maintain extra information about specific
classes of geometric objects on behalf of the person who sketches, such as the fact
that there are exactly three vertices defining any triangle, so information for three
pOints is maintained somewhere in the machine representation, with the rendering
of the triangle always interpreted as j oining the three vertices by three straight line
segments. However, when Joan Miro draws (and we see) a "triangle" in his painting
Circus Horse ( 1 927), it is not any triangle that the object-oriented program would have
recognized or given to him as a primitive. Even more allusively, in Miro's Person Throw­
ing Stone at Bird ( 1 926; figure 4 . 5), the humorous point of the scene is the hopeless

F i g u re 4.5
Joan Mir6, Person Throwing Stone at Bird, 1926.
S u b strate 1 15

F i g u re 4.6
Cy Twombly, Untitled, 1 9 70, one of a series of blackboard drawings.

inadequacy of the arc despite its perfect circularity, whose radius-its dashing connotes
the hunter's calculating intent-is mocked by the radius of the bird's body. None of
this is coded in the drawing program's geometric representation.
One might object that even a bitmap is a machine representation of visual pattern
(see figure 4.4). The difference is that the encoding in terms of "geometric" labels like
vertex and triangle presumes we are capturing essential meaning with these labels.
But we can see that even the most preliminary reading of Mira can activate much
sense exceeding the descriptive reach of either kind of machine representation. And
elaborating some machine-codable schema to faithfully represent the affective dynamic
and implications of the painting would be as pOintless as making a chess program to
enjoy a game of chess on our behalf.
And there we see the point: Why not leave the interpretation to the participant at
the moment of the event, to the one who makes the brush stroke at the moment of
making that stroke, or the one who makes the affective interpretation at the moment
of encountering the painted canvas (see figure 4.S)?
Going to 3D graphics only obscures the issue. The basic mistake is to identify
geometry with the visible, when in fact most of what modern differential geometers
1 16 C h a pter 4

F i g u re 4. 7
Calligraphic video (A) as image, and (B) as structured light.

imagine has very little to do with the pretty images that a computer graphics program
can provide.
The standard abstractions of object-oriented graphic models in computer graphics
codified in industry-standard 3D representation and rendering frameworks, such as
Maya, Renderman, and OpenGL, have built up as an enormous body of algorithms
and hardware mostly predicated on the construction and drawing of polyhedral
surfaces in three-dimensional Euclidean R3 . Historically, the resulting images typically
suffered from a deadness, a lack of breath. All the sophistication in the modeling of
geometrical optics and optics of material surfaces could not escape the fundamental
misstep, which was the obsession with polygons rather than continuous stroke and
gesture. (A major alternative graphic representation system-PostScript-admitted
abstract representations of polynomial curves that could be rendered at arbitrary
physical resolution, because, being a printer specification language, PostScript could
defer conversion to particular pixel representations till very late in the process of
printing on their target devices: paper printers. One of the principal innovations
of the NeXTStep operating system-the precursor to Apple's OS X-was using Post­
Script as its underlying drawing language, which permitted arbitrary scaling of
its screen graphiCS.) This changed in the last decade when cheap memory and speed­
ier CPUs finally permitted the "interactive" manipulation of bitmaps, in fact, of
textures mapped onto (sufficiently fine polyhedral approximations of) continuous
surfaces.
Instead of slinging polyhedral geometries in OpenGL, I reasoned differently:
1 . Ultimately, however sophisticated and arduous the processing, the image is still to
be displayed on planar two-dimensional surfaces, whether on computer display moni­
tors, plasma screens, or proj ectors beaming images onto flat planar screens. So why
Substrate 117

not conserve CPU cycles and concentrate processing power on just two-dimensional
data throughout the input and resynthesis pipeline?
2. A sure way to generate credibly rich texture is to acquire imagery directly from the
world in its full density by "sampling" video or sound. Even surer is to skip the step
of digitization and allow the participant to physically manipulate material.
Exercise: Take a brush. Dip it into ink. Drag it fast, then slow, across rice paper. Use
a wet brush, a dry brush. Notice the corporeal, physical effects lost in translation to
digital tools and screen-based displays.
Exercise: Take a lump of dough (or clay) . Knead it. Press it flat. Try to roll it out.
(Clay has memory, dough develops memory.)
Richness � Continuity. In fact the continuity is what affords expressive shaping.
This motivated the work of the Topological Media Lab on synthesizing video as
textures responding to contingent gesture: "calligraphic video, " accompanied by anal­
ogous "gestural sound. " We typically regard video, whether it appears on a computer
display or projected on a wall, as image-depicting a picture of something. But sub­
stituting a small projector for a light bulb gives us the opportunity to regard video as
structured light dynamically illuminating a physical space instead of the projection of an
image.
Together with physical materials such as the sonically responsive textiles woven
with conductive fibers, dynamical media textures-temporal textures, dense fields of
variations in the tempo and rhythm-constitute the raw material with which we
construct our responsive environments.
Regarding and synthesizing video as a painterly or calligraphic rather than an
obj ect-oriented or typographic medium implies that we dispense with syntax com­
posed of predetermined, discrete classes of gesture yielding predetermined, discrete
classes of graphical objects. Of course, definite techniques (orthographies) of move­
ment and corporeal and thoughtful comportment have evolved over historical time,
but the implement can condition rather than rigidly enforce specific sequences
of movement. In fact, among " analog" drawing instruments, more hypostatically
designed instruments with rigid action tend to have very narrow and limited use
'
before being abandoned in favor of " simple" tools like brush, pen, charcoal, and cor­
responding " syntax-free" drawing surfaces. There is no right stroke or wrong stroke;
every movement of the hand makes a mark of some sort depending on the microphys­
ics of the implement, the coloring substance, and the substance of the drawing surface.
To the extent that the goal is to produce 2D images on a flat surface like a display
or a wall, this version of calligraphic video suffices. But there can be a more ambitious
aspiration: to make structured light with sufficiently powerful projectors. Calligraphic
video (1) treats video not as image but as structured illumination, and (2) creates pal­
pable light fields, leveraging corporeal experience.38
118 C h a pter 4

My strategy has been to leverage the intuition sedimented into our bodies over a
lifetime from birth-more than a lifetime if we include the corporeal disciplines in
which we're schooled: playing music, playing sport, uttering language, and so forth.
In order to manipulate or navigate or shape light (and image as texture), if we wish
to use structured light for its palpable interference with dynamical matter, what better
way than to make it respond to individual or collective gesture as a physical material
would respond? Why physical material rather than logical statements, or cultural code
(logos, branding, fashion, design)? We appeal to physical matter because we acquire
from birth a lifetime of experience with the felt sense of matter around us, a conden­
sation of felt experience prior to language, spoken or written. Why computational
media (matter)? Why not stick to physical, noncomputational matter? With compu­
tational media, we can create media/matter that is modeled after physical matter and
can be approached or manipulated with some corporeal intuition, but that behaves
in a quasi-physical manner like and unlike familiar sorts of physical matter. Computa­
tionally modulated material textures can constitute the experimental medium (not
just the trope of "apparatus" inherited from nineteenth-century science) for proposi­
tional, speculative essays in fields of gesture and movement.
This detourned matter can be a substrate bearing marvelous symbolic charge.
I envisioned the TGarden as a responsive environment so thick with media that it
is not obvious where the body ends and the rest of the world begins. One way is to
suffuse the environment with responSive light whose temporal textures fluctuate
concurrently in concert with the movement of bodies in the same space .

Example 3: From Bodies in Movement to Bodies from Movement

In computer graphics, presently the best physics engines for games can handle tens
of thousands of mutually interacting particles before giving up the ghost, but even
twenty years ago physicists were writing special applications to simulate the move­
ments of 1 00,000 to 1 ,000,000 stars by treating them as mass densities distributed
continuously across space, rather than as individual particles. In 1 9 1 1 , Plummer
invited a " softening" technique that replaced the dynamics of particles with the
dynamics of fluids. In fact, the most recent work by Mori and Umemura is a good
example of using hydrodynamical equations to articulate the physical processes in a
galaxy colliding with another. There is always a contest between simulating discrete
particles and simulating continuous distributions of matter. Each can be derived from
the other.

Conj u ring Bodies-Not as Metaphor but as Fact of Matter

Every time we slip on a shirt, we slip on a second skin. Every time we speak through
a cell phone we speak in another voice and scatter ourselves in ether. Every time we
massage oils and electromagnetics into a body, we make it porous, sometimes with
S u bstrate 119

seductively perilous effect. But where does the body end and the world begin? What
does it mean to move, to embody, to be moved? What becomes of bodies, persons,
and subjects when we take our drugs, our computers, and our nano-toys from the
same needle? In fact, what becomes of our bodies and our selves when computers,
biotech, and nanotech are woven into the very fabric of our built world? We conven­
tionally think of the body making gestures, but how can we imagine gestures making
a body? As microcameras and sensors and sensate or luminous materials become
ubiquitous, the space itself between us becomes a sensing and kinetic tissue that
extends our expressive bodies. How can wearable, responsive textiles and temporally
textured fields of sound and light trace and transmit playful gesture?
The Topological Media Lab has worked since 2002 with active-sensate, emissive,
kinetic-fabrics, wireless sensors, wearable synthesized sound and image, to present
works responding to these questions. The interventions have ranged from scientific
research on basic gesture and the engineering of wireless sensor platforms, to parodies
of the implausible logics of the fashion shows in electronics trade conventions, to
instances of performative art. (Again, by "performative art" I refer to events con­
structed in which all the participants are equally performing as well as spectating
agents.)
Often, however, performance works assume we know what a body is, where a body
ends and the world begins, what we mean by being embodied, by movement, affect,
emotion. But technologies like medical imaging, endoscopic surgery, bioelectronic
prostheses, wireless sensors, and fabrics that can sense, display images, and move are
thickening the world between the skin and the walls of the city, between what we
called self and nonself.
What becomes of bodies, persons, subjects, and subject positions when computa­
tional and biotechnical or nanotechnical mediation becomes so thoroughly dispersed
into the fabric of our built world? What about desire or agency, human and nonhu­
man? To pursue this more rigorously, we can turn to topological media and dynamical
processes (on manifolds) .
Having said all this, nonetheless, I should respond to the would-be objector that
of course obj ects and categories exist. My concern is not to deny that they exist but
to understand how they come to be and how they pass on. Can we construe the object
in a way that (1) accommodates change, (2) spans all ontological strata as Guattari
envisioned them, and (3) accommodates non anthropocentric subjectivation?
Consider, for example, in creating real-time sound accompanying a dancer's move­
ment, the difference between motion capture or body tracking and sound as a respon­
sive texture. Expressed as a circle of real-time media processing (which seems to the
human performer simultaneous with her or his movement), this includes: ( 1 ) process­
ing the signal from the moving body (via any sensors, cameras, etc.); (2) modeling
location or body shape, i.e., geometric models; (3) parametrizing sound synthesis
1 20 C h a pter 4

models (e.g., in MSP)i (4) gesturally improvising with sonic textures. In our practical
work, we have created richly articulable media without building high-level metric
models such as anatomical skeletons for human movement.
Consider these three examples of what a substrate perspective yields in the domain
of working with movement and responsive media:
(1) Instead of modeling for example the center of gravity-a point location-or j oint
angles or some other geometric information about the body, consider processes that
respond to the entire play of light and dark across the camera's field of view. This can
disregard the identity or pOint location of "bodies. " In TGarden, this indifference to
identity and constancy of body (via pOint location) permitted the playful exchange
of "wings. "
(2) Instead o f tracking gaze, just make sound a s a function o f angle o r Hausdorff
distance between blobs. Then shadows can become ad hoc instruments, props con­
structed or, better, ascribed with use in the moment of performance. In play, shadow
can transmute from a negative absence of (projected) obj ect to become a positive
thing.
(3) A more subtle example is the transfer from support to supported in a contact
improvisation dance exercise, which the two dancers (or two parts of a mover's body)
can feel. However, this transmutation of the balance of force may be essentially indis­
cernible to an optical sensor, such as a video camera.

An Object Is an I nvariant (of a lie G roup)

Earlier in this chapter, we explored two large sets of challenges problematizing the
notion of obj ect and obj ect-oriented ontologies: formal complexity and ethico­
aesthetic adequacy. As I have said, I do not question the existence of objects or of
their roles in social and technical practices. Instead, I suggest a complementary orien­
tation to the emergence, formation, and transformation of objects.
Take this cup in your hand. Seen from above it is a disk. Seen from the side it is a
rectangle, with a semicircle attached. So the apparent shape is not what makes the
cup the cup. When you see it from all angles, what is the essence of this cup? Not the
shape, apparently. Take this orange in your hand. What color is it? Walk with it under­
neath a green light and what will you say? "Orange, " of course. But what would a
spectral analyzer read? The orange would appear nearly black, because it reflects
orange frequencies and absorbs the complementary frequencies of light, namely green.
Seen under lamps with different spectra, this piece of fruit would reflect light with
different spectra. So what makes an orange an orange is not its color, apparently.
Consider this person. These are the first exercises toward a phenomenological inves­
tigation, enormously elaborated since Husser! and Heidegger.39 But here I stop, because
Substrate 1 21

we will take a different branch. While it will be extremely useful to be able to apply
phenomenology's "transcendental reduction, " where we part from phenomenology is
where it presumes the ego cogitans, centering questions of experience on the Dasein,
which is concerned irreducibly with (human) subjectivity.
Alain Connes, Fields medalist and one of the masters of noncommutative geometry,
speculated in a conversation with Jean-Pierre Changeux40 that objects of consciousness
could be explained as topological invariants of some sort. This speculation, however
informal, struck a chord with me, but of course such fancy leaves mysteries in its wake.
What sort of invariant do we mean here? In what topological space live these invari­
ants, and under what action?
Jean Petitot, following Husserl, develops a mathematical "model" for progressive
aperspectivalization (a technique of eidetic variation) which converges on a limiting
object. But the substrate need not, and perhaps (if we examine it carefully with the
same optics that expose the cracks in naive realism) cannot, be neural matter or merely
neural processing. Why not? One reason is that even dust under gravity or mud and
gravel diffusing under water can aggregate themselves into obj ects. Petitot's ontologi­
cal project lends substance to this claim, and we shall examine it more closely, after
preparing the ground to understand it.
5 Ontogenesis

Ullhodturdenweirmudgaardgringnirurdrmolni rfenrirlukkil okkib augimandodrrerinsurt­

krinmgernrackinarockar
-James Joyce, Finnegans Wake

In our lived experience, the world is in a constant state of transformation: our blood,
our breath and speech, our capital, our legal codes, our languages. In fact, as mainte­
nance crews and electrical engineers well know, even the apparent stasis of our built
structures and of our core computer memories is the product of analog processes of
stabilization. At the level of our basic, embodied experience of the world, the processes
are continuous: as I walk toward you across the room, my feet do not pop discontinu­
ously from position to position, and my experience of you does not flicker in and out
discontinuously. As I lift my hand to pick up a cup or to wave hello, it does not jerk
discontinuously from point to point and moment to moment, at least not in ordinary
experience. And it is ordinary experience in its boundless density with which I am
concerned ethico-aesthetically, technically, and conceptually. Of course, for reasons
of efficiency and economy, we have built logical and algebraic systems of discrete
representation of various ontological strata of the world (phyletic taxonomies, the
periodic table, grammar), and processes and institutions that discipline practices to
these fixed representations. But the practical challenge I set is to see how to articulate
continuous processes in our art and our technology, given how continuou\ material
process is pervasive in our lived experience. Why material process? Following a practi­
cal approach that also retains the possibility of a thoroughly experimental, revisable
access to the means of our expression, I wish to appeal to material and consequently
immanent processes, rather than transcendental schema such as God, natural law, or
human nature. The transcendental is in principle inaccessible to our material, acci­
dental condition.!
1 24 C h a pter 5

M aterial Process

Gilles Deleuze, Elizabeth Grosz, and Mark Hansen, among many, have referenced their
discussions of time and temporal experience to Henri Bergson's critique of duration
and simultaneity. Many cite Bergson's opposition of the durational aspect of lived
experience to the "cinematic" conceit of time as the union of a sequence of instants,
and many in turn cite Deleuze's Cinema 1 and 2, which attempt to think outside the
category of atomic differential operators on unidimensional time. Mary Ann Doane2
and Orit Halpern3 have appealed to psychoanalytic and other sources as well, embed­
ding discussions of time in more general treatments of consciousness and its shadow.
Notwithstanding the deep insights of diverse theorizations of time, I propose to side­
step the linear thrust of a direct theory of time in favor of looking more closely at
aspects of experience and world usually thought of as temporal: matter in transforma­
tion or material process. The challenge is to avoid relying on psychologistic and
therefore anthropocentric accounts of temporal experience.4 I will suggest a way to
articulate material process and transformation using the concepts that we have seen
in the previous chapter, guided by the specific concept of ontogenesis. By ontogenesis
I will mean the emergence of shaped material in its material substrate; ontogenesis
will always be a material, not formal, process.
Before we launch into the positive part of the discussion, let me make an observa­
tion about the industry of time. In the appendix to their book Autopoiesis and Cogni­
tion, Humberto Maturana and Francisco Varela explicitly argued that time, characterized
as the sequentialization of events, is an artifact of an autopoietic organism's "descrip­
tive domain, " and not present in the material structural operation of that organism.
Their argument is sufficiently subtle to quote in full:
Any mode of behavioral distinction between otherwise equivalent interactions, in a domain that
has to do with the states of the organism and not with the ambience features which define the
interaction, gives rise to a referential dimension as a mode of conduct. This is the case with time.
It is sufficient that as a result of an interaction (defined by an ambience configuration) the
nervous system should be modified with respect to the specific referential state (emotion of
assuredness, for example) which the recurrence of the interaction (regardless of its nature) may
generate for otherwise equivalent interactions to cause conducts which distinguish them in a
dimension associated with their sequence, and, thus, give rise to a mode of behavior which
constitutes the definition and characterization of this dimension. Therefore, sequence as a
dimension is defined in the domain of interactions of the organism, not in the operation of the
nervous system as a closed neuronal network. Similarly, the behavioral distinction by the observer
of sequential states in his recurrent states of nervous activity, as he recursively interacts with
them, constitutes the generation of time as a dimension of the descriptive domain . Accordingly,
time is a dimension in the domain of descriptions, not a feature of the ambience [emphasis added) .s
Ontogenesis 1 25

In other words, an observer invokes "time" to describe a living organism's behavior:

time is part of the descriptive apparatus of the observer, not an aspect of the material
world "out there. " Less psychologistically, in place of "behavior" we can say "variation
of state with respect to a given state in the course of its material interactions with its
ambience. " As suggested in the passage, variation of state can be conventionally
thought of as a sequence SI1 S 2 , S 3 ,
• • , but this is far from the only way to think about
•

it, as the modes of articulation discussed in chapter 6 would suggest. And, given the
density of concurrent temporal processes, the sequential interpretation is not ade­
quate. Nonetheless, Maturana and Varela's argument rewards careful study, and justi­
fies setting aside an explicit theorization of time as a descriptive instrument that
reduces the temporal aspects of experience to a representational abstraction. The
insight to retain is captured by Maturana and Varela's opening motto: everything that
is said, is said by someone, which has a very strong implication: there are indeed observ­
ers, and what seem like objective features of the world are actually part of the descrip­
tive schemas employed by those observers. (As we also see from quantum mechaniCS,
it is no longer analytically clarifying to pose an ontological distinction between sub­
j ective experience and objective material existence.)
In this chapter, we seek a way to articulate process without reducing it to event
sequence, system, cyberneticS, networks, or informatics-all concepts of the formal,
and consequently alien to the living.6 The formal concept of time reduces the material
dynamics and the experience of the world to what Christopher Alexander called a
dead phYSiCS, a physics that construes matter to be distinct from the category of life
and value.7 But even a phenomenological investigation, carefully attentive to the
qualities of our human experience of the temporal, or to the temporality of our experi­
ence, is problematic. In his Metaphysical Foundations of Logic, Heidegger referenced
Husserl's most sustained analyses of what Husserl called internal time consciousness,
aligning Husserl's structure of remembrance, presentification, representification, and
expectation,8 and pointed out that in the end Husserl could only ground this back in
an account of the subject's consciousness, and therefore could not really account for
time in its essence as such. Heidegger writes: "We . . . call primordial time temporality
in order to express the fact that time is not additionally on-hand, but that its essence
is temporal. This means that time 'is' not, but rather temporalizes itself. "9 :�
By my lights, theorizing time "in itself" risks a reification error. Heidegger cannily
suggested replacing time by "temporality" to avoid such an error. But it's also an error
to explain "time" solely in terms of the experience of a human subject, for those of
us who live and make things as if the world, which includes us, unfolds, evolves, mutates
without us humans narrating the action. Performers-dancers, musicians, movement
artists, and many creators of time-based media-work this way. Can we instead under­
stand temporal process as part of a more ample notion of material transformation,
1 26 C h a pter 5

drawing from topological dynamics to articulate how objects (material things) dissolve
and form in the material field? This is what I call ontogenesis. Heidegger's observation
about temporality stands in sharp conflict with those notational schemes that are
intrinsically changeless over (local degrees of) time: including just about all forms of
writing. The challenge is whether there exist any notational schemes that are intrinsi­
cally temporal. I will not pursue the questions of time, temporality, and time con­
sciousness further here, because my purpose is to sidestep the industry of time studies,
especially those that retain the triadic ordering-"expectancy, retention, making­
present'1 l0-that implicitly derives from a unidimensional conception of time, l 1 and
that require a primordial concept of human subjectivity or Dasein.
I develop this account of ontogenesis setting out from A. N. Whitehead, visiting
Christopher Alexander's recent work, and landing in Michel Serres. One of my funda­
mental motivations is in fact a return to a tissue- or field-based extension of biopolitics
that respects the autopoietic qualities of enlivening systems but does not reduce to
just mimicking biological, informatic, or graph-theoretic schema. In Process and Reality,
Whitehead constructed his ontology unequivocally as a philosophy of process: " How
an actual entity becomes constitutes what the actual entity is. " 12 This process is marked
by the production of a novel occasion out of many occasions, the "production of
,,
novel togetherness. 13 Much space is devoted to what one might characterize as a
synchronic description of ontogenesis, but Whitehead also dedicated the third quarter
of Process and Reality to detailing the dynamic mechanism of his ontogenetic process.
To make this relatively self-contained, I will summarize Whitehead's construction,
then focus on his fundamental appeal to the principle of least action and contrast it
with four other ontogenetic principles. In his late and philosophically most evocative
work The Nature of Order, Alexander arrived at the same central problem of ontogenesis
from the practical art of building living structures via living processes exfoliating in
space and place, explicitly revealing his relation to Whitehead and Leibniz. However,
Alexander recognized several alternatives to the principle of least action-emergence
in complex dynamical systems; richness through evolutionary process of adaptation
or selection.14 Alexander remarked that there must be a ontogenetic dynamic beyond
those governed by "energetic" action, which he equated with the category of number­
measure. He therefore conjectured a fourth principle, a ontogenetic unfolding defined
with respect to a slippery concept of wholeness that is thoroughly Whiteheadian.
Against these four ontogenetic principles I would contrast a fifth, a Serrean noise of
excess, which has nothing to do with the abstract randomness of information or
complexity theory, but echoes the clamor of the sea.
Supporting lS this story are various concepts circling the principle of least action,
which we can relax using topological dynamics from its brittle reliance on an equa­
tional form. Indeed, Poincare's descriptions of dynamics can be phrased in topological
terms, without need for explicit equations. And consequently, we can articulate change
Ontogenesis 1 27

without requiring the calculus (differentiability) via the concept of homotopy. With
Poincare we can turn to Whitehead again and articulate a suitably an exact version of
the latter's processual theory with no a priori entities.
But if there is more to process than the principle of least action, as there must be
if we credit, for example, Serres's multitude and noise or Bataille's account of super­
productive excess in sexuality, sacrifice, and symbolic economy, how can we not just
name this excess (there can be no singular name for the plague, death, and noise,
anyway) but articulate it? To build from what Serres fleetingly touches in Genese, a
deeper form of noise in excess of the random, I call upon not just topology but
measure theory. This is the first point in the book in which we substantially draw
from a mathematical ontology that is neither discrete nor continuous, yet rich enough
to evocatively articulate some aspects of a noisy ontogenesis that may partially trellis
what William James called the "blooming, buzzing confusion" of our experience in
the material dynamical world.

Ontogenesis: The Problem of Novelty

Ontogenesis as a concept mediates matter-energy dynamics, temporality, value forma­

tion, and lived experience. Ontogenesis is less dry, less abstracted than bare time or space­
time, and closer to the phenomenological feeling of becoming in the material world.
Given these concerns, as I have described them to this point, what is interesting is
not so much a matter of taxonomy, of schemas and classifications or standards and
protocols, although those are necessarily part of the robust construction and operation
of our play spaces, but rather the dynamics of processes that stir up, shape, and
unshape the material patterns that constitute the lifeworld.
What the theories of ordinary and partial differential equations give us is a way to
articulate processes of change and ontogenesis in rich but not blinding detail. Treating
these in the larger category of differentiable manifolds, at least conceptually-reserving
for later the relaxation to more general categories using geometric measure theory, or
to more specialized applications using computational models-rather than the canoni­
cal Euclidean space Rn allows us to articulate a wide range of nonlinear phenomena
spanning objects and processes. This is already the case in practical applications such
as computer vision, speech recognition, and satellite kinematics; the question is what
implications are opened up by such perspectives in philosophical thought after Deleuze
and Guattari.

Process in the World

Given the articulations afforded by topological dynamics, can we reverse-engineer

systems theories and the host of related rhizomatic metaphors to shed light on Wiener,
1 28 C h a pter 5

von Foerster, Luhmann, Prigogine, Smale, Ruelle, Petitot? And can we do this without
committing, by adhering to a particular system of measurement, what Whitehead
called the fallacy of misplaced concreteness?16
As we saw earlier, in an essay tracing a current of dissident thought back to the
contemporaries of Isaac Newton, Akeel Bilgrami identified the key difference between
these dissidents and Newton as resting on the status of matter. Whereas Newton and
Boyle regarded matter as brute and inert, John Toland and his peers "thought of the
world not as brute but as suffused with value. " 1 7 Rather than locating moral agency
in a transcendental authority, this implies the necessity of an ethical (Bilgrami more
strongly calls it normative) disposition with respect to the world. We will return to
this concern at the end of the book.
My concern is with ontogenesis as material dynamiCS acting more primordiallyl8
than any analytic or cognitive schema, where "material" means matter suffused with
value, to use Bilgrami's felicitous phrase.

Whitehead

In Process and Reality, Whitehead's core strategy was to recast relativity theory as an
ontological theory in which the world comes in "unbifurcated" chunks of feeling,
value, and matter, his "entities" or "actual occasions. " In fact by making entities and
occasions synonymous, Whitehead treated event together with place.
I trace a set of mathematico-poetic figures from Whitehead's Process and Reality in
order to understand how he constructs a theory of the world that prehends, feels, and
becomes social. My tracing centers on two principal questions: How does Whitehead
construct a philosophy of process and organism on mathematical intuitions that retains
nonetheless all the living qualities of the unbifurcated world? And to what degree and
in what manner does he construct his pata-mathematical concepts as a mathematician
constructs and fabricates concepts? Given his philosophical and theological inheri­
tance, Whitehead responded simply and remarkably to some of the most provocative
mathematics and mathematical physics of his day: Russell's and Cantor's set theory
and Einstein's general relativity. But if he seems to respond too bluntly in some
respects, to what philosophical purposes-not scientific or mathematical-does he set
his speculation? I will try to extend Whitehead's speculation using "lures for feeling"
made from measure theory and topological dynamical systems and outline a notion
of process that does not appeal to obj ects (given the motivations set by chapter 3).
I argue that elaborating Whitehead's speculation a few steps beyond his artfully
blunted set theory and general relativity theory yields a way out of the static and
atomistic aspects of his metaphysics. Indeed, such a sympathetic extension should
substantially enrich a plenist and process-oriented concept of unbifurcated nature that
more readily accommodates local novelty.
Ontogenesis 1 29

I came to Whitehead after thinking with Deleuze and Guattari's multiplicity,

Deleuze's appropriation of Riemannian manifolds, and the asignifying semiology of
Guattari's chaosmosis. I assure you that my reading is not some truth-seeking missile
but a speculative and poetic exercise in thinking through Whitehead's philosophy of
process and with the alchemical accompaniment of all those nonhuman, mathemati­
cal objects, like the monsters of set theory (pace Alain Badiou) and the point-free
topologies of Rene Thom and Alexander Grothendieck, whose more fertile philosophi­
cal consequences have hardly been adequately developed, I believe.
In their Heraclitus seminar, Martin Heidegger and Eugen Fink tried to steer a middle
course between a close, closed hermeneutic study of Heraclitus's fragments and a free­
associative "philosophizing" with the putative sense of the Greek text. Heidegger and
Fink's seminar used the texts to develop a process theory that honored what they
found in Heraclitus but also extended their phenomenological investigation. It seems
worthwhile to read Whitehead in an analogous constructive and productive spirit to
develop a topological approach to a process world.
First let me rapidly rehearse Whitehead's ontology as he develops it. In retracing
Process and Reality's argument, we can detect, albeit faintly, what a mathematician
might recognize as the rhetoric of proof. These features include: preliminary motiva­
tions established as definitional " assumptions, " paradigmatic examples, and a network
of lemmas, theorems, and corollaries. Whitehead uses such labels almost nowhere
because he supplies almost no arguments with the robustness and precision of a
mathematical proof. (There is no call for actual mathematical argument of course, and
in fact, despite the formal precedents of Spinoza's Ethics, Newton's Principia, and Russell
and Whitehead's Principia Mathematica, such rigor probably would sink the speculative
enterprise.) Whitehead deploys a surfeit of assumptions, rather than finding a minimal
model. One can see a paradigmatic example of this in his extravagant development
of abstractive sets, about which his assumptions run into the dozens.
One difficulty is that his conceptual edifice is a set of coconstructive notions: actual
entity, prehension, concrescence, nexus (society), and later on, apparently still more
abstractly, ovate sets, abstractive sets, strain, duration. But to an archaeologist of
mathematics, elements of the mathematical physics of that era figure prominently in
Whitehead's construction, and it seems fruitful to understand what philosophical juice
he extracts from those elements.
Whitehead begins with the ontological principle: everything comes from some­
where; nothing comes from nowhere. Philosophy must start with the concrete and
explain abstraction, not the reverse. One cannot derive the concrete from the abstract
or the ideal. And the concrete does not come to us already split into symbolic and
material categories. Indeed, nature is unbifurcated, it is a single, plenist ontology in
which matter, matters of fact, feelings, subjective experience, the experiencing subj ect,
and experienced entities are deeply entangled. (Based on the approach introduced in
1 30 C h a pter 5

chapter 6, we can articulate entanglement in a fieldlike way.) Bifurcation would split

the world into causal, objective nature and a perceived nature.
However, he insists that our experience of matter comes as actual entities, because,
as he put it, " [c] ontinuity concerns what is potential; whereas actuality is incurably
atomic."19 The heart of his argument rests on an appeal to intuition and first
impressions:
In their most primitive form o f functioning, a sensum is felt physically with emotional enjoyment
of its sheer individual essence. For example, red is felt with emotional enjoyment of its sheer
redness. In this primitive prehension we have aboriginal physical feeling in which the subject
feels itself as enjoying redness [emphasis added] .zo

Contra Hume, Whitehead posits that these " aboriginal" feelings do not spring up from
unknown causes but from actual occasions transmitting or conducting feelings vecto­
rially to one another. However, this vectoriality is a weak one, from determinate enti­
ties to those yet to be determined, a topological version of time anisotropy. And about
the dynamics of actual occasions he writes: "The sole appeal is to intuition. "z1 White­
head motivates the direct apprehension of actual entities as whole obj ects rather than
as composites with his amusing observation that one dances with a whole human
partner, not with a cloud of flickering sense data:
A young man does not initiate his experience by dancing with impressions of sensation, and
then proceed to conjecture a partner. His experience takes the converse route . . . . The true physi­
cal doctrine is that physical feelings are in their origin vectors.Z2

But this microversion of the anthropic principle, for that is what it is, gives the object­
oriented argument a whiff of the tautological.

Concrescence
For Whitehead, the becoming of an actual entity, akin to Heidegger's anwesen, is actu­
ally constitutive of that entity, so process, change, and indeed duration are intrinsi­
cally part of the raw material of his ontology. Indeed, this process of becoming is a
basic, primitive element of his ontology: it "cannot be explained from higher order
abstraction nor be broken into constituents. "
Rather than a metaphysics o r a theory o f knowledge predicated o n sense data, this
is an account of a phenomenology based on embodied experience:
For the organic theory, the most primitive perception is "feeling the body as functioning. " This
is a feeling of the world in the past; it is the inheritance of the world as a complex of feeling;
namely, it is the feeling of derived feelings . . . . The body, however, is only a peculiarly intimate
bit of the world. Just as Descartes said, "this body is mine"; so he should have said, "this actual
world is mine. "z3

Whitehead prepares the ground for an ontology that does not bifurcate between body
and unfeeling world, or between local and global. His unbifurcated ontology is com-
Ontogenesis 131

posed of occasionally infinite hierarchies of nested entities vivified by relations of

feeling and sensing in time. This is materially where all the action lies. Using the
notions of collectivity, "nexus, " and " society, " he tries to generate rich structure in
the world in an unbifurcated way, but whether they are set-theoretic or, perhaps in a
richer sense, category-theoretic notions, in any event they are built out of discrete
entities, on points rather than continua. Even in the limited (but infinite) world of
mathematical logic and set theory, logicians face the essentially unavoidable technical
difficulty of producing the continuum from a set of pOints.
Whitehead speaks of all actual entities (not just live versus nonlive organisms) as
having their concrete properties and characteristics reproduced in what he calls prehen­
sions. Every character is reproduced in a prehension, and, most importantly, there is an
indefinite number of prehensions. This indefiniteness yields a radically open meta­
physics. A prehension is directional, hence has a "vector character, " and, unlike raw
,,
sense data, "involves emotion, and purpose, and valuation, and causation. 24 Via these
prehensions, sensing, feeling, and pulling, the actual entities are engaged in the "pro­
,,
duction of novel togetherness, 25 the coming together of many actual occasions into
the novel actual occasion, a process that he calls concrescence. Whitehead constructs a
temporal theory of process to suit this dynamic of the world using a notion of past,
present, and future that does not rely on metrized, geometrized clock time, but on more
elementary, topological notions of causal past, causal future, and the acausal comple­
ment in space-time. The acausal complement to an event is that part of the world whose

F i g u re 5 . 1
Causal future and causal past based a t a space-time event E. Diagram b y author.
1 32 C h a pter 5

occasions cannot affect or be affected by the event. Whitehead's causality is infinitely

richer than the physicist's test of accessibility by light (along the geodesics with respect
to the space-time manifold), but it formally parallels the logic of general relativity. This
should be familiar to readers of Stephen Hawking and G. F. Ellis's classic The Large Scale
Structure of Space-time,26 in which they demonstrated the expressive and explanatory
power of a topological approach to space-time, accommodating even places of material
singularity where the space-time curvature and matter-energy densities increase without
bound. In brief, topology articulates what exceeds number and metrized geometry.
I pass silently over Whitehead's construction of the "immediate" present, although
it is one of the most important aspects of his treatment of time, and fold the discus­
sion of time into a consideration of the larger dynamics of organism on which he
rests his process theory. For the moment, we note that Whitehead tries to maintain
his unbifurcated ontology by describing prehensions as feelings in time, and by asso­
ciating groups of entities (nexus) with societies-making the multiple social by propo­
sitional fiat. The atomicity of his actual occasions yields, in his concept of the world,
a notion of time that is correspondingly atomic. Note that there is no condition on
the " size" of these atoms: Whitehead's entities are not necessarily small or brief; his
atoms are neither nano nor macro. However, any atomistic conception of the world
that purports to account for the richer structures evident in our experience inevitably
needs to posit dyadic and more generally n-adic relations between these atoms. Given
the atomicity of Whitehead's ontology, it is not surprising that he maintains that
actual entities' concrete experience must be vectorial, Le., directed. We are not very
far from Leibniz, except that Whitehead's monads are changeless and Whitehead's
places are themselves immovable. This is a profound difference . If, by the ontological
principle, everything actual is made of atomic, changeless, actual entities, how then
can we account for change, and the potential for novelty and creativity?
In one of his most significant coercive neologisms, Whitehead makes " actual
entity" synonymous with "actual occasion. " It is important to recognize here, as
Stengers does, that Whitehead is making a propositional, conceptual identification,
not a metaphorical one. There are arguable and even demonstrable consequences to
such an identification. Stengers would say, rightly, that this is not a "merely" meta­
phorical identity, but it does seem that Whitehead deploys the term "occasion" to
infuse temporality by connotation as well as fiat. That being said, we can still ask
whether an equation like Whitehead's "actual entity actual occasion" derives from
=

and exerts different conceptual forces on those who think it than does the tensor
equation that articulates Einstein's equivalence prinCiple:
G = 8:rrT,
where G is the metric tensor for space-time and T is the stress energy tensor for the
dynamic distribution of matter-energy in the universe. Arguably one of the central
Ontogenesis 1 33

identities in the theory of general relativity, this equation articulates Einstein's equiva­
lence principle, the equivalence between geometry of space-time and matter-energy
in motion. This tensor equation encodes and implies a large, conceptually definite set
of algebraic symmetries and conditions that on one hand are invariant over multiple
subj ectivities and on the other hand express a certain belief about ontology: that
the geometrical structure of the world is identical with the dynamic distribution of
the world's matter-energy, and that this identity works at the level of dynamics. By the
way, the geometry encoded by the curvature tensor G in Einstein's equivalence is not
the geometry of space but that of space + time, which includes the temporal in a
single, unbifurcated manifold. This sort of geometrization is not the geometry of Berg­
son's critique of geometrized time. (We will see later that Whitehead's measurement
reverts to the geometry of spatial Euclidean space.) Whitehead's blunted version of
Einstein's equivalence principle forgets this dynamic, I believe.
Whitehead uses the distribution of strain acting on what he calls flat loci to provide
the dynamic for the world. We see the strong parallel between his technical descrip­
tion of strain and geometry in the latter part of Process and Reality and the account of
concrescence with which he starts the book. But this attempt to provide a dynamic
is predicated on a curious reappropriation of force. Force is vectorial, and as such aptly
traces the directional modes of consciousness, but it is not coherent to glue vectors
of force to particular geometrical obj ects such as flat loci, because generally they
operate in different modalities of the plenum world. To use a concrete example: the
vectorial difference between a dog and a cat, however it is measured, is not a dog, or
a cat, or any actual mammal. Moreover, a vectorial theory does not account for modes
of consciousness that are not so directed as to be easily subsumed by a vectorial
account of prehension and of concrescence. Whitehead himself describes this as the
gradual objectification of vectorial experience into a " scalar" form, which is an inspired
way to gloss what in phenomenological terms would be nonperspectival apperception.

Dynamics
A strain yields movement resolving the strain. Whitehead identifies duration with
strain (duration being a [topologically] complete set of mutually contemporary actual
occasions) . But this Aristotelian (or, in terms of mathematical physics, zerdth-order)
equivalencing introduces two mysteries. The first mystery is mismatched physical
dimensions: energy is force x distance or, more finely, the integral of force along a
trajectory, and conversely force is spatial difference of potential energy. On the other
hand, duration is measured in time, which is not commensurate with the units (Le.,
" dimensions") of force at all. This seems nonsensical, and so needs an argument at
least as convincing as Einstein's argument for the equivalence principle.
The second mystery is that Whitehead gives no reason why such an equivalence
should obtain, and he makes no observation about what it would afford the world.
1 34 C h a pter 5

In fact, general relativity can be articulated in the same way via the language of dif­
ferential geometry. That is, any local coordinate region in a space-time manifold is
always automatically "fixed" and invariant with respect to time because, by definition,
it carries itself. This is by definition a feature of all space-time geometries, Whitehead­
ian, Einsteinian, or otherwise. To elaborate, a space-time manifold is a topological
space that locally has the geometry of a four-dimensional space with three spatial
dimensions and a fourth dimension with the opposite signature. This is a local condi­
tion, so that what constitutes the "temporal direction" can vary continuously as we
pass from event to event. This negative signature means that there is a qualitative
difference between trajectories that flow temporally, those that flow spatially, and
those along which there is zero space-time metric displacement. The last sort of tra­
j ectories is exactly the set of geodesic paths traversed by light. The immotility of place
characterizes any locus-any neighborhood of an event-on any such manifold, so
Whitehead's argument for the fixity of actual occasions does not select his account
over Einstein's general relativity.
Somewhat surprisingly given his concern with process, Whitehead does not appeal
to the calculus, in particular differential calculus, until very late in Process and Reality.
There is a key moment when he argues against a purely geometric interpretation of a
line element ds integrable to distance s along a curve y, and proposes impetus (or
impulse), which would intertwine matter and momentum. This intertwining echoes
his 1 922 book The Principles of Relativity, in which he writes about "adjectival
,,
particles. 27
In an essay comparing Whitehead's and Einstein's theories of general relativity,
Yutaka Tanaka insightfully says that Whitehead makes matter an "adjective" to space­
time. While this is quite a suggestive borrowing from Whitehead, in more precise
terms Whitehead's most striking contribution is to insert a factor I[s] into the line
integral to form what one could interpret as an "impulse density" I[s]ds along a trajec­
tory.28 What if 1 were not a constant like " 3 " or "green" but a (unction varying according
to the space-time locus and disposition?

F i g u re 5 . 2
Integral along an arc y. ){s] parameterized by arc length s. Diagram by author.
Ontogenesis 1 35

Parenthetically, Whitehead applies the calculus of variations to what he calls "total

impetus"-a sum of matter kinetic energy and electromagnetic field energy along a
path g[tl from X l to X taken by a particle with charge q, with electric field given by
2
the scalar electric potential and magnetic vector potential A : R3 -7 R3 . 29 The impulse
(action density in more recent language) is given by the Lagrangian:

�
J[tl = -mc2 1 -
v2
c2
- q¢ + qx · A

where speed v = v[tl , electric potential ¢ = ¢ [x [t] , t], velocity x = x [t], and vector poten­
tial A [x [t] , t] are functions of time t and position x [t].
We can define a total action (Whitehead's impetus) by integrating (accumulating)
the density as a function of time:
f
5 = J [t] dt.

But what we need is some insight into why this integral is important for his project
at all. That motivation comes from the variational principle of least action of dynamiCS,
which states that among all possible paths r j oining A to B, the one actually taken is
the one that minimizes the total action I[ rl . To be clear, the principle of least action
is far more general than the particular form taken here, but I am following the standard
model from classical mechanics in order to present this principle in a form familiar
to modern eyes. However, what we should expect is some discussion of the philosophi­
cal adequacy of appealing to any variational principle whatsoever, because this under­
lies much of physics, and in this case metaphysics. I expect that Whitehead should
disallow an apparently transcendentalist appeal to the principle of least action because
it would contradict the ontological principle's injunction to start with the "concrete, "
which according to Whitehead is denominated in atomic, unchanging actual entities.
As this lies at the very heart of the question of ontogenesis, let me describe the rest
of Whitehead's arrangement, and return to a fuller discussion of the least action prin­
ciple at the end of this chapter.
Now, in order to intuit Whitehead's derivation of dynamiCS we need to comprehend
the intuition behind getting kinetics (and more generally, dynamics) from q potential
field. Force, which is directional, can be thought of as a spatial difference of potential
energy, a field that to each position associates a directionless number. This is a scalar
magnitude with no associated direction in both Whitehead's as well as conventional
mathematical usage. But since energy fields can be a function not only of locus but
also of directional entities, i.e., vectors (think of how much easier it is to swim with
a prevailing current than against it), we need some sort of measuring device that would
yield a potential energy field from positional loci and directional entities. The simplest
such machines are abstract algebraic functions, called tensors, which provide a linear
response to vectorial parameters. (And here I use abstract in the same sense that
1 36 C h a pter 5

Deleuze and Guattari mean when they write about the abstract linguistic machine of
language.) Analogically, linear response implies that doubling the prevailing wind
doubles the energy expended to swim against it, etc.
We can obtain a nonlinear behavior by making nonlinear functions of the compo­
nents of tensors. So, evaluating tensors on vector arguments at various loci yields a
field of numerical values, a scalar potential field. If we imagine the states of a quasi­
physical system evolving in time as particles on a scalar potential field, the least action
variational principle derives change by having the particles sliding "downhill, " from
higher to lower states of scalar potential.
We will see a concrete computational application of this principle of least action
in the very heart of the Ozone media choreography system that I designed twelve
years ago for the sort of responsive environments I envisioned that would sustain the
sort of object formation and dissolution consonant with the material dynamics
described in these chapters on poiesis, substrate, and ontogenesis.

Measurement
Whitehead needs to be able to measure his changeless, unmoving actual entities/
occasions in order to feed them into his dynamical apparatus-his zeroth-order
dynamics . And his Newtonian absoluteness will not allow him to resort to Einstein's
moving clocks and meter sticks. In lieu of moving measuring devices, Whitehead offers
a limiting process of fixed entities, traced by abstractive sets. But since what he wants
to measure is any entity or res vera, 3 0 he needs a more general sort of measure, and for
that he appeals to a construction on sets that does not assume anything special about
the metric, size, or geometry of what is being measured; in fact, he constructs "flat
loci" that are defined prior even to the " spatial" and "temporal" categories. White­
head's flat loci are generalizations of lines, more precisely of simplicial complexes,
analogous to the vectors and multivectors that serve as arguments to ordinary tensors.
Whitehead constructs his blunt version of lines from a simpler notion of extension,
which for him is captured by the union and intersection of sets. To this end, he tries
to build up "lines" as abstractive set limits of generalizations of planar ovals . But if
they are to measure any entity in the world, why should these model sets be two­
dimensional? It is notorious how properties for ordinary shapes in two-dimensional
Euclidean geometry can fail to extend to general sets in higher dimensions. For
example, a continuous closed loop in the plane separates the plane into two simply
connected components, an inside and an outside. Finding a mathematically credible
proof is surprisingly nontrivial (perhaps not too surprising after one considers the
topology of a Jackson Pollock painting) . However, a continuous image of a two­
dimensional sphere may fail to separate three-dimensional Euclidean space into two
simply connected components. Whitehead tries to generalize from ovals to "ovate
sets" with analyzable intersection properties, from which he can build any line like set
O ntogenesis 1 37

as a limit of intersections (on the way back to vectorial experience) . But why ovals?
The intersection of two ovals is usually not an oval; so the set of ovals is not closed
under the natural topological operation of intersection, and his attempt to generalize
to "ovate" sets seems rather awkward and confusing. The concept he is groping for is
convexity, because the intersection of two convex sets is convex. The difficulties ensue
from trying to force ovals and ovate sets into serving the general "measuring" purposes
of topological basis.
With hindsight, we can see that Whitehead's project is weakened by an insistence
on Euclid's ideal geometry of three-dimensional space. Topology would be more apt
for his proj ect, because it articulates notions such as containment, boundary, point,
density, intersection, union, and limit without appealing to number, measure, or even
dimension. It is not geometry but topology that is "the investigation of the morphol­
ogy of nexus."
Why limit the discussion to three-dimensional Euclidean space? In Process and
Reality, entities are so general that there is no call for measuring them by three­
dimensional sets at all. (Consider the set of all trajectories taken by the set of all people
on Earth this year.) Aside from the reliance on Euclidean 3-space, there are three other
challenges to Whitehead's approach to measurement.
The first challenge is a fundamental sequentiality in Whitehead's thinking about
limits. In The Concept of Nature, 3 1 where he tries to get at the pOints and linear subsets
of (Euclidean) space as limits, as convergence to an "absolute minimum of intrinsic
character" via his abstractive sets, Whitehead makes an unexamined relation between
geometric limits and analytic limits. Here I am using "analysis" in its technical math­
ematical sense.
Considering a sequence eIt ez, e3 , . . . , with associated qualities q(e1 ), q(ez), q(e3 ), . . . ,
Whitehead claims that the associated values converge to a definite limit, but this is
in general not true except for trivial sequences, like e, e, e, e . . . . In fact, the conver­
gence of a sequence of elements in a topological space depends on the topological, not
metric, properties of that ambient space, for example whether the ambient space is
compact or sequentially compact. We can use, for example, the theorem that every
closed subset of a compact set has a limit point, or that every infinite sequ�n ce has a

F i g u re 5 . 3
An unbounded video stream is an example of a nonconvergent sequence.
1 38 C h a pter 5

convergent subsequence in a compact set, or that a real-valued continuous function

from a compact set achieves its maximum and minimum on that set. None of these
require the existence of a metric, and therefore do not need a so-called epsilon-delta
proof from the calculus of the reals.32 Instead, topological theorems such as these can
be established only using the notions of open sets or neighborhoods. Therefore, in
particular, the integers and metric (particularly Euclidean) geometry do not provide
an adequate articulation of measure for Whitehead's dynamics.
A second challenge is that Whitehead relies on sequential compactness rather than
a notion of compactness that does not rely on enumerable series, subject to the
general, post-Pythagorean fixation on counting and countability. Roughly, sequential
compactness is the phenomenon in which an infinite sequence of elements or points
in a topological space contains a limit point: an element near which an infinite
number of its peers can be found, no matter how tightly one forms a neighborhood
containing it. Quite understandably, Whitehead indexes infinite sequences of sets
using the integers, whereas by the transfinite axiom of choice it is also possible to
index a sequence of sets from an arbitrary and uncountable index set. But this is a
subtle fact. A topology of open sets includes countable intersections and arbitrary,
possibly uncountable unions. Countability is important because it makes a difference
as to whether the intersection of a nontrivial descending chain of sets is null or not.
Whitehead characteristically overpowers the problem by piling on assumptions to
define it away.
Still a third challenge is that the construction of abstractive sets is a road mined
with confusion, where in fact a lattice-not graph-theoretic lattice, but the set-theoretic
notion-could serve as a radically fertile alternative. Lattice theory provides a way to
work with sets without any reference to constituent points, starting only with the
notion of partially ordering on sets A � B and the binary operations "meet" and " j oin. "
These operations generalize the intersection and union o f geometric subsets o f Euclid­
ean space. The most promising aspect of this approach to what mathematicians
sometimes humorously call "pointless topology" is the rigorous but nonrigid (or
anexact, as Deleuze would put it) way to describe extended sets and substances without
starting with atomic or pointlike elements.
I trace Whitehead's process at this level of detail in order to show how he blunts
mathematical process to serve his philosophical purpose, with artful but confusing
results. Given all these challenges, what could a repotting of measurement in different
mathematical soil offer Whitehead's account of process? Can we salvage measure­
ment? Mathematical analysis's measure theory provides an alternative, understanding
limits not merely as sequential limits but as points of accumulation, or as intersections
of infinite families of open sets. (Indeed those families can be higher-order infinities,
indexed not on some countable set but on transfinite index sets, like the field of
Ontogenesis 1 39

ordered pairs of real numbers or even the set of square-integrable functions on the
real line, which is infinite-dimensional.)
Geometric measure theory approaches classical geometric entities like geodesics
(locally length-minimizing curves in space-time) and tangent planes as limits of
infinite processes in much larger, wilder, even monstrous spaces of mathematical
structures.

Plenum and Process

Whitehead tries to honor a plenist spirit-explicitly acknowledging his debt to
Spinoza-but I think he still commits in the end to an atomism: "Continuity concerns
what is potential; whereas actuality is incurably atomic." His ontology is dogged by a
lack of access to the pOint-free topology constructible, for example, from lattice theory,
measure theory, and differential geometry, some of which was developed after The
Concept of Nature and Process and Reality were published. On the other hand Bernhard
Riemann's fundamental invention of differentiable manifolds and Poincare's work
with dynamical systems had already been circulating for decades.
Of course, every such observation about a philosopher's conceptual (not metaphori­
cal) use of mathematics should be tempered by the sympathetic understanding that
speculation or improvisation requires a trellis, not a carapace, in order to sustain the
imaginary. Therefore, alternative, nonaxiomatic constructions of ontology such as
Whitehead's appropriation and blunting of the stress-energy tensor, and his attempted
invention of what later was systematized under point-free topology, light the way for
further poetic exercises in the philosophy of process.
Whitehead also tries to honor the instinct toward process that characterizes an
understanding of the world alternative to that of atoms and synchronic taxonomy. He
is concerned, as he puts it, with the creativity "by which the many . . . become the one
,,
actual occasion" in a "production of novel togetherness 33 which he terms concrescence.
But the basic dynamic, concrescence-this process in which many unities become
integrated into a novel unity-being itself predicated on a discrete topology, is fun­
damentally as ideological a commitment as any metaphysics. It would be worthwhile
to see what ego-free phenomenology and languaging ("the tree greens" in�tead of "I
see a green tree") would be articulated by a topology that requires no points. It may
be difficult to imagine after decades of war declared by the logicians and engineers of
graph theory, but far richer patterns than enumerated sets and graphs are ready to
hand, such as Alexander Grothendieck's topos theory. In any case, even short of such
high-octane mathematics, Riemannian differential geometry and Poincare's dynami­
cal systems and topological dynamics from the perspective of geometry furnish rich
and suggestive handles for emergence, ontogenesis, and becoming. The concepts of
continuum mathematics require no appeal to a differencing of discrete entities, or to
1 40 C h a pter 5

discrete, computable algorithms. (I have in mind, for example, the Russian school
following Vladimir 19orevich Arnold's geometric analysis of dynamical systems.)

Concluding Remark on Whitehead

Whitehead considers Heraclitus's "All things flow" (fragment 41) and converts it to:
the vectorial impact of feelings of past entities upon an actual occasion being trans­
formed into " scalar" feelings. Mapping Heraclitus's " All things flow" to "All things are
vectors, " Whitehead comes close to a process theory adequate to life, but does not
quite reach it. Let me elaborate this process philosophy by taking three additional
steps, to see what unfolds. First, topology offers a way to articulate openness, neigh­
borhood, and, most deeply, continuity without committing in advance to dimension,
coordinate, degree of freedom, metric, or countable finiteness. Who articulates this
topological dynamic? I would say, with Deleuze, that it is not any subject or Subject
but rather the world that articulates. Second, in a deep sense, acting in the mode of
topology of continua obviates the recourse to counting, which is twin to number and
discreteness. Articulations of continuity, and continuous articulations as developed by
L. E. ]. Brouwer in topology, Heinz Hopf in global differential geometry, John Milnor
in topology and analysis of manifolds, and William Meeks in minimal surfaces exem­
plify some of these rich modes of material discourse.34
Heraclitus wrote also, "The earth melts into the sea as the sea sinks into the earth. "
This field-based process o f the world i s topological a s I have traced i t i n its simplest
mode, Le., nonmetric and continuous. It is a poetic-philosophical figure of the earth
and sea which neither reduces to counting points nor inflates to complexity and
chaos, but articulates richness. As I see it, and this is my third speculative step, measure
theory's monsters and "pathologies" hint at an infinitely richer mathematical ontol­
ogy ever more prolific than the present imaginary. The monstrous, in fact, occupies a
region between the impossible and the potential real more fertile than what White­
head explicitly articulated but that he, with Heraclitus, may have imagined.

least Action Principles

Given an unbifurcated, plenist process ontology that I introduced in chapter 2,

whether Whiteheadian, Spinozan, or Heraclitean, it is natural to ask: what is the gen­
erator, what impels ontogenesis, and how does it work? Let's return to the dynamical
crux of Whitehead's process philosophy, which is the appeal to the prinCiple of least
action. First, in terms of mathematical analysis (the calculus of real-valued35 functions)
the principle is phrased as discovering not minima but extremal pOints, which may
be maxima, minima, or critical points that are neither maxima nor minima.36 Second,
I should say that there is no unique and universal principle, but more a motley family
of principles of least action, where the action is defined according to a situation and
Ontogenesis 1 41

interest. So "the" principle of least action is more a seemingly ubiquitous set of

approaches to accounting for and anticipating ontogenesis.

leibniz
Before we revisit in more detail how the principle of least action works, let's look at
an antecedent philosophical formulation by Leibniz in his Monadology ( 1 7 14):
53. Now a s there are a n infinity o f possible universes in the ideas o f God, and but one o f them
can exist, there must be a sufficient reason for the choice of God which determines him to select
one rather than another.
54. And this reason is to be found only in the fitness or in the degree of perfection which these
worlds possess, each possible thing having the right to claim existence in proportion to the
perfection which it involves.3?

Leibniz's maximal perfection principle is in fact but a version (in mirror form) of the
least action principle, but he anchors his arguments in transcendentals: God and the
principle of sufficient reason, which is a secular shadow of God. In the present century,
in an epoch in which we are trying to live unchained from transcendental anchors,
what immanent recourse do we have?38 In place of Leibniz's God, and after Spinoza,
Whitehead's god is neither an originary Father nor a reverse projection of Man into
the transcendent, but the consequent limit of all concrescent processes. Therefore
Whitehead's god in fact reverses the ontological arrow: an eternal object is the result­
ing limit rather than the origin of a process. As we know from topology, the limit of
a sequence of subsets of Q may or may not be in Q: consider, for example, a sequence
of points in the open unit disc doing a Zeno march to its boundary. (If the set contains
all of its limit points-meaning if every single sequence's limit point is also in the
set-then we call that set "complete.") Note that the mathematics is entirely agnostiC
as to whether the limit is transcendental (outside Q) or immanent (in Q) . If the set is
complete, then by definition all sequences, all limit processes and their limits are
immanent: they exist inside Q. Moreover, in plena we can replace Leibniz's maximal
perfection principle by radically localizing it in texture. The substrate in which we
vary is no longer a discrete set which can, in principle, be evaluated by isolated, atomic
acts of inspection. Instead, we work in a potentially dense, open (unbounded) "con­
tinuous" set, in which case we need an adequate articulation of the substrate set.
(Certainly this cannot be a discrete set.) We will return to this later in this chapter
with Serres's multitude. This variational setup in a smooth setting raises the problem
of global optimization of a nonlinear function: a mountain peak may be strictly higher
than any point nearby, but it may still not be the globally highest peak. In mathemati­
cal terms, a local maximum may not be a global maximum. This motivates the third
reflection of our present condition to which I will adhere: material process, warranting
a trellising from, but not restriction to, physics.
1 42 C h a pter 5

Two paragraphs further on, Leibniz also anticipates the modern (or postmodern)
concern with nonlocality, writing:
Now this interconnection, relationship, or this adaptation of all things to each particular one,
and of each one to all the rest, brings it about that every simple substance has relations which
express all the others and that it is consequently a perpetual living mirror of the universe.39

This nonlocality anticipates David Bohm's implicate order and the nonlocality of
quantum mechanics. Broadly speaking, central insights and existential questions
introduced by quantum mechanics have not been absorbed and traced as much in
the epoch of the digital, partly because public and intellectual attention have drifted
to other concerns, and partly because public accounts of contemporary physics have
been too technical, or too atmospheric, or too laden with metaphysics that is inade­
quate to or inconsistent with the physics.40

On Local and G lobal

To give a more refined articulation of the local or global, let's see how this works in
classical-Le., not quantum-mechanical-differential geometry. Then we will weaken41
it considerably by retelling the story in terms of the much weaker category of measure
theory or, a bit more concretely, Lebesgue integration on manifolds. We can evaluate
the value of fIP] at a point p in M; this is called the point value of f. Or we can consider
a global value such as the integral of f over the entire surface M. Or we can consider
an intermediate quantity like
f.JdA
which is the regional or local aggregate of the K accumulated, integrated, over the region
Qc M. The Gauss curvature K of a surface M is a real-valued function; for every point
p E M, the Gauss curvature
K : M --7 1R

gives a number which can be defined as the product of the geodesic curvatures of two
locally orthogonal curves (parametrized by arc length) . K[p] can be negative, positive,
or zero, and gives some idea of how the surface is flexing in a neighborhood of the
paint p. The key point here is that K [p] is entirely defined by the local differentiable
structure on the surface.
For an example of a global quantity, take a surface M as a whole, and count the
holes in it. The number of holes, g, independent of their place, shape, and size, is a global
invariant of the surface M. Now lay out a net of triangles in the surface and take the
alternating sum of the net's vertices minus edges plus faces. This integer is called the
Euler characteristic. By a pretty theorem which we will not prove here,
X [ M ] = 2 - 2g .
Ontogenesis 1 43

In words, the Euler characteristic is identically a simple function of the genus.

The proof of this theorem demonstrates that the Euler characteristic, which is cal­
culated by an intricate and quintessentially local procedure, is in fact a topological
invariant, something that does not change as the surface is deformed in continuous
ways, without puncture or cut.
This accumulation is simply algebraic, by adding signed integers, but one can also
derive global measure by accumulating local measure using analytic (calculus) methods.
But the method of accumulation relies on an arduous construction using the calculus,
generalized to manifolds, and for robustness using measure theory and Lebesgue
integration.
The Gauss curvature Kfp] is built out of purely local pointwise data as a function of
point p E M, and therefore K cannot " see" the forest for the trees. It intrinsically
depends on the local metric: K measures how a curve diverges from an initially parallel
curve, with respect to its arc length. However, c . . F. Gauss proved an extraordinary
theorem now known as the Gauss-Bonnet theorem which states that the Gauss cur­
vature, when integrated over the entire surface M, expresses a global topological
invariant, and is not a function of the metric:
f K da = 2nx [M] .
M

This is already a striking fact. But local information, especially something that depends
on analytic (differentiable) information, can be sensitive to some global information
too. We have, for example, the striking result from analysis on Riemannian manifolds
that the spectrum of the Laplacian 11, which controls the analogs to heat and wave
dynamics on a general manifold, is related to global invariants such as volume and
some total integrals of curvature.42 Now we see that "local" has two senses: pointwise
and regional. The regional is that which is local to a topological domain which is not
the full (topological) space.
As an aside, this discussion offers an approach to an elaboration on Deleuze and
Guattari's coupling of the global and the local, by the intermediation of a regional
accumulation modeled on Lebesgue integration that does not require any c�mputable,
explicitly determined function. \
Given a way to aggregate or integrate, we can get a global quantity out of a local
or "pointwise" quantity, with one important requirement: we need to know that the
integral is invariant with respect to the class of automorphisms that is appropriate for
the phenomenon in which we're interested. More precisely, suppose h: M -t R is a
real-valued function on the manifold M. One sense in which h can be "invariant"
would be if h [m] = h [tf>[m]] for V m E M, V tf> E automorphisms [MJ . For example, suppose
we are interested in the total mass of all the people on the planet; then that integral
is fundamentally independent of where they are-so it is invariant with respect to
1 44 C h a pter 5

transformations that permute the people's locations on the Earth. The total mass is
also invariant with respect to many other transformations, for example a mapping of
language spoken. (We are assuming that a person's mass does not change much over
[infinitesimal] time as an effect of a change in language spoken or geographic loca­
tion.) This is not surprising. What is much more surprising, however, is that local situ­
ations can be affected by global invariants, as we have seen above. Ernst Mach's
Gedanken experiment with the spinning bucket of water is another example of this
entanglement of the local with the global:
Think, for example, of Newton's rotating bucket in which the water is not yet rotating. If the
mass m has the velocity Vj and it is to be brought to the velocity V2 , the force which is to be
spent on it is p = vz)/t, or the work which is to be expended is ps m(vj 2 + vz2) . All masses
m(vj + =

and all velocities, and consequently all forces, are relative. There is no decision about relative
and absolute which we can possibly meet, to which we are forced, or from which we can obtain
any intellectual or other advantage. When quite modern authors let themselves be led astray by
the Newtonian arguments which are derived from the bucket of water, to distinguish between
relative and absolute motion, they do not reflect that the system of the world is only given once
to us, and the Ptolemaic or Copernican view is our interpretation, but both are equally actual.
Try to fix Newton's bucket and rotate the heaven of fixed stars and then prove the absence of
centrifugal forces [original emphases] Y

So what lessons do we draw from these detailed considerations for other applica­
tions? Local telos (e.g., local derivatives) does not imply global telos. Global telos may
leave local telos radically undetermined. Nonetheless, sometimes a local, even point­
based phenomenon can manifest a global quality. This need not be so mysterious at all
if we regard the world as a processual magma.
My favorite figure is of a tree (figure S .4) : sectioning the trunk, you may see spots of
material on opposite sides of the disk that are the same yet are separated by the entire

Figure 5.4
Plant stem. Ferdinand Saussure, Course in General Linguistics, 87.
Ontogenesis 1 45

diameter of the "universe" which is the trunk. How can such distant parts of the universe
be the same material if there is no way for a signal to reach "instantaneously" across
the simultaneity of this section? But remembering that this tree has a history, that the
two spots of material belong to a thin circular band of similar material which in fact
was born on the surface of the trunk years ago in the simultaneous presence of the tree's
ambient, we see that there is no need to postulate a mysterious simultaneity or action
across arbitrarily large spatial separations when we acknowledge ontogenetic process.

Calculus of Variations
Let me articulate how we can understand a physical system using terms of the calculus
of variations.
The core idea is beautiful and profound: we extend the method of finding extrema
(maxima, minima, or inflection points) of a simple curve, viewed as a graph of a
so-called "real-valued" function from R to R, "one level up. " We conceptually replace
a number (e.g., a point x E R) by a mapping (e.g., a mapping f: M -t R) . Newton's and
Leibniz's calculus gives us a way to characterize the critical points of a real-valued
fu nction f: R -t R. In our extension, we replace R by the space of mappings, say C[M,
Rl ; and f-a real-valued function of real numbers R-by a real-valued operator A on
functions C[M, Rl . To give some idea of how radical a move this makes: R is a vector
space of dimension 1, but typically C [M, R] is an infinite-dimensional space. First I
review how we characterize critical pOints in the usual calculus of real-valued func­
tions. Then I will review the calculus of variations.
Look at a curve which is the graph of y ([x] , f: R -t R. If the curve is smooth, i.e.,
=

differentiable, then by definition the curve has a unique, well-defined tangent at every
point <x, f(x» on the curve. Then we can relate the tangents to the maxima and
minima on the curve-the peaks and valleys: the maxima and minima are where the
tangents to the curve are horizontal, i.e., have slope O. Therefore, for a differentiable
function f[xl , a critical point is a point c where fT c] = O. The calculus gives us a precise
way to test whether the critical point is a maximum or minimum: if f"[cl > 0 then the
pOint c is a minimum, if f"[cl < 0 then c is a maximum, and if f "[c ] = 0, then c is an
indeterminate critical point. We would have to further test the second derivative f "[ cl,
if it exists, in order to further determine how the curve turns near that inflection point
to see what sort of inflection it is (assuming that the function f is twice-differentiable
at c, which may or may not be the case) .44 Figure 5 . 5 shows a fifth-order polynomial
function of x which has derivative fTc] equal to 0 at three points. These are called the
critical points of f[x] .
To extend the logic is not at all trivial. As we've seen, the set over which we maxi­
mize or minimize is not necessarily just a one-dimensional line in R along which you
can imagine varying left or right in order to check to see if the function increases or
decreases around a putative critical point. In an infinite-dimensional space like
1 46 C h a pter 5

F i g u re 5.5
Graph of trxl = (x + 3)(x + 1)3(x - 2) . Diagram by author.

it's not even clear how one "varies" from a specific mapping qJo : M � lR to
C [M, R] ,
"nearby" mappings in the space of mappings C[M, R] . Working by analogy, we need
to make sense of the notion of taking a derivative of an operator A acting on C [M, R] ,
with respect not to real numbers x in R, but with respect to mappings qJ in C [M, R] .
By analogy with the calculus, we need to find a critical point Xo such that the
"derivative" dA / dqJ vanishes, i.e.,
dA
[qJo ] = O .
dqJ

Typically this expands into a set of differential equations which are called the Euler­
Lagrange equations for the problem. However, whereas in the calculus the dependent
variable x is an element of the one-dimensional set of real numbers R, here qJ is an
element in a space of maps, a set which is often not only infinite, but has infinite
dimension.
Let's go to the concrete situation of the dynamics of a mechanical system of masses
in motion. Although I will describe the calculus of variations in terms of a mechanical
system of particles in motion, this enormously powerful approach can be, and has
been, used to articulate many phenomena from light in gravity to economic dynamics
to quantum mechanics, string theory, and the evolution of space-time. Consider a set
of particles (say bits of cigarette smoke) in a room. Each particle Pi (where i is its index)
comes with its mass mi (a scalar), position Xi, and velocity Vi. If the particles are in R3 ,
then the position and velocity are vectors in R3 , so this particle's kinematic data are
given by a point in R7 . Let's take the relatively simple Newtonian case where the mass
is constant, i.e., mass does not change as a function of time, pOSition, or velocity.
Then the dynamics of the entire ensemble of N particles is defined by 6N parameters:
Ontogenesis 1 47

at any moment of time, a possible dynamical configuration (the potentially variable

positions and velocities) is a set of N positions, with N associated velocities, which
can be represented by a vector of dimension 6N. In other words, a point in R6N. This
6N-dimensional space of possible positions and velocities is called the configuration
space of the system. If we were to consider only the oxygen in a 60 kg human body,
there would be about 2348635298 1 000000000000000000 atoms of oxygen. Just to
model the possible of set of configurations of where the atoms are and how they are
moving in a 3D volume of space would require a configuration space whose dimen­
sion is 6 times the number of oxygen atoms, in other words a vector space of dimen­
sion 1 409 1 8 1 1 7886000000000000000000, V == R I409 1811 7 886000000000000000000 . Now a single
point in this space R I409 1811 7 8860000000 represents the kinetic state of the entire
ensemble of 2348635298 1 000000000000000000 oxygen atoms, and as the body
evolves in time, the state of the entire ensemble of oxygen atoms traces a path 'Y of
configurations in V, where each point on that' path is a vector of dimension
1 409 1 8 1 1 7886000000000000000000. There is some energy accumulated/expended
C
along with the change of the state from an initial configuration I E V to a final one
Cz E V. We can represent that energy by an integral called the action:

fcz y [s] ds.

A [y] =
cl
The principle of least action states that among all possible paths from the initial to
the final configuration, the way the system actually evolves is along a path y that is
a minimum (an extremum) of the action integral. Now, to use the mode of articula­
tion from chapter 6, the space of paths in V is actually a space C[I, VJ of maps from
say a unit interval I = [a , b] into the space V. This is a different and usually far larger
space than V. Even in the simplest case of the maps being continuous, the space C[I,
VJ has not only infinitely many elements, but also infinite dimension. So we are mini­
mizing a function-the action integral A-over a set of elements-paths 'Y in V that
CI
start from configuration and end in configuration c -in the infinite-dimensional
2
space C[I, VJ . SO, as in the general calculus of variations, we look for solutions ')6 to
the Euler-Lagrange equations: \

This same argument can be (and has been) applied to derive, for example, the evolu­
tion of the entire space-time of the universe.45
This principle of least action is actually a metaprinciple, a family of principles, since
we need to define an appropriate configuration space and an integral action for every
phenomenon (whether it's a set of oxygen atoms from a human body or the geome­
trodynamic field of relativistic cosmology) . So its scope is enormous, and indeed
unbounded. But is this all there is to ontogenesis?
1 48 C h a pter 5

Given Artaud, given Bataille's appeal to superfluity and excess, and the chaos­
mosis of politics and the historical lifeworld, perhaps we should not so quickly be
contented.

Other Principles of Ontogenesis

Christopher Alexander
In this epoch of engineering, not philosophy, it's inspiring to find at least one living
architect practicing and theorizing practice, mindful of these processual approaches
to the world: Christopher Alexander.46
Briefly, Alexander finds 15 basic patterns in the built and natural environment:
strong centers, local symmetries, boundaries, alternating repetition, positive space,
good shape, contrast, gradients, not-separateness, deep interlock and ambiguity,
roughness, echoes, the void, simplicity, and inner calmY It is true that the adj ectives
Alexander attached to his patterns-"good, " "positive, " and his pattern-as-state "inner
calm"-confuse categories such as matter and value, subject and object. But as a result,
analytical distinctions of matter and value, fact and value, or subject and object yield
to alchemical processes that transmute value into matter and matter into value, and
transform objects into subjects and subjects into objects. In fact Whitehead's " adjec­
,,
tival particles 48 are a characteristically bare-handed attempt to unbifurcate ontology
as well. There is nothing sacred about Alexander's 1 5 patterns except that they are
legible to us Western readers, inheritors of particular cultural-material dynamics. It
could be another basis, it could be another dimension; more or less does not matter
so long as the basis is legible to a community of designers and makers.49
In the logic of his exposition, Alexander first presents patterns as zeroth-order
distinctions in the material world, but reveals in a subsequent chapter that these
should be construed as processes, in other words, not as nouns and adjectives but as
verbs.50 To take an example, consider border not as a noun but as a verb. Rather than
count or classify borders, a maker looks for opportunities to create borders that yield
more life. This is a profound conceptual move, one that complicates and potentially
generates enough richness to make this a plausibly useful approach to art and techne.
Such practical material-aesthetic tactics making built environments more "living"
yield tactics that nuance a general theory (e.g., Rene Thorn's) of ontogenetic action.
Alexander proposes his own "principle of unfolding wholeness . . . the next step
after a given configuration will be the one which does the most to preserve and extend
the structure (structure being defined by the wholeness) . " He claims that under living
process, wholeness in an otherwise undisturbed system tends to increase and, most
significantly, that these are continuous processes.51 With Foucault, I recognize the value
of singularity. 52 But, pace Foucault, singularities can indeed emerge from smooth processes
(see figure 5 . 6) .
Ontogenesis 1 49

F i g u re 5.6
Water drop pinching off. (Source: http://commons.wikimedia.org/wiki/File:2006-01-2 LDetaching
_drop.jpg.)

So I recognize the sociopolitical significance of the autonomy of the actual occa­

sion, but this does not require atomicity. In a plenum world such as what I see with
Whitehead, every entity is contiguous in a Leibnizian way with the entire world, and
therefore with every other entity! How can such autonomy emerge? Consider boiling
water or any other such phase changes in boiling an explosive liquid, or changes of
topology under flow by curvature, or the reverse heat operator. Whole "universes !! can
blow up out of infinitesimal fluctuations. 5 3
Alexander perspicaciously proposed three alternatives to the quasi-physical prin­
ciple of least action:
( 1 ) emergence,
(2) evolution adapting to niche, and
(3) his own: unfolding living process.

Emergence
EmergenceS 4 is the phenomenon in which extra pattern emerges in the ensemble,
constituting a qualitatively distinct order from that found at the level of individual.
Emergence conventionally associates with networks of interacting entities, canonically
modeled by a set of nodes in a graph or "network. !! In some network models, each
1 50 C h a pter 5

F i g u re 5 . 7
A sequence of disks with decreasing area, but increasing complexity. In the limit, the tentacles
can be dense in all of JR3. Frank Morgan, Geometric Measure Theory, 4-5 .

node acts upon some quantity passed to that node along some connection (dia­
grammed as an arc) from another node, then sends the result out along another con­
necting arc to a neighboring node. The actions of a single node can be described by
some rules. So-called complexity theorists delight in the appearance of behaviors
characterizing the larger network that they do not design into the behavior of the
node and arc in their computational models.
However elaborate the nonlinear dynamics studied by Ilya Prigogine, Benoit Man­
delbrot, Mitchell Feigenbaum, or Stuart Kauffman, I agree with Alexander that the
phenomenology (physics) generated by computational models of complex systems is
too thin: it cannot account for the features of lived experience such as value, care (the
primordial to politics), or affect (the primordial to emotions) .

Evolution
What about biological evolution 55-what Deleuze distinguishes as differenciation (with
a c) in Repetition and Difference-the creation of patterns characteristic of biological
Ontogenesis 151

individuation at the levels of cell, organism, o r species? One type o f biological onto­
genesis is ontogenetic individuation, in which a prototypical process is that in which
initially undifferentiated neonatal stem cells reshape, differentiate, and specialize as
they migrate to new locations in the body and constitute the maturing organism.
Another is the phylogenetic process of a species fitting to its niche over generations
of selection for individuals more adapted to the current ambient environment. A more
adequate formulation would ( 1 ) treat all the species in a whole ecology, and (2) sym­
metrize the adaptation so that the species and the habitat evolve in tandem. Address­
ing both issues, Rene Thorn uses his topological theory of ontogenesis and structural
(in)stability to articulate biological individuation, introducing two concepts: salient
and pregnant forms. As Thorn puts it:
The primary experience in any receiving of phenomena is discontinuity. But discontinuity pre­
supposes the continuous. As our first experience of the c<.m tinuous is that of consciousness, i.e.
that of time, the most original auditive discontinuity will be, for example, the eruption of sound
in the midst of silence . . . . I shall call salient form any experienced form clearly separate from
the continuous background against which it stands out.56

Of course it is no accident that the first example Thorn chooses is a time-based

medium. In the class of salient forms, he distinguishes a subset of forms that evoke
persistent change in an organism's behavioral or physiological state. Borrowing from
Gestalt theory, he calls these pregnant. For now, I trace Thorn's theory no further,
except to say that his reinterpretation of Aristotelian physics in topological terms
spanning biology and semiotics is echoed in later work, notably by Jean Petitot, Gilles
Deleuze, and Felix Guattari.

U nfolding Wholeness
Alexander's principle of unfolding wholeness states that under life-generating or life­
sustaining processes, a natural system tends to unfold in a direction that does not
decrease centers and symmetries. Looking at his spectrum of (primarily visual) exam­
ples drawn from living organisms, the vernacular built environment, and art, it
becomes clear that although he uses mathematical terms of symmetry ap d center,
these are by no means Euclidean or even formal. Alexander writes:
[W] holeness, as a structure of symmetries and centers, evolves in time, and will always have a
natural dynamic of such a nature that as many as possible of these symmetries (and especially
some of the larger ones) are preserved as the system moves forward in time. As the system evolves,
it destroys as little as possible. 57

This phrase lias little as possible, " although it refers to symmetries, calls for a nonen­
ergetic (geometrical versus scalar-numerical) principle of least action. Here it seems as
if Alexander is describing ontogenetiC process as a steady state, or at least a kind of
antientropic process, and therefore, by Schrodinger's characterization, as a living
process.
1 52 C h a pter 5

F i g u re 5.8
"Yin-yang" crenulation. Drawing by the author.

For example, we can interpret Alexander's BORDER pattern as a boundary, a distinc­

tion in the plenum of the world. First, for pedagogical purposes, take a classical view:
zoom into a disk and look at the boundary, locally, as a diameter of the disk. Under
the impact of enlivening processes, one can imagine that this diameter will bulge in
one part and cave in the other part, forming an S. Imagine this not as a specific curve
but as a generic one that emerges in the course of ontogenetic process. If we continue
this process, we can imagine the curve to furrow more and more, to finger and coun­
terfinger until it becomes practically impossible to tell which parts of the disk lie on
one side and which on the other of the boundary. We should interpret this not liter­
ally as a picture in the plane. The implications depend on the nature of the space and
topology in which one performs this BORDER articulation. (As a simple example, such
a continuous transformation cannot yield nontrivial simply connected domains in
the plane R2, but in R3 it can yield topological monsters like ]. W. Alexander's horned
sphere [figure 5 .9] , a continuous unbroken surface homeomorphic to the sphere S2,
but whose complement in R3 has a component which is not simply connected.)
Of course if we were to restrict ourselves to geometric spaces, it would be solely of
mathematical interest, but the point of this exercise is to understand the process and
then imagine it as abstract continuous process, where abstract is in the sense of Deleuze
and Guattari but applied to processes rather than machines. Now, Deleuze's Le pli (The
Fold) is a classical theory, in that it uses the calculus's assumption of differentiability,
and consequently the existence of a tangent at every point of the curve. Even the
notion of inflection point in his interpretation relies on this differentiability. The value
in this discussion is not in the geometry or the calculus per se-they are elementary
and useful mainly as pedagogical aids-but in what is evoked beyond the denotative
limits of the calculus of the real line-material ontogenesis in fields of matter, energy,
media, and affect.
Law, for example, is full of examples of crenulation under the impact of need,
precedent, and contest. On a public bus in California, I saw a sign warning that the
bus emitted substances that were "known to the State of California" to be carcino­
genic, and so forth. I invented an imaginary history behind this sign: buses full of
Ontogenesis 1 53

F i g u re 5.9
J. W. Alexander's horned sphere, a compact surface homeomorphic to S2, but whose exterior is
not simply connected. Image by the author.

" naive" passengers, stinking exhaust, a complainer, a lawsuit, a materially negligible

posting of signs, and so on. In each round of contest about liability between the liti­
gious public and the State of California, the bounding membrane between their
domains of accountability becomes ever more crenulated. Well-being m �
not be
served, but legal codes and public signage proliferate.
To take another example, from twentieth-century theater, ]erzy Grotowski experi­
mented with mixing spectators and actors in multiple configurations of action. We
see diagrams of audience gaze/attention intertwined with the configuration of a set
constructed in the course of the play. We can imagine how the spaces of attention for
performer and spectator interpenetrate when we dissolve the proscenium and allow
actors and spectators to mingle in the same event space. We will return to this in
chapter 7.
1 54 C h a pter 5

In light of the twentieth century's radical theoretical development, we can (and

must) depart from Deleuze's classical interpretation of Leibniz and the baroque. We
can motivate this by observing the urgent need for a way to handle complexity, as I
noted in the first two chapters. But more fundamentally this may give us a way to
measure event and process without number or metric. One way is via quantum
mechanics's puncture of the subject/object divide, familiar to students of critical phi­
losophy. Another is via measure theory's dispensing with geometry, replacing differ­
entiable manifolds (manifolds that have a well-defined, unique tangent at every point)
by integrations against "test" distributions, in other words by operators on the space
of currents.
I take as an axiom that life is a baroque process, and conversely, under baroque
processes, that life and living structure inevitably appear in the world. Alexander's
ontogenetic principle is negentropic, and therefore, by Schr6dinger, a living process.
Therefore it is baroque. And we have earlier established that it is alchemical.
Now, the baroque is familiar ground, but under this ground lies Whitehead's
"buzzing world, amid a democracy of fellow creatures" and James's famous "blooming,
buzzing confusion. " My thoughts are musical ones, meant to add musical voice to
other writers . But thinking hyletically we can reorient our attention to texture, or, to
take an example from sound arts, to noise.
NOISE *- RANDOM
MULTITUDE *- MULTIPLE

Serrean N oise
In Genese ( 1 992), Michel Serres proposed a "new object for philosophy" which he
called the multitude. Serres's multitude is not the union of atoms (or elements, or
objects) . It is not the SOCiety of individuals, not the multitude cited by Antonio Negri
and Michael Hardt or their capitalist adversaries. Serres starts with a fundamental
disquieting provocation:
The arithmetic of whole numbers remains a secret foundation of our understanding: we're all
Pythagoreans. We think only in monadologies. Nevertheless, we are as little sure of the one as
of the multiple.58

For Serres, even a theory founded on relations is inadequate: "There is still the problem
of finding out how relation is transformed into being, and being into relation. "59 So
much for actor network theory, or any theory modeled on the trope of the graph! One
efficient response would be to dispense with relation altogether, where given sets X
and Y, a relation R on X is understood as a binary mapping R:X x X -? Y which, for
every pair Xv Xz E X, associates a R(xv Xz) E Y. If we give up the expectation of making
a binary association on points, which formally reduces to an arc on a graph labeled
by R(xv xz), then we can gain a tremendous expressive freedom. If we want to step
Ontogenesis 1 55

away from this "thin" trope,60 to what else can we appeal? We could complexity the
model by generalizing the binary relation to an n-ary relation R(x l l X 2, . . . , xn),
R:X x X x . . . X � Y. Or we could consider fields instead. If we take continuous,
superposable fields defined as mappings on the space X, there is no complex combi­
natorial problem of relation to be solved. What we can do then is explore how the
variation and superposition of fields articulate ontogenesis. Serres encourages us:
The multiple as such. Here's a set undefined by elements or boundaries. Locally, it is not indi­
viduated; globally, it is not summed up. So it's neither a flock, nor a school, nor a heap, nor a
swarm, nor a herd, nor a pack. It is not an aggregate; it is not discrete. It's a bit viscous, perhaps.
A lake under the mist, the sea, a white plain, background noise, the murmur of a crowd, time.61

For now we cannot go deeper with this, because our discussion is not about general
set theory, except to say that the multiple that Serres works with is a more generous
notion-I will not say concept-than Badiou's. In fact, if we start with such a multi­
tude as we do with a plenum in its density, we start with the continuum and therefore
have no need to try to piece it together out of bare atoms, or bare sets for that
matter. 62
The multitude is shaped, and in Serres's playful account, even shapely. As we have
seen, topology gives boundless ways to articulate that shape, to shape the material.
Gilbert Simondon provides an evocative analysis of how a brick mold "informs" the
clay material by conditioning the movements of the clay (its stuff, or its clay particles,
depending on your preferred mode of interpreting matter) . But I will say "shape"
instead of "inform, " to avoid reverberations with abstractionist information theory.
But since the present chapter concerns the material fluctuations and transforma­
tions of the multitude, let me turn to one of the central themes of Genese: noise. Serres
introduces it beautifu lly, so I cite the passage at length:
This word noise crosses the seas . . . . In Old French it used to mean: noise, uproar and wrangling:
English borrowed the sound from us; we keep only the fury. In French we use it so seldom that
you could say, apparently, that our language had been cleansed of this "noise . " Could French
perhaps have become a prim and proper language of precise communication, a fair and measured
pair of scales for j urists and diplomats, exact, draftsmanlike, unshaky, slightly fr<>ten, a clear
arterial unobstructed by embolus, . . . [t]hrough becoming largely free from stormy weather,
sound and fury? It is true, we have forgotten noise. I am trying to remember it; mending for a
moment the tear between the two tongues, the deep sea one and the one from the frost-covered
lake. I mean to make a ruckus [cherche noise] in the midst of these dividing waters.

Sea Noise

There, precisely, is the origin. Noise and nausea, noise and the nautical, noise and navy belong
to the same family. . . . We never hear what we call background noise so well established there
for all eternity. In the strict horizontal of it all, stable, unstable cascades are endlessly trading.
Space is assailed, as a whole, by the murmur. . . . This restlessness is within hearing, just shy of
definite signals, just shy of silence. The silence of the sea is mere appearance. Background noise
1 56 C h a pter 5

may well be the ground of our being. It may be that our being is not at rest, it may be that it is
not in motion, it may be that our being is disturbed. The background noise never ceases; it is
limitless, continuous, unending, unchanging. It has itself no background, no contradictory. . . .
Noise cannot be a phenomenon; every phenomenon is separated from it, a silhouette on a
backdrop, like a beacon against the fog, as every message, every call, every signal must be sepa­
rated from the hubbub that occupies silence, in order to be, to be perceived, to be known, to be
exchanged. As soon as a phenomenon appears, it leaves the noise; as soon as a form looms up
or pokes through, it reveals itself by veiling noise. So noise is not a matter of phenomenology,
so it is a matter of being itself. It settles in subjects as well as in objects, in hearing as well as in
space, in the observers as well as the observed, it moves through the means and tools of observa­
tion whether material or logical, hardware or software, constructed channels or languages; it is
part of the in-itself, part of the for-itself; it cuts across the oldest and surest philosophical divi­
sions, yes, noise is metaphysical. It is the complement to physics, in the broadest sense. One
hears its subliminal huffing and soughing on the high seas.
Background noise is becoming one of the objects of metaphysics. It is at the boundaries of
physics, and physics is bathed in it, it lies under the cuttings of all phenomena, a proteus taking
on any shape, the matter and flesh of manifestations.
The noise-intermittence and turbulence-quarrel and racket-this sea noise is the originat­
ing rumor and murmuring, the original hate. We hear it on the high seas.63

Here Serres is playing with the implications of his ideas for the distinction in electrical
engineering between signal and noise, which Claude Shannon encoded in his seminal
paper defining information in terms of the entropy.64
For me, however, the "random" is not a concept but a label we use to cover our
inadequacy facing the world. Random is another name for our ignorance, our inade­
quate senses, and, in the computational setting, the sparseness of our reach. Noise, for
me, is not just the random in space, or time, or shape, but the hovering of patterned
material (matter, energy, symbol, affective field) at the limit of measurement, and
therefore observation. Walking down the corridors of the Atlanta airport, I was struck
by the speckles on the floor, providing texture that contrasted the blankness of the
even more mechanically produced walls and pillars. But all these random and homo­
geneous textures bear little resemblance to the inhomogeneities of marble-the
random does not vein matter as lava veins the stone through which it courses. 1 00
million years ago, at its birth, molten rock did not course in arbitrary directions, did
not make its way "randomly" at all, but followed local gaps, gradients, and
inhomogeneities.
Serres plays on the profound complemetarity between the electrical engineer's
signal/noise distinction and the criteria for significance that are implicitly or explicitly
employed. Noise is an artifact of the criteria. John Cage made the same point for music
as organized sound.
Could there be a noisy ontogenesis that would sustain nonintentional (as well as
intentional) adventurous, improvisatory gesture?
Ontogenesis 1 57

I believe so, and here I speculate from Alain Connes: Replace "coordinates" based
on values in real or complex numbers by acts of observation as articulated in quantum
mechanics, Le., by operators on the Hilbert space of possible configurations, state
space. Real (and complex) numbers commute: x*y y*x, but in general two operators
=

do not commute: AoB "* BoA because the order in which you compose two operators
matters: adding sugar to your tea and then sipping it is different from sipping the tea
before adding sugar to it. In the post-quantum-mechanical world, these operators of
observation A, B are in M(H) where M is the von Neumann algebra on the infinite­
dimensional Hilbert space H of all possible states of a quantum mechanical system.
Then space-time itself is replaced everywhere by operators, in fact by the nonspatial
set M(H) . What I sketch here are the beginnings of an approach from noncom mutative
geometry, which exceeds the scope of this book. 65

Applications

Given the motivations for this book, let me describe two areas of relevance inspired
by Felix Guattari, anti-psychiatry and art, then close with a comment on time.

Anti-psychiatry
Guattari's anti-psychiatry not only challenges the power of the clinical therapist's
analytic view of the patient, but respects (rather than second-guesses) an organism's
entanglement with the world. As Wittgenstein said to his student Drury when the
latter decided to leave philosophy for the practice of psychiatry: "Should I go mad,
,,
the one thing I would fear from you would be your common sense. 66 A schema, in
this context, is nothing more than the regularization and formal representation of
normatizing "common sense. " While deeply informed by the tradition of psycho­
analysis of Freud and Lacan, Guattari's decades of work with schizophrenics in his
clinic La Borde parted from psychoanalysis in a most radical way by refusing a priori
schemata in interpreting the patient.67 Guattari left behind psychoanalysis's aspiration
to sCientificness, to discovering the truth about the subject's world, and recognized
instead that all forms of expression are actually also simultaneously forms of content,
that every one of us cocreates the world and coadapts to the world. Guattari recognized
that the schizophrenic is as much a costructuring member of the clinic-as-an-event as
the doctors and nurses who ostensibly run the clinic. One of the most illuminating
examples in Guattari's Chaosmosis tells about families who came as a group to sessions
in which actors introduced extra characters in filmed events. The participants were
asked to revise, improvise, enact, and reenact their relations for each other and for
later viewers. In the course of this they used vocal and manual gestures or movements
whose meanings are not predefined or evident but arise organically from being exfo­
liating in the world, in a signifying process that Guattari (and Deleuze) called pathic
1 58 C h a pter 5

subjectivation, the formation of feeling selves. The subjects later reviewed these events
and narrated for themselves what they saw themselves doing. This is radically different
from the subjectification imposed according to schema by an analyst who announces
to his patient: "By the power invested in me from my training as an analyzed Analyst
,
and interpreter of the DSM-IV I declare, 'You are schizophrenic. " 68 It's one of Guat­
tari's clearest examples of ethico-aesthetic play in the magma of asignifying semiolo­
gies, and of improvisation over rehearsal and experience sedimented over the lifetime
and-acknowledging Lacan-beyond the lifetime of the ego. This is not theatrical
role-playing, nor everyday activity observed in the wild behind a screen, nor purified
laboratory interrogation. There are no blueprints or recipes for any of this kind of
playful, rigorous work, and in fact it would be a terrible betrayal to make a method
out of this.

Productive P rocess
At a macroscopic, social scale, each performative event can also be construed as a
compact region of the plenum intersected by multiple life courses (figure S . 1 0) . Over
the duration of the production of an exhibition or a performance, people from diverse
epistemic cultures are required to collectively create a single event. Although an event
has not the physical compactness typically assumed of what Susan Leigh Star called
a boundary object, a performance or an artistic installation-event nonetheless coordi­
nates and condenses understandings in the course of its launch, flight, and landing.
Typically, in the construction of an event that engages people who have not worked
together before, or epistemic cultures that have not traded with one another before,
to borrow from Peter Galison and Karin Knorr Cetina, one management tool is to fix
an a priori order: schemas of tasks and roles, milestones, and so forth. It may seem
more sophisticated to ask the participants to create a common working language (as
was explicitly done in the design phase of the TG2001 production hosted at Banff in
200 1 ) . But however mature the tangent epistemic cultures and their languages, no
pidgin is created prior to contact, and no creole prior to use. In practice, a common,
articulated descriptive frame emerges, at best, at the end of the project, so a common
language can only be born in the process of production.69

Time-Based Media
Time-based media, commonly construed as computationally synthesized or sequenced
video and sound, engage yet other kinds of process.
At the finest scale, we interpret computational video not as frames or even as pixels,
but as bundles of massively parallel processes. (So we are quite remote from the frame­
based "cinematic" model of temporal experience Bergson so rightly criticizes.) Each
pixel is not just a color value (R, G, B, alpha) but the vector-valued result of an arbi­
trarily long computational process. When we watch video, there are as many processes
O ntogenesis 1 59

What does a n event do?

,.
,.

Perco late ideas across discou rse com m u n ities

F i g u re 5 . 1 0
A project a s a n event transected b y members o f different epistemic cultures and discourse com­
munities. Diagram by the author.

as there are pixels on the screen, so we are looking, literally, at millions of concurrent
processes. Up until recently, most of those processes have been trivial or totally cor­
related, but the computational means of synthesizing video opens up the potential
for radically molecularizing the process of synthesizing light, hence calligraphic and
gestural video. A way to understand and articulate such dense media is to think in terms
of topological dynamic textures that induce temporal sense and dynamics ;

On Time and Event

Having said that, what if we take a more generative concept of event than naive space­
time? What if we take the extent of an occasion (to borrow Whitehead's term) as an
effect of the exfoliation of processes in the world? An example would be how Gins
and Arakawa define their architectural body's extent as bounded by where its attention
lands. So the architectural body, or, as they term it, the organism-that-persons/o is
marked out by the set of perceptual landing spots. I would retain the contingent
experimental aspect of " landing, " but replace " spot" by generalized, measure-theoretic
1 60 C h a pter 5

" surface. " "Landing, " in turn, I would interpret as an operator. (See the next chapter
for a mathematical sense.) Then we would be able to more adequately regard the
lighting of a cigarette as an event as well as a process: the lighting, the initial drag,
the exhalation, the drift of smoke through the air, the multiple perceptions of smoke
as it coarticulates its extent together with the people who have been prepared to
smell it.
So, instead of a logicist definition, what could be a material, eco-phenomenological
notion of event?-A compact set inside a topological manifold of dynamic material =

matter-energy-affect. Given such conceptions of event, we can articulate extensively

from Serres's atmospheric figures of kairos and chronos.71 This extends the discussion
of event in chapter 3, and also anticipates our turn to a topological approach to mate­
rial dynamics.
6 Topology, Manifolds, Dynamical Systems, Measure, and Bundles

M athematics as Poetic M aterial and Material Mode of Articulation

At a symposium focused on Deleuze and Whitehead, l I realized how to articulate the

proposition that one could use mathematics as poetry rather than as instrument or
measure, or a replacement for God, or a conceptual battering ram. (I must confess,
however, to some pleasure in Alain Badiou's polemical thesis mathematics ontology.)
=

I agree with Badiou that mathematics is substantial, and not merely a description of
substance. Shaping mathematics as poietic material in fact differs in kind from using
mathematics to describe the universe as physicists see it. Part of the charm of FoAM's
trg responsive environment is its attempt to make palpable a concept of the world
(recent quantum-field-theoretic cosmology) by forcibly identifying it with the percep­
tual field-a cosmic ambition. The artists could only begin to approximate this by
restricting trg to a very compact physical duration and place in Kibla, and by making
allegorical simulations in software. Allegory makes the world of difference between
depiction and enaction, perception and phenomenology. Allegory is allied with depic­
tion because it makes a picture and a necessary gap between the picture and what the
picture homologously represents; therefore it always implicates questions of knowl­
edge, which devolve to questions of sense data. But in that case, as we have seen in
chapter 2, we are dogged by all the epistemological problems of language as represen­
tation raised from Wittgenstein and Debord to the present day.
This book is part of an experiment to see mathematics not as an apparatus for
cognition, not as representations or models of some aspects or strata of the world, but
rather as modes of articulation, especially poetic material modes, that consequently are
adequate to life.2 We could work with sharply different sorts of poetic matter: continu­
ous topological dynamics, geometric measure theory, or even fancier stuff like non­
commutative algebra and etale cohomology. But I choose to start with the simplest
symbolic substances that respect the lifeworld's continuous dynamism, change, tem­
porality, infinite transformation, ontogenesis, superposability, continuity, density, and
1 62 C h a pter 6

value, and yet are free of or at least agnostic with respect to measure, metric, counting,
finitude, formal logic, linguistics (syntax, grammar), digitality, and computability, in
short all formal structures that would put a cage over all of the lifeworld. I call these
substances topological media. Simplicity here is not a requirement of the theory (no
Occam's razor here) but merely an acknowledgment that I do not understand enough
about the lifeworld to bring out fancier stuff yet, of which there is so much more up
the wizard sleeves.
The fundamental difference in this approach is to use mathematics as substance in
a workmanlike way, patching here and there to see what values ensue, always sensitive
to whether the poetic material accommodates transfinite, incommensurable, imma­
nent passion. Totalizing carapaces like Wolfram's computational equivalence princi­
ple, which at bottom is a transcendental atomic metaphysics founded on making
counting sacred, would hammer us into a very sparse ontology. And to a hammer
everything is a nail.
This chapter introduces several modes of articulation with which we can articulate
substance and infinity using notions of proximity, convergence, limit, change, and
novelty, without recourse to number or metric. For the moment, I will label these
fields of concepts very loosely as topology, differential geometry, and Lie groupS. 3 Inno­
cents should be alerted that I am using these terms with reference to their technical
meanings in mathematics, which do not correspond with everyday, literary- or cul­
tural-theoretic senses of function or analysis. For example, the term functional analysis
implies no adherence to what psychologists or philosophers mean by functionalism,
or a functionalist attitude to descriptions of human experience. These concepts should
honor the full density, richness, and felt meaning of living experience. Moreover,
analysis as drawn from the context of mathematics does not entail any elements of
psychoanalytic theory, or more generally any explicit psychosocial theory, at least in
any conventional sense. Mathematicians, be alerted that I am using these terms
slightly (but responsibly) askew from the areas that they traditionally have labeled in
math reviews, just for the sake of concision. I will elaborate them more accurately as
we proceed.
The motivation is that topology furnishes us with concepts well adapted to articu­
lating the world alternatively as plenum and as stuff (the stuff of chapter 3 ) . Continuous
topological dynamical systems are good for articulating ontogenetic process, without
restriction to models using explicit differential equations that need explicit numerical
constants. Often the vagaries of social, biological, material processes preclude the
gathering of data in sufficient precision and quantity to adequately parametrize such
explicit models. And we know how every choice of equations in a model deeply
encodes theoretical assumptions that inescapably color the phenomena we can detect
through the optic of those models. This book wagers that topological methods may
support some flexible and less schema-bound approaches to phenomena and processes
Topology, M a n ifolds, Dynamical System s, Measure, a n d B u n dles 1 63

05

68, 94

F i g u re 6 . 1
MathSciNet classification o f mathematical research weighted b y activity, with areas related to
computer science highlighted. Modified image courtesy of Dave Rusin.

of human interest, material messiness, and living density. To be honest, I �hould say
that I will introduce both more and less than what mathematicians call " i,?pology.1I
More, because I will also introduce concepts from fields of mathematics quite distant
from and more elaborate than point set or continuous topology, such as measure
theory, dynamical systems, and Riemannian geometry. Less, because we will of course
spare the schoolbook approach and take a high road, more akin to Gilles Chatelet's
treatment of mathematics via essential intuitions. Like Chatelet, I will try to respect
the intuitive essences of the concepts and their derivations, which in mathematics
take the form of logical (but not formally mechanized) proof.4 And finally, I wager
that the modes of articulation I introduce in this chapter for their pOietic potential
1 64 C h a pter 6

have implications for art, philosophy, and engineering beyond the scope of the par­
ticular applications in this book. So, rather than give a "just-in-time" set of unordered
lily pads, let me try to make a loosely connected narrative, from modes that require
less structural constructions to modes that require more. However, I will not pretend
to make systematic application of all the scaffolding concepts introduced in this
chapter. In fact, we should like to see what others make of this material.

Conti nuous Topology, Topological M a n ifolds

Klaus )anich's uniquely vivacious book on basic topologf inspires this chapter's tour
of the sorts of patterns and morphisms that scaffold the articulations I propose. In a
book of speculative philosophy and art, the challenge is always to describe the notions
in just the right degree of detail or concreteness. It's not only the what but the how
and why that we're concerned with. It takes some judgment to estimate at what level
of detail we need to halt, enough to offer the reader the conceptual grit and grip
needed to make his or her own concepts, but not so much as to obscure the essential
ideas. (Some editors have not recognized that with technical concepts such as concepts
of mathematical objects and related morphisms, one can err on the side of too much
explanation.) So much for the what. But in mathematics, the how and why really
require us to go through the actual proofs. Understanding a proof may require years
of concentration on a paragraph of mathematical writing. That said, I will present a
proof only in order to advance or thicken the argument, rather than demonstrate the
truth and force of a theorem.
Before we begin, I should emphasize that topology as mathematicians have devel­
oped it over the past hundred years comprises an enormous range of spaces, mappings,
properties, and concepts, immeasurably richer than the discrete, graph topology cited
by computer scientists and their clients. For example, B. C. Smith uses "topological"
in a typically loose way: "By 'topological' I mean that the overall temporal order of
events is dictated, but that their absolute or metric time structure (e.g., exactly how
fast the program runs) is not."6 Graphs are a particular and relatively uninteresting
class of topological spaces, but the vast majority of topological spaces are not graphs.
For the purposes of this book, when I say "topological" I will mean the general proper­
ties of the class of topological manifolds and not the special properties of discrete
graphs. In fact, one of my strongest technical reasons for introducing the topological
is to provide an alternative to all the figures derived from discrete sets and graphs.
It may be helpful to keep in mind some working examples in which you, the reader,
can check your developing intuitions about the topological concepts that I am
about to describe. For each example, the fundamental question concerns proximity:
what do you consider to be a neighborhood, without necessarily appealing to any quantita­
tive means.
Topology, M a n ifolds, Dynamical System s, Measure, and B u ndles 1 65

The Earth
One example comes from atmospheric and geophysical boundaries: where does the
planet Earth end and space begin as one ascends into the atmosphere? One could
apply all sorts of criteria. The point at which one loses consciousness in a rising high­
altitude balloon? The barometric pressure? The flux of ultraviolet light or cosmic rays
intersecting a meter held in the hand? The visibility of the people waving their hands
goodbye? Take the barometric pressure for example. A macroscopic body intersecting
the atmosphere at extremely high speed (tens of thousands of miles per hour) and at
a shallow enough angle may even glance off the atmosphere the way a rock can skip
off the surface of a lake, but the same body brought slowly through the atmosphere
will easily penetrate it. So the manner in which one approaches the planet certainly
affects the boundedness of the planet.
Of course accepted boundaries are conventional, but the conventionality under­
lines the material fact that there is no sharp atmospheric boundary around the planet
Earth.

Flows
Another example is the set of flows in a stream of water. Consider its eddies, the leaves
or specks of dust carried along by its flows, its vortices, or its sources, the points from

F i g u re 6.2
Leaf in stream flowing from right to left, overlaid with motion vectors and the vertical and hori­
zontal components of motion . Dotted lines mark flow lines. Diagram by the author.
1 66 C h a pter 6

which a particular stream is born far upstream, the ultimate points to which a speck
of dust may collect far downstream as time grows without bound.
There are at least three ways to represent the flow of liquid in a body of fluid-call
it M-prior to modeling the physics. One way is to explicitly parameterize the orbits
as functions «1>[t] given by the flow «1>: t�<x [t] , y[t] > as a map from a segment of a line
into the plane. In other words, a planar curve is traced by having coordinates <x [t] ,
y[t] > corresponding to the parameter t, for t i n some range [a, b] . The problem i s that
there can be an uncountable infinity of trajectories of flows in a body of fluid, each
one represented by a different mapping of the form «1>[t].
Moreover, the body of fluid M is a single continuous mass of matter in continuous
movement, and therefore a traj ectory is not merely a physical object but an artifact
of a Gedanken experiment. The Gedanken experiment goes something like this: drop a
speck of dust into the fluid and let it flow according to the moving fluid. The path
that it traces is a particular curve in the moving fluid. Dropping the speck at a differ­
ent moment of time and at a different initial location can yield an entirely different
trajectory. And the set of all possible trajectories is infinite, un count ably infinite, even
an infinite-dimensional continuum. Clearly we could benefit from a set of concepts
with which we can articulate this more precisely.
For most phenomena in nature and societies we have no explicit function, certainly
not in neat closed form, that is a finite algebraic expression composed of elementary
polynomials and trigonometric functions. Moreover the devil is in the constants of a
problem. Not only is the behavior of a system of differential equations modeling a
dynamical phenomenon potentially exquisitely and even catastrophically sensitive to
small differences in the coefficients that tune its specific values; often we do not have
adequate instrumentation, or our available measures are far cruder or less accurate
than what a model requires in order to make plausible quantitative descriptions.
Explicitness is an obstacle to understanding, not a virtue, when we are dealing with
a thick, material experience because of the measurement problem: any situation that
might be materially and humanly interesting likely will be too rich for adequate
description by a finite system of measurement. What we have learned from the
detailed analysis of technoscientific practice is that the very attempt to bracket a
phenomenon ever more precisely introduces artifacts, creates entities, and complexi­
fies practice as well as ontology. Technoscience is as sociopolitical as it is technicaU
(We will return to this in the section on ontogenesis.) One motivation for Rene Thom
and Jean Petitot's topological approach to phenomena as diverse as biological ontogen­
esis and the evolution of linguistic structure is to relinquish unjustifiably, misleadingly
detailed and brittle quantitative "measures" of a phenomenon and gain expressive,
even rigorous traction in its articulation.
A second way is to consider not the trajectories of flows themselves but how the
flows may be generated from a much more compact set of so-called differential equa-
Topology, M a n ifol d s, Dynamical System s, Measure, and B u n d l e s 1 67

tions, whose solutions are whole traj ectories <ll-m appings from a one-dimensional
interval parameter into a given manifold. In other words, the set of differential equa­
tions yields specific mappings as their solutions. We shift attention from actual tra­
jectories to the conditions of possibility for a family of trajectories. In fact this mode of
thinking is a germ of the intuition behind the paired concepts actual/potential.
Systems of ordinary differential equations are the heart of the theory of dynamical
systems, which in turn provide the germ of all theoretical discussions of complexity
theory, systems theory, and cybernetics. The relation between a (system of) differential
equation(s) and a particular mapping that satisfies the differential equation(s) is the
mathematical analog to the relation between potential and actual, or to a relation
between pregnance and salience, to borrow from Rene Thom's semiophysics.
Now, even this description, however flexibly it unchains us from an unwarrantedly
explicit description of material experience, is still too explicit, and subj ect to White­
head's fallacy of misplaced concreteness. In the absence of any concrete data about
the "physics of materials," Le., the constants of the model, analogous to constants of
thermal or electrical conductivity or the gravitational constant G or the speed of light
in electromagnetism, what can we say with rigor that on the one hand does not make
unreasonably "concrete" demands on deSCription, yet on the other hand honors the
phenomena in question? If we dispense with explicit equations also at this potential
level of ordinary differential equations, we can still make provably certain statements
about the behavior of the possible solutions to a given system . Some qualitative but
rigorously treatable features include periodicity, or the existence and uniqueness or
structure of periodic trajectories (aka orbits) .8
We can articulate rich physical phenomena using these notions: the wash of ripples
along the banks of a river, the accumulation of leaves in the eddies trapped in the
crook of a tree trunk fallen into the water, or more symbolic entities like the destina­
tions of lanterns set out to float on the current, or the origins of a river and all its
tributaries. The destination(s) and origins of a trajectory regarded as limits as trajectory
time goes to infinity or negative infinity can be regarded as symbolic limit events.

Demographics \
'
Suppose we are deciding whether a person merits membership in a group G based on
some measure of desirability. Suppose we have an axiom: If a person Q is deSirable,
then Q is a member of G. Then the following monotonicity theorem seems reasonable:
If a person P is more desirable than person Q, and Q is a member ofgroup G, then P should
also be a member of group G.

Where's the Smoke?

Stand a group of people in a room; ask someone to light and smoke a cigarette. Ask
each person to raise a hand upon smelling the smoke. Here's a reasonable way to
1 68 C h a pter 6

empirically define where the smoke is. But notice several features of this example. The
extent of the smoke changes with time. The extent is determined by subjective
means-different people have different sensibilities and each person may be more or
less sensitive to smoke according to how much she or he thinks about the smoke. In
fact, just asking people to smell for smoke primes their sensitivities.

Songs
Consider the set of all songs, alternatively defined as (1) performed live, with contin­
gent warble, glide, and rubato; (2) transcribed to a formal system of notes in a normal­
ized and regularized set of pitches and durations; (3) paralleled and labeled by words:
titles and lyrics; (4) as variations in air pressure-time series of acoustic amplitudes
over time. Each of these characterizations enable quite different ways of considering
which songs are similar to which.
As a fifth alternative, we can regard songs as a set of social practices whose cultural
and microlocal meaning and value inherit from local as well as nonlocal histories. A
performance of one song also conditions other performances. As we recall from the
brief excursion into the history of Arab musical performance at the cusp of Western
notational, recording, distributional economies, the formal notation of a particular
performance freezes it as a canonical representative of a Wittgensteinian family of
songs whose border is in fact constantly renegotiated by social practices. A key point
here is that those social practices, even though they may be conditioned, boundlessly
and endlessly unfold in ways I will suggest are noncomputable in essence. This antici­
pates a material, ontogenetic application to sociocultural dynamics later in this
chapter.

Events and Responsive Environments

The course of an event in a responsive environment can be regarded as scripted or
aleatory, depending on the composer's attitude toward the causal relationships between
the processes active in the environment. But as we expand the attention fixed on
particular " agents" and their proxies or prosthetics to the whole environment's mate­
rial dynamics, we are motivated to consider the conditions of possibility of the articu­
lation of distributed dynamical material.
Buoyed by these examples, let us tour some of the relevant mathematics trellising
this project on poiesis.

Point Set Topology

The basic axioms of set theory include the notion of inclusion (membership), subset,
intersection, and union. Already enormously powerful at this level of description is
the absence of any comment on the nature of a set S, whether material, abstract, finite,
Topology, M a n ifolds, Dynamical System s, Measure, and B u n d les 1 69

or infinite: there is no restriction at all on how the set may be defined. In a most
fundamental difference with computer engineering, a set does not have to be defined
by explicit enumeration. Much of the imaginary of the computer scientist is delimited
by the notion of a finite, denumerable set {X l ' Xz, X3, , xn} where n is some explicit,
• • •

finite integer. But a set can be defined by a rule, such as lithe set of all real numbers, "
or lithe set of all moments of introspection, " or lithe set of all pleasures. " It is set
theory's lack of structure (mass, dimension, color, emotion, race, class, gender, religion,
history, etc.) that makes it such an ample notion: anything can be in a set. And it is
this very omnivorous nature of the concept of set that gave rise to the most funda­
mental crisis in the twentieth century, which is instantiated by Russell's paradox and
the paradox of the set of all sets. But here I stop, since my concern is not to explicate
or repair set theory but to pass on to fields richer than bare sets. In fact, the very
enormity and brilliance of Badiou's effort to construct a neo-Platonist ontology on set
theory testifies to the sparseness of the theory which necessitates the effort. Just one
step up from bare set theory takes us to pOint set topology, the next sparsest set of
concepts in mathematics, built from the raw material of sets but now admitting more
structure.
It may appear marvelous that what seems like the barest whiff of structure yields
such an enormously powerful set of concepts and theorems. But this should be no
more surprising than Galileo's observation that the book of Nature is written in math­
ematics, if one regards mathematics from a Latourian perspective as relatively high­
level machines of inSCription of material processes.9 In this chapter, we can only touch
on the most elementary concepts and theorems, but they already seem fertile for our
interests in philosophy of media, art, and technoscience.
Point set topology is one of the most primordial modes of articulation available to
us. It is even more primordial than counting. Primordial does not mean foundational,
however: it means that no other compactly articulated concepts are ready to hand
from which to construct an argument, in the given scope of reasoning. Topology is
one of the most fundamental fields of mathematicS, and the open set is its most fun­
damental notion. We begin with pOint set topology, not set theory, because, pace
Badiou, I believe that set theory is too sparse to accommodate being in\the world
without severe distortions of our felt experience. Two observations substanftate this:
( 1 ) Russell and Whitehead took hundreds of pages to establish, from set theory alone,
the integers 1, 2, 3, . . . as sets built out of the empty set {0, {0}, {0, {0}}, {0,{0,
{0}}} . . . } . They prove theorem * 1 02, that 1 + 1 2, after about 1,000 pages.
=

(2) In a tour de force effort, for which he received the Fields Medal, Paul Cohen
established the independence of the continuum hypothesis from the axiom of choice.
In our context, this demonstrates that the continuum is ontologically distinct from
even the transfinitization of ordination, number, count.
1 70 C h a pter 6

Point set topology provides articulations of the notions of open (closed) set, extent,
neighborhood (proximity), connectedness, convergence, limit, and continuous trans­
formation, all without relying on numerical measure or metric. Yet, as we will see, we
can make more certain statements about qualitative, i.e., topological, behavior than any
that can be made with numerical measure. Moreover, having such primordial structure
means that topological arguments start with less conceptual machinery, which appeals
to the minimalist taste. Readers who slogged through epsilon-delta proofs will appreci­
ate a notion of continuity built only out of the elementary notions of open set and
inverse map.
The open set captures the notion of a set that welcomes members, rather than having
a sharp litmus test for membership. In fact its most fundamental characterization is
the following: If x is in the set a, then there is some complete neighborhood of x
entirely contained inside O. What are some examples of an open set? A mundane one
would be from demographics. Say that we are restricting access to a movie theater to
people ages 13 to 1 7. At those boundary ages, disputes inevitably emerge: how close
to the edge may one be and still be admitted? If we were to say 13 and older, someone
who is 12 years, 3 64 days, 23 hours, and S9 minutes old may argue that they are really
already 13 up to the imprecision of clock technology. Let's say we restrict admission
to those who are strictly older than 13 and strictly younger than 1 7. Then one would
have a margin, but an undefined sort of margin: any margin will do, so long as that
margin is not nil. For example, one test could be for the would-be theatergoer to pull
in someone who is younger, but provably older than 1 3 . That would suffice.
A more nuanced example comes from political economy. Anthropologist Cori
Hayden has studied the complex and ever-shifting taxonomies of pharmaceuticals in
Mexico, ranging from brand-name "originals" to "generics, " interchangeable generics,
and "similar" drugs. She identifies brand-name originals (the "Originator" holding the
initial patent), generic medicine (same compound, no brand name, not proven bio­
equivalent), branded generic (same compound, branded by generics manufacturer, not
proven to be bioequivalent), interchangeable generic (same compound, bioequiva­
lent), and " similar" (nonbioequivalent copy) . The last category is recognized by the
World Health Organization, but not by Mexican health regulations. Indeed, a chain
of pharmacies has been built along this last category, Dr. Simi's Similares™. Hayden
considers how Similarity and equivalency are contested in Mexico among transna­
tional pharmaceutical corporations, other retailers, advertisers, and the public (itself
a contested set of sets) . Numerical measurement is inadequate to the shifting
but definite and perhaps overlapping regions of similarity in the world of these
pharmaceuticals. to
The rigorous concept of open set concretizes this notion of a comparative test and
region from their particular contexts. l l The conceptually deepest aspect of the con­
cretization is that it leaves behind the concept of number, or, even more deeply, the
Topology, M a n ifolds, Dynamical System s, M easure, and B u n d les 1 71

very concept of in-principle-numeric measure. In other words, one does not need to
measure anything using some metric (a distance, whether physical or "abstract") or
number in order to apply this test for openness.
The open set is the most basic notion in paint set topology, but a set is never defin­
able as open in itself; it is always defined relative to a topology, which is a set X,
together with a family of the subsets of X that are declared to be open, satisfying some
basic axioms. Which family sets are called open is up to you, the designer of the
topology, provided only that the subsets in this family satisfy the following set of
conditions:
Axioms of Topology
1 . If A and B are open, then A n B is open.
2. The arbitrary union of open sets is open.
3 . The total set X and the empty set 0 are both open.
I wish to underline the openness of the concept of open set: given a set X-a
universe-there is not necessarily a unique topology. More than one topology may be
defined on a given set. For example, the weakest or sparsest topology would be X and
the empty set 0. And the fullest or finest one would be all the subsets of X. The main
lesson here is that the art of a topologist even at this elementary level contains a great
deal of creative flexibility, that there is no transcendental principle determining a
unique topology for a given set X. A topology is always a choice relative to a universe­
set, satisfying some light conditions that enable a conversation built upon provable
theorems (see figure 6. S ) . By definition, a subset of X is said to be closed if its comple­
ment with respect to X is open. By that definition and the axioms, it follows that X
and the empty set 0 are both open and closed.
Certain topologies are more natural to us than others. For example, you may expect
that given any two distinct points a, b in X you ought to be able to find two open
sets around each that do not meet, Le., that they can each be contained in their own
bubble. But it may be that the elements (points) of a topology are all entangled in
some way (e.g., if they are the rays that meet at the origin) and the set of sets declared
"open" is too sparse to separate these elements. One example of a very sparse topology
would be the one in which the only open sets are the empty set 0 and the entire
space X. No two distinct points are separated according to that pathologically sparse
topology. (Mathematicians call such unpleasant and complicating situations patholo­
gies, but have various ways to deal with them by careful construction and definition.)
To exclude such pathologies, we use the following:
Definition: A space X is Hausdorff (separable) if any two points a, b are contained
in disjoint open neighborhoods U, V: a E V, and b E U, U n V 0. =

A key aspect o f Hausdorff spaces is that two nonidentical points can b e distinguished
using fine lenses of open sets, and vice versa. We will make this more precise when
1 72 C h a pter 6

we consider the notion of limit and points of accumulation. We can prove another key
feature of Hausdorff spaces, which is that if (if!) a sequence has a limit, then that limit
is unique.
Definition: A subset C of X is closed if its complement is open in X.
Every set X has at least two topologies. The coarsest topology is the one where the
only open sets are X and the empty set 0. And the finest topology is the one in which
all the subsets are declared to be open.
An arbitrary subset U of X may be neither open nor closed. Take, for example, the
set of points in the cone of half-open segments based at the origin of Xi ;:: 0, but whose
distance from the origin is strictly less than 1 : (x ) 2 + (X 2) 2 + . . . + (xn) 2 < 1 .

Cover, Basis
Given a subset Q c X of the topological space X, a covering of Q is a set of open sets
in X such that their union contains Q. A key pOint is that the sets be open in X. A

F i g u re 6 . 3
Half-open cone in JR2 : it includes points on the vertical and horizontal rays, but excludes those
on the arc. Diagram by the author.
Topology, M a n ifolds, Dynamical System s, Measure, and B u n d l e s 1 73

- '" ...
, �
... ,
,
'" .-I
... - u .. ...
V '\
. :�
... ,
• • ' •
a b ..
" •
-
,
a '
'"
•

•
b )
,
,
" , -"' .r
-

F i g u re 6.4
Hausdorff separability: any two points a, b are contained in disj oint open neighborhoods U, V,
a in U and b in V. Diagram courtesy of author.

Figure 6.S
Two coverings by a union of open discs and a union of open irregular subsets of the plane.
Diagram by the author.

covering does not have to be finite (or even countably infinite) . For example, any
subset S of a metric space, no matter how pathological-imagine a monstrously het­
erogeneous cloud of shards and dust-has a covering. Just ):ake for the covering a set
of balls of radius < centered on the points of Q:
U B, (x) .
XEQ

For any radius < > 0, this set contains Q. There are as many balls as theret�re points
in Q, so if Q contains an uncountable number of points, then this covering has an
uncountable number of balls. It does the j ob, but extravagantly, transfinitely.
A basis for the topological space X is a family of the open sets in X such that every
subset of X has a cover comprising elements from that family. There can be more than
one basis-usually an infinite number of bases-for a space X relative to a given
topology.
As one example of a topology, consider the topology Tl generated by open discs.
Compare it with the topology T2 generated by infinite strips. For every set B that can
be covered by some union of open diSCS, can B also be covered by some union of open
1 74 C h a pter 6

strips? If it can and the converse is true as well, then we say that the two topologies
are in a practical sense the same. It's not true that any family of subsets of a topologi­
cal space V can be extended by arbitrary unions and intersections into a topology for
V, even if the initial family itself contains an infinite number of sets and the union
of the family has unbounded extent. Regarding the x-y plane P as a subset of R3 , con­
sider the family of sets generated by (countable) intersections and arbitrary unions of
subsets of P. Any union or intersection of two subsets of P will be another subset of
P. Now take a "thick" subset of the full R3 , say the unit ball B centered at <0, 0, Vz>,
which intersects the plane P, but most of whose points are not in P. No union or
intersection of planar subsets in P can cover the ball B.
Notice that these notions of openness and covering do not require any notion of
dimension, so they're more primordial than dimensionality. A topological space does
not have to have the property of dimension! But if our topological space happens indeed
to be dimensional, in particular if it has the structure of a vector space like R3 , then
we see that there's some deeper relation between a set's characteristic of being an open
set and its dimensionality. Two-dimensional, in particular planar, subsets of R3 cannot
be open in any topology R3 .

Vector S paces
A vector space V is a set that has the structure of Rn, in other words its structure is
isomorphic to the product of n copies of the real number line R. Therefore any element
of such a space V can be indexed by an n-tuple of real numbers, i.e., a vector of dimen­
sion n: <X l ! Xz, . . . , Xn>. Although a vector space may seem canonical in man-made
parts of our world, in fact the ubiquity is itself an artifact of the convenience of a
particular form of linear thinking. (Here, linearity does not refer to a geometric line,
but to the order of differentiability of a mapping between spaces. We will deal with
mappings in a few paragraphs.)

An Application to Alexander's Patterns

Reterritorializing (to borrow Deleuze and Guattari's term for a tactic) these concepts
to perhaps more familiar ground, consider how we might characterize a community
in less demographic terms. An obvious convention would be to define community in
terms of set membership, or as a network of explicit nodes and relations. But com­
munities take shape as much around centers as along borders, perhaps more SO. I Z In
other words, communities can have the character of open sets and textured gradients
more than crisp boundaries. And to enter or exit a community can have a dynamic
that bears less affinity with a formal crisply temporally marked demarcation than with
degrees of covering.
A set (space) may not have any features that resemble a vector space.
Topology, M a n ifolds, Dynam ical System s, Measure, and B u n d l e s 1 75

Of Alexander's 1 5 fundamental patterns of the vital built and natural environment

(described in chapter 5),1 3 the m ore shape-oriented include: interlock, border, good shape,
and most importantly center. Of course, the space of features that build vitality is
infinite and infinitely nuanced, and much more specific in every concrete instance,
so how can we interpret Alexander's 15 patterns? One way is to see them as a basis in
a subspace of the topological space of patterns of built structure. Certain patterns are
indeed geometrical, or more accurately have to do with spatial relations such as degree
or diversity of spatial rhythm, or the propensity to develop centers of tension or atten­
tion. Notice that, as is clear with the "smoke" example, these patterns intrinsically
intertwine the observer with the observed.
Moreover, we do not need to have any notion of scaling a pattern, e.g., to multiply
the number of centers by some numerical constant, or otherwise numerically quantify
a pattern.
We can interpret the foundational character of Alexander's 1 5 patterns in the sense
of a covering generated by a particular family of patterns (subsets) in the space of all
patterns of living in the built environment. In order to operationalize a topological
approach, we would need to operationalize intersection and union of two patterns. The
obvious move would be to logically combine them: for example, a design configura­
tion that exemplifies both "good interlock" and "no two parts the same." But another
could be first to apply the operation of making a design have more interlock, and then
to give series more individuation. Indeed, given that Alexander emphasizes that his
patterns are actually transformations rather than particular forms, the latter could be
a more plausible approach to topologizing an Alexandrian space of patterns.
This emphasis on transformation, rather than "things themselves, " plus our previ­
ous discussion of dynamical examples, motivate us to a notion of mappings of topo­
logical spaces.

Mapping
Given topological manifolds X and Y, we can define maps (also called fu nctions or
mappings) from one to another. With f. X�Y, to every element x in X (written x E
X) we associate an element labeled ((x) in Y. The image of ( is the subset o}y defined
by all the points y in Y for which there is an x that gets mapped to y under f More
compactly: {y E Y 1 3x E X, ((x) y} . The only condition is that the result of applying
=

the mapping ( is well-defined; i.e., that the result is determinate and unique for the
given x. A rigorous test: If ((X l ) :t ((xz), then X l :t Xz for any X l i Xz in X.
Given two topological manifolds M and N, consider the set of all mappings that in
some sense respect the topological structure of these spaces; approximately put, open
sets in the range space N come from, under f, open sets in the domain M. We call such
mappings continuous homeomorphisms, and we label the set of such mappings
Hom(M, N). One particularly interesting, infinite-dimensional subspace of Hom(M, N)
1 76 C h a pter 6

is the set of differentiable maps Diff(X, Y) from X to Y. (To define this requires some
calculus, but for now we will say that in the case that X and Y are vector spaces, a
differentiable function, at every point x, has some local approximation by a linear
mapping.)14 On top of Diff(X, y), we can define further a mapping defined not on the
base spaces X and Y, but on the function space Diff(X, Y). We will call such a mapping
an operator, just to help us remember that it maps a mapping to a mapping. An impor­
tant example would be a differential operator like V: Diff(X, Y) � Hom(1M, TN). V is
a mapping that maps a function f at a point p in M to its differential
(6. 1)
a linear mapping from TpM to Tf(p)N, where TpM and Tf(p)N are the vector spaces tangent
to M and N at p and f(P) , respectively. This provides an enormous expressive range to
any analysis of transformation and functional change. You can see that it allows us
to lift the discussion of mappings to a tower of structures, or to higher-order
operators.
Computer engineers, cognitive scientists, and their clients in cultural studies or
social sciences are typically quite cavalier about the domain or range of a mapping.
But in order to makes sense of a map f, it's necessary to ask: What is its domain? What
is its range? For example, George Lakoff defined "metaphor" as a structural homomor­
phism from one cognitive domain to another. 1s What does that mean? What is a
cognitive domain? Is it like an open set in a topological vector space? If this metaphor
is supposedly a map called, say, f, is this map nontrivial: Image [f) '* 0? Is it even well­
defined: f(x) '* f(y) =} x '* y? One expects that a metaphor, if indeed it can be regarded
as a mapping, can certainly map one entity to two or more entities.

Continuous, Connected, S i mply Connected

Leibniz, one of the proponents of the view of matter to which I am subscribing in
this work, introduced a material law of continuity which he described in a letter to
Fontenelle in 1 699 as "not altogether of geometric necessity, as when it decrees that
,,
there is no change by a leap. 16 This expresses an axiom about the fullness of the
world, a world not atomic but plenist. One way to introduce this idea is via a related
concept of a simply connected set. Intuitively, we can say that a set is simply connected
if we can draw a curve between any two pOints in that set without having to lift the
pen. But this is a Gedanken test, a quasi-physical action to be imagined in order to
determine some quasi-physical property. If the curve is broken, then one imagines
there is a point at which the pen must be picked up off the paper and set down
somewhere else to continue the drawing of the curve-Leibniz's leap. But there are
vastly different sorts of sets, not just curves, for many of which it does not make
sense to speak of dimension, and which cannot be modeled by a two-dimensional
sheet of paper. For example, consider the clouds in the sky, or an aroma of perfume,
Topology, M a n ifolds, Dynam ical System s, Measure, and B u n d l e s 1 77

or the set of all metaphors. So we would need some concept that articulates the
intuition of continuity more generally. Look more carefully at a (bounded) curve
segment, broken at (at least) a single point. In that case, it is also the union of two
open subintervals, or subarcs. It is this Gedanken test that we can generalize to arbi­
trary sets: Can a set be decomposed into two open subsets? If each of the two com­
ponent sets is open, then we imagine that we can slip a boundary-a "leap" -in
between them. So, a set is connected, by definition, if and only if it cannot be decom­
posed into two open subsets.
What this Gedanken test does is enact in the imagination a transformation, a
mapping, from one set, the interval, into another set, a curve which may be broken
or unbroken. It is a subtle and profound shift of conceptual register to turn our atten­
tion from sets to the transformations of sets, to what is called a space of mappings.
To articulate continuity, we really are asking a question not about a set (an object) U
c X but about a mapping between topological spaces, say rp: X � Y. In this case, we

say that a mapping rp from topological space X to topological space Y is continuous if

and only if the preimage of any open set in its range space Y is open in its domain space
Xi mnemonically, " rp-l [open] is open. " (For any subset K in the range space Y, the pre­
image of K with respect to a map rp: X � Y is the set of points p in X such that rp[P] is
in K.) This is a considerably more expressive and supple test than trying to draw a
curve in your imagination, but takes a fair amount of experience to internalize. This
was one of the more profound conceptual moves in the history of twentieth-century
mathematics, whose philosophical consequences we are just beginning to realize. Such
a supple concept of continuity offers us a way to begin to articulate continuity in the
full extent of felt experience of the world without any recourse to metric or dimension.
And yet this notion of continuity agrees with the more familiar, restricted, metric
concepts of continuity. For example, in the case of the real line R, a classical formal
way of describing continuity is to use the ordinary Euclidean distance from absolute
value on R:
Metric " epsilon-delta" definition of continuity for functions of the real line:
f. R � IRI is continuous at Xo if and only if for all E > 0 there is a /» q such that
Ix - xol < /) ::::} If[x] - f[xol l < Eo fis called a continuous mapping if it is continuo� s at every
point x E R at which it is defined.
We can apply this so-called epsilon-delta characterization of continuity to any func­
tion of the real line. You should draw some diagrams and convince yourself that this
epsilon-delta definition of continuity agrees with the more purely topological notion
of continuity. (In other words, if a function mapping R to R is continuous in one
sense, then it is continuous in the other sense as well, and conversely.)
Theorem 1 (connected image) . The image under a continuous map f. X � Y, of a
connected set K is connected.
1 78 Chapter 6

Proof: Suppose not. Then there are two disj oint open subsets of Y, call them V and
W, such that tIKl c VuW. Since {is continuous, by definition tl [V] and tl W] are both
open subsets of X. We will prove that these are disjoint, and cover K, which will con­
tradict the hypothesis that K is connected. To show that these two preimages are
disjoint, suppose p is a point in their intersection. But then tIP] is in both V and W,
which cannot be, because V and W are disjoint; so their preimages are also disjoint.
Next, take any element q in K. By our hypothesis, the element tIq] must be in either
V or W; say V. Then q is in the pre image tl [ V] . In either case, q is in the union of
tl [V] and tl [W] , so we've shown that K is covered by these two preimages, which are
disjoint, open sets. This contradiction implies that our hypothesis must be false.
Therefore tIKl is a connected subset of Y. Q.E.D.

Compactness, limit, Convergence

Recall that a covering (or cover) of an arbitrary set K in a topological space X is a
family of open subsets Bi c X such that K c U Bi. A set K is called compact if every
i
cover of that set has a finite subcover. At first sight this seems like a strange property
to introduce, especially right after such preliminary concepts. But the great insight is
that compactness is the way for local qualities to be global qualities as well. (We will
see a rich application of this later when we discuss the existence of flows on compact
differentiable manifolds.) "Local" means in a neighborhood of a pOint. Notice that
this has no requirement regarding distance.

Theorem 2 (compact image)

The image of a compact set under a continuous map is compact.

Sketch of proof
Look at the image. Take any covering by a potentially infinite number of open subsets.
Their preimages are open sets since the map is continuous. The union of the preim­
ages covers the domain. Since the domain is compact, a finite number of them form
a subcover. Project that into the target via the map, and voila.
Compactness makes sense even in the absence of a metric, yet agrees with notions
like properties of sequences in the case of metric spaces.
For example, suppose the set were the set of all points with rational coordinates in
the unit square bounded by <0, 0>, <0, 1>, < I , 0>, < 1 , 1>, Le., coordinates of the form
p/q where p and q are integers. Given that the set of rational numbers is a dense subset
of the real numbers, we can show that this infinite set is dense in the unit square. But
since every two rationals is separated by a continuum of real numbers, this set of
points with coordinates of the form <p/q, r/s>, p, q, r, 5 integers, is a radical form of
dust. Consider the infinite family of balls Br(z) where the radius is any small positive

number, say r = ;n. For any fixed radius, we have an infinite family of balls centered
Topology, M a n ifolds, Dynamical System s, Measure, and B u n d l e s 1 79

on all the rational points, and since the rational points are dense in the square, we
have an extravagantly thick cover of the square. But given that the square is compact,
being bounded and closed in RZ, we know that no matter how finely we draw these
balls, a finite number of them suffice to cover the square. Now, we can use our geo­
metric intuition to scaffold this small exercise, but the powerful insight is that this
does not need any geometry to hold. In fact, this could work for a unit "cuboid" in
an infinite-dimensional space, say R�, a cuboid that is a product of an infinite number
of copies of the unit interval [0, 1] .
One example of where this can come in handy is when we articulate a bundle of
services invented by a community. Since the number of services is limited only by the
ingenuity and the evolving needs of a community, so long as the community endures,
more and more services can be invented. Given that services are as distinct as human
fictive faculties can make them, to array them each along their own "dimension"
requires less analytic categorizing work than to create categories that are artificial by
definition. Pigeonholing fictive objects into a finite number of categories is a practical
reduction of the unboundedly rich universe of fictive imagination that we inherit
from the age of premechanical memory. (Here I am borrowing Simondon's distinction
between mechanical and human faculties of memory.)
Given an infinite sequence of points {X l i Xz, X3, }, a tail is all the elements of
. • •

that sequence beyond a certain finite index, i.e., the sequence {Yl l Yz, Y3, . . . } is a tail
of {X l i Xz, X3,
• } if there is some integer N such that Yi XN i, for all i 1, 2, 3, . . .
• • = + =

Given a sequence of points {X l i Xz, X3, } in a topological space M, a point Z E

. • •

M is a called a limit of the given sequence if for every neighborhood 0 containing z,

a tail of the sequence is entirely contained in O. We say that the sequence converges
to its limit z. (Later in the chapter I will suggest how this strengthens Whitehead's
construction of abstractive sets as well as his notion of eternal objects.) As an exerCise,
try proving that in Hausdorff space if a sequence has a limit, that limit is unique.
(This is another hint at the power of defining proximity without recourse to metric
measurement.)
Notice that this needs no measure of distance, and so makes sense in any set that
has a topology. We could dispense with the notion of sequence altoget�r and say
that for an arbitrary subset 0, a point Z E M is an accumulation point of 0 if a'J;1y neigh­
borhood of z contains all but a finite number of elements of O. One of the chief
advantages of this perspective is that it gives the power to articulate limit without
reliance on unidimensionality or any geometric conceptual figure.
A set is sequentially compact if every sequence {X l i Xz, X3, } has a convergent
. . •

subsequence {Xa( l ), xa(Z), Xa(3),• }, where a(n) is an increasing sequence of indices

• •

indexed on the natural numbers 1, 2, 3 . . .

Theorem 3
A subset 0 of a metric space is sequentially compact if and only if it is compact.
1 80 Chapter 6

Theorem 4
A subset Q of a metric space is compact if and only if it is closed and bounded.
Recall the theorem above that the continuous map of a compact set in the domain
space X is compact in the range space Y.
Perhaps the most fertile quality, at a primordial level, of a continuous map f on a
compact set K taking values in the reals R, which is complete and ordered, is that we
can say something definite about the distribution of values taken by f over the
compact domain Q.

Theorem 5 (maximum modulus)

A real-valued continuous map from a compact set achieves its maximum and minimum
on that set.

Sketch of proof
By Theorem 2 the image of this compact set Q under the continuous map is compact
in the reals R. Since the image is compact, by Theorem 4 it is a closed and bounded
subset of R. With some thought, we see that the image contains its infimum and
supremum values under the map. Then we can recover the pOints in Q that map to
those maximum and minimum points.
(Remember that the term "image" has a technical, set-theoretic meaning and does not
presume human vision or opticality.)

I nterlude

Topological Dynamics of Social and Cultural Ontogenesis

Let's pause to see where we are and where we are headed. Based on some primordial
concepts of open set, topology, basis, mapping, and continuity, we have built up a
miniature theory that allows us to describe phenomena in qualitative terms and make
definite yet nonreductive statements about them. These statements, being axioms and
theorems, hold in all the situations where we have checked that the three basic condi­
tions for a topology are satisfied. They are propositional in Isabelle Stenger's sense.
Now we work on describing dynamical systems, which are usually introduced as
systems of differential equations, using such qualitative articulations. On one hand we
will be able to give more delicate and concrete nuances to flow, change, and becoming
than are typically available to readers of process philosophy. And on the other hand, we
do not bind ourselves to numerical empiricism or to reductive forms like graphs.
In discussing the process of cultural dynamics, we must always account for the
radical entanglement of the observer with the observed. This implies that descriptions
of a situation or a process are always situated. (As Maturana and Varela write in The
Topology, M a n ifolds, Dynamical Systems, Measure, and B u n d l e s 1 81

Tree ofKnowledge, everything that is said is said by somebody, somewhere.) I ? So descrip­

tions are articulations, and the mode of articulation matters. Topology provides an
an exact (in Deleuze's sense) mode of articulation that does not need numerical
measure, equations, exact data, statistics.
One of the central challenges to anthropology and social sciences has been the
contest between "quantitative" and "cultural" methods. The three-volume Foundations
ofMeasurement, published between 1 9 7 1 and 1 990, epitomized significant approaches
to measuring cultural and social dynamics, across a much ampler range of techniques
than the statistical or numerically based models that typify quantitative discourse. I S
Despite such an encyclopedic project, we can still advance the hypothesis that any
sufficiently thick account of a human phenomenon, especially as a dynamical process,
would be too dense to be adequately modeled by numerical models alone. This seem­
ingly simple hypothesis evokes incompatible and equally certain responses. The
incompatibility of those responses marks this as a proposition worth investigating.
Technoscientifically powered rationality demands rigor, prediction, and generaliza­
tion. Cultural, literary, and historical approaches are rigorous in their domains, but
compete with difficulty against the rhetorical and political strength of the predictive
and general powers afforded by a system of quantitative measurement. Let's call this
debate about the adequacy of quantitative or qualitative methods the social scientific
measurement problem. 1 9
The most minimal mode of articulation available to us is the mode of collectives,
sets. But as I've noted above, bare sets are too bare, and in fact offer grip to Russellian
paradoxes in their bareness. The next simplest mode of articulation is that of proxim­
ity, the motivating notion for topology. In fact it is scaffolded by the more primordial
notion of "open" set, augmented by the set-theoretic notions of intersection and
union. Along the way, we avoid numerical measure, for several reasons. A practical
one is that, far from Galileo's claim, most phenomena in the world come to us without
numerical measure or metric. In fact, the move toward "data-driven" applications
confuses number-measure for the numbered thing, which is a desiccating move. We
propose to try the topological as an an exact mode of articulation that retains as much
as possible the wet, juicy messiness of the world, without metrizing or premature
orthogonalization.
Topology gives us a way to articulate the concept of limit without reference to
sequence in a metric space. Originally invented to Sidestep conundrums dating back to
Zeno's paradoxes, this enables, for example, a way to articulate destinations with refer­
ence to regions of world rather than to sequences of particular point-events, with wide­
ranging consequences. For example, a topological formulation of convergence and
limit permits very powerful and expressive modes of articulation for processes such as
those that provide the ontogenetiC dynamiCS of Whitehead's process-oriented ontol­
ogy. But very much unlike Whitehead's sequences of ovate sets in his constructions in
1 82 C h a pter 6

the second, more technical part of Process and Reality, topological open sets do not have
to have any particular geometric property. They do not even have to be connected.
Moreover, under adequate, qualitatively expressed conditions, we can rigorously estab­
lish "qualitative" phenomena such as periodicity, convergence, and existence of
maxima or minima.

Methodological Significance
So far in this chapter, I have only introduced a handful of the most elementary concepts
of topology as a contribution toward more generous articulations of cultural dynamics
without number or metric. What is the methodological significance of such an
approach? Rather than begin with a complex schema and observational apparatus,
we can try to take a minimally scaffolded approach to the phenomena: minimal in
language and in formal schema. As we dwell in the phenomena, site, event, we can
successively identify salient features and then successively invent articulations that
trace the phenomena. We do not pretend at any stage to completely capture what we
articulate. Indeed, as I wrote at the outset of this chapter, I introduce these topological
concepts and theorems not for the purpose of providing a truer model of reality or even
of perception, but as a mode of articulation and on occasion of poetic expression.
There is a much stronger methodological potential: topological concepts can
provide enough grip that we can apply theorems. The fundamental point is that typi­
cally a mathematical theorem's hypotheses do not need to be calibrated by any
numerical measure, and therein lies its potential for supple adequacy. In fact, the vast
majority of mathematics avoids explicit numerical constants and explicit equations,
and this is especially true of topology. What this implies is that we can make argu­
ments that are both qualitative and definitive.
So what, in sum, have we encountered so far on this j ourney? We have a non-ego­
based, number-free, and metric-free account of experience that respects evidence of
continuous lived experience but does not reduce to sense perception or ego-centered
experience. We have an essential concept of continuity both as a quality of lived
experience and as a mode of description of such experience. We have here the seed
of an approach to poiesis and expressive experience that is "nonclassical" in the senses
of quantum theory and measure theory, avoiding recourse to stochastic methods,
statistics, and informatic sweepings of the lifeworld under the rug. And we have the
possibility of a radically decentered, deanthropomorphized concept of experience and
cultural dynamics. This avoids methodological and critical problems with reductive
modeling and the more canonical interpretations of phenomenology. And it provides
a conceptual trellis for the condensation of subjectivity in the endless exfoliation of
experience in the world.
At the very least, we should recognize that the classical figures of the line, the circle,
and the sinusoidal wave are not adequate to the temporality of human experience
and phenomena. Before we pass to the measure-theoretic, we scaffold the discussion
with groups and manifolds.
Topology, M a n ifolds, Dyna m ical System s, Measure, and B u n d l es 1 83

An Excursion into the Discrete: G ro u p Algebra

As against the body of concepts from continuous mathematics that I would like to
introduce into humanistic and artistic discourse, it is useful to know something of the
contrasting discrete mathematics. By discrete set, I mean structure that is modeled by
a set of distinct, isolated points, such as the four cardinal directions, or the integers,
or conventionally a set of people. An algebraic set has some additional structure: at
least an operation and rules governing how elements of the set combine under appli­
cations of the operation. An example is the rigid symmetries of a polyhedral solid, say
a cylindrical faceted prism, that carry it into a congruent shape with the same center
and axis of symmetry. Another example might be a word algebra: the concatenation
operation on letters from a fixed alphabet, together perhaps with some rules on what
to do with repeated contiguous letters. Whereas the first example is a finite discrete
set, the latter is an infinite set.
Popular and philosophical accounts of mathematics commonly identify two large
continents of activity: that of sizing and shaping and that of counting, with priority
given to the latter. But this is far too coarse a distinction, because it identifies radically
different and even incommensurate modes of articulation and argument. Even our
cursory foray into topology already furnishes our imagination with a great variety of
quite distinct modes of sizing and shaping. The same is true of the more major cat­
egory of mathematical action: counting.
Groups are sets with an additional structure that draws only a handful of the essen­
tial aspects of the real numbers. Group structure is more primordial than the structure
of real or complex numbers, and yet it already permits a very rich articulation. For
example, two areas of activity in which we find group structure are the growth and
symmetries of crystals and the permutation of letters or characters in a piece of text
in an alphabetic language.
In the simplest case, a finite group is a finite set of objects G, together with an
operation *, such that:
( 1 ) Given any two elements g, h of G, g*h is also an element of G.
\
(2) There is a distinguished element, called the identity element, say e, sU h that e*g
= g*e g for any element g in G.
=

(3) For every element g, there is an element h called the inverse of g, such that g*h =

h*g e.
=

That's all. The symmetries of a regular crystal, that is, the operations of rigid rotations
that take a crystal shape into a congruent shape with the same axes of symmetry as
the original object, form a group. The set of integers form a group Z under the opera­
tion of addition. Their identity element is O. This group Z has an infinite number of
elements: for any element r in Z, r + 1 > r is another element in the group. But a group
can also have a finite number of elements.
1 84 C h a pter 6

Think of a Rubik's cube, a block with faces colored differently for convenience, and
call S a set of symmetries that carry the shape into itself. Instead of addition, one easy
operation can be composition: any two symmetries g and h can be combined to form
a new operation goh: first apply h to the Rubik's cube, then apply g. The composite
operation still maps the cube into itself, so it is an element of S. A rotation g by 90
degrees about an axis through the midpoints of opposite faces of the cube is an
element of S. Another element of S is the reflection across a plane bisecting the cube
parallel to a face. You can verify that this set S is in fact closed under composition
operator and every element in S can be undone by another element of S, so S is a
0,

group. But this group S differs quite significantly from the group of integers Z in several
respects. First, it has only a finite number of elements. Some elements when composed
with themselves a certain number of times become the identity; in fact gogogog g4 = =

identity element e. So this is quite different from how the integers work.20 Now this
group S is a discrete set of elements-you can enumerate and characterize each of its
basic elements. But we can in fact situate S inside a larger group of symmetries, that
of a sphere.
Now, given two groups, there are ways to map from one group to another. This is
a higher order of mapping than the transformations that constitute the example
groups in the previous section. A group homomorphism is a mapping q> from one group
{X, * } with operation * to another group ( V, e ) with operation e that respects the
respective group structures in the following sense. q> carries the identity element of X
to the identity element of Y, and for all elements r and s in X,
=
q>[r*s] q>[r] e q>[s] . (6.2)
Here is one mathematical relation best understood when spoken aloud: "the map of
the product is the product of the maps. " Given this definition of a homomorphism
as mappings between groups that respect their group structure, an isomorphism is a
homomorphism that is also a bijection. In this paragraph's notation, for every element
p in Y there is exactly one element in X that is mapped by q> to p. If there is an iso­
morphism between two spaces X and Y, then in a rigorous sense we can say that X
and Y bear a structural, operational equivalence. This provides a more supple and articu­
lated notion of equivalence than bare identity. Given this tour of algebra, let's return to
the main road from topology to differential geometry.

Differential G eometry: Continuous and Differentiable Structures on Topological and

Riemannian M a n ifolds

Given the relatively flexible concepts of topology, we will explore how topological
concepts help articulate the formation and dissolution of objects from material fields­
the process-oriented conception of material (rather than formal) ontogenesis as
Topology, M a n ifolds, Dynamical System s, Measure, and B u n d l e s 1 85

described in chapter S . Most importantly, we can explore how to articulate matter

infused with value in terms that Bilgrami may recognize. After Rene Thom, Jean Petitot
and his students have been nearly unique in developing this rigorous and poetic
approach to matter and experience via rich mathematical structures. We will visit
Petitot's approach via fiber bundles to the apperception of qualities after developing
some of the background concepts.
An n-dimensional continuous (or differentiable) manifold is a topological space
that locally looks like a piece of Rn, together with a compatibility condition. "Looks
like" can be made precise by saying that there is a homeomorphism (diffeomorphism
in the case of differentiable manifolds) between some neighborhood of every point
and a standard neighborhood (say, an open disk) of the origin in Rn. To be more
precise, a differentiable manifold M" of dimension n is a topological space with some
additional structure. There is a family of neighborhoods, call them l Ui}, such that their
union covers M, and a corresponding set of diffeorriorphisms l/Ji: U � Bn(O), where Bn(o)
is the unit ball centered at the origin in n-dimensional Euclidean space Rn. Each of
these diffeomorphisms confers coordinates to points in their domains. For consistency,
we require that these coordinate patches agree on their overlaps. Agreement, however,
is considerably more relaxed than identity. In differential geometry, this means only
the following: Take two coordinate domains of the covering that intersect nontrivially,
say U Ul n UZ "* 0. Then define Vl and Vz c IRn as the respective images of U under
=

the coordinate maps l/Jl and l/J2. The compatibility condition is that the composite map
l/Jz i/Jl l : Vl � Vz is a diffeomorphism mapping Rn to Rn. This composite map is called
a

the transition function between the two coordinate patches Vl and Vz• In order for
these coordinate patches to piece together, they have to satisfy a compatibility condi­
tion. (We'll return to this in the discussion of Deleuze and Guattari later in this
chapter.)
The fundamental distinction between geometry and topology, or perhaps more
accurately between a geometrical attitude and a topological one, is the concept of
metric. A metric g on a differentiable manifold M is an assignment of a number to any
pair of points that has the essential features of distance. For any points p, q, and r in
M, g has these features: \
(1) Positive-definiteness: g(p, q) ;? 0; g(p, q) 0 if and only if p
= = q.
(2) Symmetry: g(p , q) g( q, p)
=

(3) Triangle inequality: g(p , q) ;? g(p, r) + g( r, q)

In other words, detours always cost you. Whereas the topology of a set ultimately is
determined from a family of subsets that are declared to be open (satisfying the axioms
for a topology), the geometry of a set is that which ultimately derives from a metric,
a generalized distance relation defined for every pair of points in the set (satisfying
the axioms for a metric) . Just as there can be a number of distinct topologies on a
1 86 C h a pter 6

given set M, there can be a number (an infinite number in fact) of different metrics
on a given set M, each one constituting a different geometry which constitutes that
set as a metric manifold. In order that we be able to speak of change with respect to
position in the manifold, the manifold must be differentiable, that is, it must have at
least locally some features of the differentiability of a Euclidean space. That allows us
to measure, for example, the way a manifold is curving as one follows along the
manifold itself. In other words one must have a notion of intrinsic differencing, which
is called a covariant derivative, a difference that remains invariant with respect to
changes in coordinates (perspective) . Think about a displacement in a differentiable
manifold. Such a displacement must be measured by the metric on that manifold. So
in principle, displacements are calibrated by the first derivatives of the metric.
In the case of a curve in the plane, think of the tangents to the curve. If the curve
is smooth-with no corners or breaks-then at every point, there's a unique line
tangent to the curve that best approximates the curve. More precisely, at a given point
p in M, consider the curves passing through that point. Now take the general case of
M, a differentiable manifold of dimension n. Each curve in M has a well-defined tangent
vector at the point p. The set of all such tangents at p fills out a "hyperplane" that
has the structure of a vector space whose dimension is the dimension of M. If M is an
n-dimensional manifold locally like Euclidean space R", then at each point p in M the
associated tangent space, TpM, is isomorphic to the vector space Rn .

Excu rsion on Fiber Bundles

So, for each pOint p in M, there's an associated vector space TpM equivalent (more
precisely, isomorphic) to Rn . We can take the entire set E of pairs {p, TpM} and imagine
that to each point p is associated a vector space-the n-dimensional hyperplane gen­
eralization of a line. Then for any region in E, we can define a proj ection of E back
down to the manifold M, by simply mapping {p, TpM} to p. With a bit of work, one
can see that the total set E inherits the structure of a differentiable manifold from the
Riemannian structure of the differentiable manifold M. This whole assembly E, M, and
projection is called a tangent bundle, which provides a concrete model for a more
general concept of fiber bundles. This concept underwrites ideas ranging from Deleuze
and Guattari's smooth versus striated space to Jean Petitot's Husserlian interpretation
of the perception of objects, which we will visit at the end of this chapter.
A fiber bundle E over a base manifold M with a group or vector space F is a manifold
together with a mapping n:E � M such that for each x in M, its inverse image under lC,
denoted by lC-1 [X] , is homeomorphic (diffeomorphic) to F. The inverse image, lC-1 [X] , is
called a fiber. When F is a vector space, the bundle is called a vector bundle, and in that
case the fiber is homeomorphic (diffeomorphic) to a vector space. The map lC locally
acts like a projection: the preimage under lC of each neighborhood U in M is homeo­
morphic (diffeomorphic) to a product of U with the fixed group or vector space F.
Topology, M a n ifolds, Dyna m ical System s, M easure, and B u n d les 1 87

Curvature
The second feature of geometry, and the feature that distinguishes the classical Euclid­
ean geometry from the nonuniform geometry explicitly in Einstein's relativistic cos­
mology and Deleuze and Guattari's Mille plateaux, and implicitly in Whitehead's
Process and Reality, is curvature. In the previous section, we've seen that the first deriva­
tives of the metric of a differentiable manifold yield tangents to the manifold. We can
ask how those tangent vectors themselves vary from point to point on the manifold.
Think of how a fabulously unlucky ship is sucked into the maw of a vortex-its nose
turns ever more quickly and violently as the ship itself moves toward the center of
the vortex. Whereas the tangent is itself a first-order derivative, the change in the
tangent is a second-order derivative. In the example of the vortex, a flow that is non­
uniform across the manifold induces a nonzero, second-order derivative in the metric.
This second-order change of the metric is measured by the curvature.
Thinking this way leads us to consider the systems of difference-in more precise
terms, ordinary and partial differential equations on a manifold.

Ordinary and Partial Differential Equations on Manifolds

Flows
A flow on a manifold is a smooth map
(6.3)
such that at time 0, all points are mapped to themselves: ¢(x, 0) x, and such that
=

the flow moves points of M in a deterministic21 way along each trajectory: if you
know that a speck of dust at pOint x slides for some time t, to say a point y, and
if you know that a speck of dust at point y slides to a point z in a given duration s,
then you know that a speck of dust will slide to z in duration s + t. Put concisely:
¢(¢(x, t), s) = ¢( x , s + t). (6.4)

Given an n-dimensional manifold M', a vector field V is a mapping V: M � TxM that

assigns to each point x E M a tangent vector at that point x.
We can construct a vector field, a mapping M � TM, associated with a flow ¢(x, t)
in the following way. Let 1'P[t] be the trajectory passing through the point p defined
by the flow ¢; in other words, the mapping given as t � ¢(P, t) . This yields the tangent
vector to the curve, 1'P E TpM. This defines the vector field V associated with the flow ¢:
dyp
V(p) == : M � Tp M. (6.S)
dt
So a (tangential) vector field assigns a tangent vector to each point of M. Think of a
vector field as the wind over the manifold, but in the stratum of the potential. How
would the material in the manifold-the stratum of the actual-move under the force
1 88 C h a pter 6

of that wind? This is the relation between a potential (the vector field defining a
dense field of forces on M) and the actual movement in M that is induced by that
potential field.
We have shown that every flow-a globally defined entity-generates a vector
field-a locally defined entity. What is surprising is that, on a compact manifold, the
converse is also true: every vector field generates a flow. We sketch a proof, using a
theorem and a concept: the local existence and uniqueness of a solution to a system
of ordinary differential equations, and the concept of compactness. The existence
theorem yields, for every pOint p in M, a local interval Ip of "time" over which there
is a unique traj ectory. This interval depends on the point p. But since M is compact, it
follows (and this takes a bit of topological thinking) that there is a finite number of
such intervals whose union covers all the solutions. The intersection of a finite number
of parameter intervals all containing the origin must contain some nonempty interval,
say (-c, c), for some fixed c strictly greater than zero. (This is where compactness is
used.) This (-c, c) serves as a uniform domain of a flow that is defined on all of M. I
sketch a proof in such detail in order to show how powerfully compactness helps us
move from local to global phenomena without an explicit calculus or numerical data.
We turn our attention from curves that are the traj ectories under flows in general
to specific kinds of curves, those that minimize the distance traversed according to
the metric on M.
A geodesic is a curve y: [0, ) -7 M in the manifold M that locally minimizes lengths.
00

Equivalently we can characterize y[t] as the solution to a length minimization problem

between two points p, q E M: What is the curve y[t] such that y[a] p, y[b] q, whose
= =

length
Sly] '" s: Jg[y, y]dt (6 . 6)

is the minimum over the infinite-dimensional space of all curves j oining p to q, where,
with respect to local coordinates X i in a coordinate patch on the manifold, gii g[dX i , dx i ]
=

i
are the components of the metric tensor for M, and y is the ith coordinate of the
dy
tangent vector . From the calculus of variations, we know that the Euler-Lagrange
dt
equations for this functional are:
(6. 7)

where rt are functions of the first derivatives of the metric (called the Christoffel
symbols) . This set of ordinary differential equations (k 1 to the dimension of the
=

manifold) is called the geodesic equation.

Theorem: (local) existence of geodesics on a manifold. At every point p E M and
for every tangent vector V E TpM, there is a solution y[ t] to the geodesic equation (6. 7)
such that y[O] p and Y[t] V.
= =
Topology, M a nifolds, Dynamical System s, Measure, and B u n d les 1 89

TM ..-, g

! 1
M ..-, G

Fig u re 6.6
Lie group action on manifold M, lifting to their respective tangent spaces TM and Lie algebra g.
Diagram courtesy of author.

The proof of this relies on the fundamental theorem of the existence of solutions to
what are called "ordinary differential equations"-systems of differential equations
that involve derivatives of only one parameter.
Thus we have a way to define change and relational change not in terms of differ­
ences between atoms, but as emerging from continua and continuous fields defined
on continua-more precisely differentiable manifolds, vector bundles, and cross sec­
tions of vector bundles. Furnished with both the concepts of algebraic groups and
differentiable manifolds, we can ask whether any entities have both "discrete" and
"continuous" structure.

lie G roups
Lie groups are named after the Norwegian mathematician Sophus Lie ( 1 842-1 899) . The
theory of Lie groups has become one of the largest and most richly developed areas
of modern mathematics, and is used in huge areas of both pure and applied mathemat­
ics, ranging from analysis on manifolds to satellite tracking and cybernetics.
A Lie group is a set that is both a topological manifold and an algebraic group. This
means little until we say how the topological structure intertwines with the algebraic:
the group operation *: G x G -7 G is a continuous map. From this simple intertwining
comes some very rich structure.
First off, a Lie group, like any group, has a distinguished element, e, its unit element,
so as a manifold it has a distinguished point. This is quite different from most manifolds,
which do not have such an element distinguished by how it acts with other elements.
Nonetheless a Lie group can have considerably less structure than Euclidean space­
there is no global coordinate grid anywhere in sight. Moreover we can move from
point to point in this manifold in a most unusual way from a geometric point of view:
we simply multiply the unit element e by g to "arrive at" that given point g. We can
treat "motions" as multiplications by the group's operation. This is what makes qua­
ternions work to convert rotations into multiplications by a four-dimensional analog
of a (real) number, a generalization of the multiplication of complex numbers.22 This
classical nineteenth-century precursor to Lie algebra is a standard tool in computer
graphics, but this is a special case of a profoundly more extensive phenomenon.
1 90 C h a pter 6

I nvariants
An invariant of a manifold M is an entity defined on M that does not change under
continuous, invertible maps of M into itself (self-homeomorphisms, or automorphisms).
For example, the total mass of your body, as the sum of your cells' masses, is the same
no matter how you move the cells of your body around. So the total mass of your
body is an invariant with respect to nondestructive transportations of the cells inside
your body. The invariant entity could be a number, or some element of an algebra, or
an element of more complicated space; let's call the space of possible entity values C.
A little more precisely, let Aut[M] be the space of continuous invertible homeomor­
phisms of M to itself. (Aut[M] is a group under composition.) Let n be a map that
assigns an element in C when evaluated on the entire manifold M. (For example, M
could be space-time, and n could be total spin of the universe .) Then n is an invari­
ant if and only if
'iI/J E Aut[M] , (6.8)
n[M] = n[I/J[M]] . (6.9)
To be more precise, we can define more explicitly how n is evaluated on M. For
example, if M is a smooth manifold, and C is, say, the complex number field, we can
define a notion of integration on M, and " sum" a function f defined pOintwise on M
as a map f M � C, by integrating f over the manifold M with respect to its area
element. (Note that the set of reals and the set of complex numbers are groups, so we
retain the ordinary notion of a quantity as we11.)23
We will return to this in our discussion of continuous approach to objects of experi­
ence following Alain Connes and Jean Petitot.

I m pl ications and Applications

These concepts may seem synchronic, devoid of temporality, but, to borrow a figure
from matter, they adsorb the temporal qualities of experience, rather than demand an
a priori abstraction of a dimension of time. In this way we can articulate experience
and matter as saturated with time, with implications for what we mean by a dynamical
(temporal) system.
Perhaps even more primordially, these concepts may help articulate a mode of
experimental, material dynamic phenomenology that is not anthropocentric. An
object" in a topological manifold M is that which is invariant under the action ofa subgroup
1/

of a Lie group of continuous actions on M. This automatically encodes the variational

character of Petitot's notion of a concept. To encode the relational aspect, the phe­
nomenological attitude recognizes that every act of perception is rooted with respect
not only to the condition of the object but also to that of the perceiver. In fact, con­
scious experience is relational, not merely the reception of sense data from some
Topology, Man ifolds, Dynamical Systems, Measure, and B u n d les 1 91

thing-in-itself external to the perceiver. From this complex relational concept of per­
ception, I would cast off two implicit assumptions: ( 1 ) that there is a monadic observer,
and (2) that the perception is visualist.24

Exercise: A Viconian Model of Tim e

Before getting t o the heart of the matter, a s an exercise let's work through a Viconian
model of spiral time.
Covering space provided a way to metaphorically treat time in the responsive media
spaces called TGarden that I designed for collective improvisatory movement. In the
space-time diagram of a TGarden event, each life trajectory becomes a path not just
in the physical "base" space, M, but in the product space M x I, where I [0, T) and
=

T is the time extent of the event. Therefore a loop L in the physical space, M, becomes
a spiral in M x T that projects down onto 1 . What seems recurrent on the base mani­
fold can be regarded as the projection of an unbounded curve in the covering space.
So, what may seem to be a recurrent traj ectory-a closed curve-g in one manifold
(say the base manifold in our example of 51 covered by spiral R1) may be realized as
the projection under p of an endless, nonclosed curve in the covering space. That path
is also called a lift of g.
However evocative such generalizations of metaphors for time may be, I believe
that most questions of time seem intractable because they consider time abstracted
from the magmatiC plenum processes. This difficulty, Augustine's famous slipperiness,
suggests that the naming of this contingent aspect of experience as the "dimension
of time" commits a reification error. Time as a unidimensional quantity does not exist
in itself. This error does not depend on whether time is conceived as linear, multilin­
ear, or nonlinear, whatever that means. It's the very attempt to tease time out of
experience as if it were a thread from the muddy tissue of life that is the confusion.
In an appendix to the second essay of Autopoiesis and Cognition, Maturana and
Varela argue that time, as a concept, is a part of an organism's linguistic description
of its experience, rather than something one can find materially in the organism
itself.25 Furnished with that deep insight, what if we take a more generative notion of
space-time-what if we take the extent of an occasion (to borrow Whitehead's term)
as an effect of the exfoliation of processes in the world? This necessarily includes the
organisms that co articulate an event with its dynamical, material substrate.

Whitehead's Eternal Objects

In Process and Reality, Whitehead's ontology contains four types of objects (entities) :
actual occasions ("objectified"), eternal objects, propositions, and nexus.26 "Any entity
whose conceptual recognition does not involve a necessary reference to any definite
actual entities of the temporal world is called an 'eternal object . 1 II27 There is a
graded degree of relevance of an eternal obj ect with respect to actual entities, via what
1 92 C h a pter 6

Whitehead calls ingression, the mode in which the potentiality of an eternal object
is realized in a particular actual entity.28 "Eternal objects in any one of their modes of
subjective ingression are then functioning in the guise of subjective novelty meeting
the obj ective datum from the past. This word 'feeling' is a mere technical term. "29
Whitehead elaborates this in terms of a topology of the causal manifold:
The percipient prehends the nexus of contemporary occasions by the mediation of eternal objects
which it inherits from its own past. Also it selects the contemporary thus prehended by the
efficacy of strains whose focal regions are important elements in the past of those nexus.30

In other words, an eternal object can be articulated as a temporal-causal accumulant,

or a limit of a set of actual occasions. Given Whitehead's treatment of causal "cones"
in the generalized-nonmetric-topology (reminiscent of Hawking and Ellis's topo­
logical treatment of the large-scale structure of space-time), and given the inadequacy
of sequences of discrete objects for a continuum experience and continuous matter
(the subject of this book), it is appropriate to employ our topological limit, or a notion
of point of accumulation, rather than a sequential limit.31 Topological limit furnishes
us with a consequentialist way to understand Whitehead's eternal object without telos
or unidimensional seriality.

The Smooth and the Striated

Gilles Deleuze and Felix Guattari's capstone chapter in A Thousand Plateaus plays the
constant dynamic between the smooth and the striated in several models (to use their
terminology) : fabric, war, music, and mathematics. They credit Pierre Boulez with the
particular terms, which he applied in music: " smooth space-time one occupies without
counting, whereas in a striated space-time one counts in order to occupy. " This recalls
the most primordial distinction between number and topology or geometry. One limit
case of an infinitesimal, microtonal approach to sound is the tone cluster, a point set
which is an arbitrary collection of points in the set of all tones, without any regard
to the logos of key, scale, octave. But we get ahead of ourselves. Let's transversally tour
Deleuze and Guattari's chapter on the smooth and the striated, starting with espace
strM (striated space), passing to espace lisse (smooth space), and ending with a comment
on espace troue (what Massumi translated as "holey" space) .32
The basic model for the striated space is the vector space Rn , where a fiber would
be a copy of the real line. (Again, here "real" refers to the set of numbers forming the
continuum, and is not an ontological adjective.) In the Cartesian model of analytic
geometry, the line, the plane, and in fact any higher-dimensional Euclidean space are
given coordinate axes, with respect to which any point can be assigned a unique
ordered tuple of real numbers: "numbered numbers. "33 Here the word "number" is
better thought of as a point in Rn , with coordinates of the form <X l! X , . . . , Xn>, where
2
each Xi is a real number. In Deleuze and Guattari's terms, "the striated or metric is an
Topology, M a n ifolds, Dynamical System s, Measure, and B u n d l e s 1 93

,,
aggregate with a whole number of dimension, 34 which recalls to mathematicians a
vector space, described earlier in this chapter.
Deleuze and Guattari point to the warp and weft of woven textiles as the techno­
logical model of the striated, which immediately enriches the discussion by using a
soft rather than rigid model for striation, suggesting that the dynamics of material use
and wear at individual, social, and historical scales transforms regularly arrayed woven
cloth into unordered felt and scrap.
To striate is to impose the structure of a vector space,3 S or, in more precise terms,
to define a fiber bundle over a smooth manifold. There is a stronger sense of striation,
however, that of logos, with the power to reify by naming. Looking to music, "the
striated always has a logos, the octave for example, " borrowing from Boulez.36 Looking
to science, Deleuze and Guattari see also the dynamical relation between the logos of
major science and minor science: "Maj or science has a perpetual need for the inspira­
tion of minor science; but the minor would be nothing if it did not confront and
conform to the highest scientific achievements. "
I n a striated physics, "forms organize matter, " whereas " i n the smooth materials
signal forces and serve as symptoms for them. "37 For example, refinements (or con­
cretizations, to use a Simondonian notion) of the automating machines that process
building materials, such as asphalt, concrete, steel, or glass, have progressed in tandem
with refinements of the material that those machines process.38 But the very refine­
ment itself has led to demands for more and more homogeneous material, material
without any heterogeneities or nonuniformities that would impede the machine's
form-based operation or even break it.
This line of thought is akin to Rene Thom's late work Semio Physics (Esquisse d'une
semiophysique), in which he gives a fertile and idiosyncratic interpretation of Aristotle's
Physics. After arguing for the continuous as a precondition for experiencing any phe­
nomenon via discontinuity, Thom defines his fundamental concept of salient form:
The primary experience in any receiving of phenomena is discontinuity. But discontinuity pre­
supposes the continuous . As our first experience of the continuous is that of consciousness, i.e.
that of time, the most original auditive discontinuity will be, for example, the eruption of a
sound in the midst of silence. The tinkle of a bell is perceived as an autonomous fbrm, filling
the gap between two empty zones of silence. I shall call salient form any experienced form clearly
separate from the continuous background against which it stands out [emphasis added) .:19

Thom defines a particular subset of salient forms, called pregnant forms, pertinent to life:
[Some) forms . . . carry a biological significance for the animal . . . concerned. Among these are
the forms of prey for the (hungry) predator, of the predator for its prey, of a sexual partner at
the appropriate time. The recognition of these forms gives rise to a very ample reaction in the
subject: the freeing of hormones, emotive excitement, and behavior designed to attract or repulse
the inductive form. I will call such forms pregnant, and this specific character of theirs pregnance.40
1 94 C h a pter 6

Inhomogeneity implies a non exchangeability of matter or material features. In Deleuze

and Guattari's musical example, "two sounds of equal pitch and different intensity
,,
cannot be compared to two sounds of equal intensity and different pitch. 41 Physicists
capture this materiality in what they call dimensional analysis. A numbered measure
of a physical quantity is not just a mathematical real number or integer, but a number
with a "unit"-it's not "25 sugar, " but "25 grams of sugar, " not just "acceleration of
32" but "acceleration of 32 feet per second per second. " So in that sense a difference
of say 1 00 hertz in pitch cannot be exchanged with a difference of 1 00 decibels in
intensity. (As an aside, dimensional analysis does not directly account for a rich
aspect of materiality: how things weather-how material things and we as material
beings toughen or soften, loosen or compact, fray, shine, and endure the world in
myriad ways.)
Bergson's duration is a different sort of multiplicity. It is not indivisible but, in
Deleuze and Guattari's words, "cannot be divided without changing in nature at each
subdivision (Achilles' running is not divided into steps, his steps do not compose it
,,
in the manner of magnitudes) . 42 This nonfactorability seems to be an essential quality
of the smooth. Earlier, Deleuze and Guattari write: "unlike magnitudes, [pitch and
loudness] cannot divide without changing in nature each time. " While these are good
examples, more primordial examples may clarify what is in play. Take the integer 6,
and partition it into 6 2 + 4 3 + 3 1 + 2 + 3. In each partition, the parts are also
= = =

integers. Take Achilles running toward the finish line arriving at three spots along the
path: A, then B, then C. His path ABC is not the same as the backward path CBA, or
the acrobatic CAB. Bergson's duration essentially changes when broken into pieces.
The same is even more evident in higher dimensions. Take a circular ceramic dish.
Break it into a few pieces. Each piece is no longer a circular dish.
Deleuze and Guattari write of the irregular scraps of cloth that quilters stitch into
a whole. Stitching nonsimilar shapes yields nonflat, lumpy blankets. In fact, this
recalls a standard approach to defining a Riemannian manifold as a collection of
coordinate patches, each with its diffeomorphism mapping the patch to a neighbor­
hood of the origin in the model space Rn (as described earlier in this chapter) . They
quote Lautman:
Chaque voisinage est donc comme un petit bout d'espace euclidien, mais Ie raccordement d'un
voisinage au voisinage suivant n'est pas defini et peut se faire d'une infinite de manieres . L'espace
de Riemann Ie plus general se presente ainsi comme une collection amorphe de morceaux jux­
taposes sans etre rattaches les uns aux autres,43

then continue in their own words:

et il est possible de definir cette multiplicite independamment de toute reference it une metrique,
par des conditions de frequence ou plutot d'accumulation valant pour un ensemble de voisinages,
conditions tout it fait distinctes de celles qUi determinent les espaces metriques et leurs coupures
Topology, M a n ifolds, Dynamical System s, Measure, and B u n d l e s 1 95

(meme si un rapport entre les deux sortes d'espaces doit en decouler) . Bref, si l'on suit cette tres
belle description de Lautman, l'espace riemanien est un pur r.44

The English terms that more closely convey the most salient senses of the key terms
in these pages would be set theoretic union rather than accumulation for accumulation,
topological neighborhood rather than vicinity for voisinage, and, most centrally, Rie­
mannian manifold (in German, Mannigfaltigkeit) rather than multiplicity for multi­
plicite. This is key to understanding more rigorously yet anexactly Deleuze and
Guattari's adaptation of what in differential geometry is standardly called an atlas of
charts.45 The neighborhoods or coordinate patches, as they are called in English, are
linked in uncountably many ways, but not arbitrarily at all. In fact, the nonflat geom­
etry of an n-dimensional manifold is constituted by which of these neighborhoods
are linked with one another, and by their linking transition functions (as described
earlier) . Deleuze and Guattari write that these transition functions are built out of the
coordinate patches' diffeomorphisms into neighborhoods in R". While it is true that
the transition functions may in general vary by an infinite number of what amount
to reparametrizations-like changing from meters to kilometers, or from polar to
rectilinear coordinates-those variations are not arbitrary: they have to be differen­
tiable (smooth) so that the resulting manifold has a smooth structure, and they have
to satisfy basic compatibility conditions (described earlier), analogous to the material
consistency of stitching scraps of cloth to form a continuous surface.
Moreover there is a subtle but quite important difference in their stance with
respect to metric. Deleuze and Guattari say that it is possible to define the manifold
independently of any reference to a metric. However, a Riemannian manifold indeed
constitutes a metric, and in fact its differentiable-smooth-properties are indeed all
derivable from its metric. (The presence of metric is what distinguishes the category
of differentiable manifolds-the smooth-from the more primordial topological
spaces.) Actually, the Euclidean space, with its zero (or "flat") curvature, is just a special
case of general Riemannian geometry. The profound difference between Riemannian
geometry and the classical geometry of Euclid is that the Riemannian manifold's
metric is a consequence of the mathematician's art, a human experimental construc­
tion rather than a transcendentally warranted axiom. (Under this interpretation, one
could consider how a metric could be an eternal object in Whitehead's theory.)
Deleuze and Guattari claim two positive characteristics of smooth spaces in general:
when there are [dimensions] . . . that are part of one another and pertain to enveloped distances
or ordered differences, independent of magnitude; when, independent of metrics, dimensions
arise that cannot be part of one another but are connected by processes of frequency or
accumulation.46

Here I replace "determination" by the more mathematically motivated term dimension

to see what alternative traction we can gain on this difficult passage. I interpret the
1 96 C h a pter 6

first characteristic of smooth space as expressing the fundamental dimensionality of

an n-manifold, primordial to magnitude and coordinates. The second refracts how the
Riemannian metric can always be expressed in terms of the fundamental coordinate
I -forms dX i' the differentials of the coordinates weighted by coefficient functions gii :
g == L gij dx i /\ dx i . (6. 1 0)
i,j
One can interpret these coefficient functions as densities, weights, or, in Deleuze and
Guattari's terms, accumulations. If we accept the interpretations, we harvest an insight:
the symmetry of these coefficients, gij gji, infinitesimally encodes the idea that extent
=

of a path from A to B is the same as the distance of that same path traced from B to A.
More so than perhaps some of their readers, Deleuze and Guattari recognized the
inextricably intimate relation between the smooth and the striated that is modeled
by a Riemannian manifold and its bundle of tangents: {TpM: p E M} :
the means b y which Riemannian patches o f smooth space receive a Euclidean conjunction (the
role of the parallelism of vectors in striating the infinitesimal) Y

Indeed, "conjunction" here can be well expressed as the assignment of the tangent
space TpM to each point p E M. Recalling the earlier exposition on Riemannian dif­
ferential geometry, the tangent space TpM is the Euclidean vector space of the same
dimension as M that represents the "best" approximation to M at that point p in a
sense that can be made robust. That tangent space TpM can also be defined as the
space of all differentials defined by all the curves through the pOint p. So it's not so
much "parallelism" but the vector space's structure to which Deleuze and Guattari
gesture. To be preCise, in place of "parallelism" we should say additivity: vectors can
be added to one another, whereas points can only be accumulated into point sets.
Elements in a general smooth Riemannian manifold have no algebraic relation such
as addition among them, which gives them a fundamental freedom from the striating
structure of vector space. This is the precise insight behind Deleuze and Guattari's
comment:
The mode of connection proper to patches of Riemannian space ("accumulation" [set union] ) is
not to be confused with the Euclidean conjunction [vector addition] of Riemann space.48

Visiting for a moment the technological: in the wake of the World Wide Web, we
seem to have resigned ourselves to explicitly or implicitly filling in form after form
with almost as little thought as turning a knob. But whereas the knob affords an infin­
ity of nuance, web forms striate our action by vector space models of identity, called
databases. Scaffolding by data can only enforce consistency as a substitute for truth,
and can never in prinCiple substitute for the dynamics of corporeal, affective, material
smooth spaces. Databases amplified by the Internet mediate some of the most coercive
structures of governmentality. However, this scaffolding also affords handholds for the
Topology, M a n ifolds, Dyna m ical Systems, Measu re, and B u n d les 1 97

construction of fictive identities and whole fictive worlds, exemplified by the prolifera­
tion of role-playing games and ad hoc identity fictions.
Limitations on space and time preclude Deleuze and Guattari's important and
strategic references to espace troue, translated as "holey" space in the English edition.
From the images of the Sierpinski cube and other self-similar subsets of Rn , and from
their references to the "fractal, " Deleuze and Guattari clearly refer to the beginnings
of the domain of measure theory. However, here we reach the limit of this book's
mathematics and leave further exploration for another day. Suffice it to say that a first
step would be to eliminate the conceptual shackle of referring only to the model of
" self-similar sets" and move to the far more ample context of measure-theoretic exten­
sions of geometry without any requirements on (finite, computable) discrete, or con­
tinuous, or differentiable structure.

A Nonanthropocentric Approach to Apperception

As a student of Rene Thom, Jean Petitot inherited the francophone fearless use of
mathematics as substantive rather than technical or merely illustrative material for
philosophical investigation. On one hand, Petitot offers a version of ontogenesis that
shares with Whitehead an appeal to the prinCiple of least action,49 but also introduces
a mathematically more sophisticated and arguably more open-ended formulation
based on continua and continuous concepts. On the other hand, Petitot has inspired
some contemporary French engineers in computer vision and machine learning to
build robotic systems to experimentally test such approaches.
Petitot uses continuity fundamentally to derive the possibility of quality as exten­
sion, a plausible way to understand qualities that ride above the unboundedly many
particular predicates attached to particular obj ects. If we treat matter as continuous
substance, then it follows that the sorts of transformations we must consider acting
upon matter are not combinatorial permutations (more generally algebraic transfor­
mations) but continuous topological transformations.
Petitot appeals to the concept of a fiber bundle to accommodate the continuous
summing of perspectival " adumbrations" to a single quality. Regarding qualities, he
perspicaciously observes that
a "simple" (i.e. constant, uniform) color such as "red" is a common quality shared by all red
things. But even if traditional, this naive extensional point of view is not convincing. First it
takes for granted the "atomist" nominalist axiom of the primacy of individuated things. Now,
things result from highly complex noetico-noematic processes of constitution and are by no
means primitive data. Second, it does not take into account the fact that, in a covering, quality
can vary continuously [emphasis added] .so

Continuity and the axiom of compatibility of transition functions are essential to this
model to arrive at the existence of locally constant section.51 Now, this way of think­
ing about qualities leads us to think about how they are perceived as a continuous
1 98 C h a pter 6

fu nction of varying perspective taken in a general, not just an oculocentric way. To my

mind, one of the most profound contributions of Petitot's essay is a clear model for
the Husserlian process of summing evidence (aletheia) in adumbration to a quality
which is necessarily aperspectival. Turning this piece of fruit in my hand, I say that
this orange is round, not that it is round from this or that point of view. In Petitot's
framework of fiber bundles, observing is selecting a particular value-the observed
quality or value-in the fiber over a particular neighborhood around a particular base
pOint. In the topology of fiber bundles, neighborhood is a technical concept precisely
articulating what we colloquially call a point of view. In this framework, a particular
continuously coherent set of such observations can be well characterized as a continu­
ous section of the bundle. Resorting to such a formulation provides a way to conceive
qualities in ways that do not privilege a particular sense modality, and yet retain the
phenomenologically relevant qualities of embodied experience. We need a way to not
only conceive of simply sensed qualities of a piece of fruit but also qualities like those
of a person's "character. " At a macroscopic scale, I could say that this person is quite
imaginative from the professional point of view of a computer graphics engineer, or
I could say that this same person is quite imaginative from the point of view of a
radiologist, or a sibling, and so forth. As I change my point of view, I am essentially
moving around from neighborhood to neighborhood in the base manifold of this
fiber bundle .
Returning t o the visual, one o f computer vision's basic problems is t o infer the
objects in a scene based only on the colored pixels presented in a video image from
one or more cameras trained on that scene. Right away we see an old dualism rein­
scribed in computer vision algorithms, a dualism that assumes that there is an a priori
obj ect " out there" waiting to be inferred from the camera data. However, the notorious
correspondence problem in computer vision demonstrates very precisely a fundamen­
tal limit of this assumption: without extra information, a computer cannot tell whether
a particular spot of red in one image corresponds to the same point of a putative object
in a scene as does a spot of red in a second image of the same scene. Intricate (if
sloppy) proj ective geometry yields some constraints (to projective lines) that reduce
the correspondence problem to requiring that only a small number of pixels in the
two bitmaps be identified as coming from a common ancestor "out there." But there
is no way to altogether eliminate the indeterminacy except by the intervention of an
observer, or more precisely, thanks to C. S. Peirce, an interpretant.
So what if we assume no a priori object? What if we do not accept the naive
form of Platonism, and consider the emergence of obj ects in continuous fields of
camera data? Aside from the intrinsic interest of discovering mathematical alterna­
tives to Platonist computer vision, one of the strongest philosophical motivations
for studying models of machine vision is to be able to construct alternative working
models of ontogenesis that do not reduce to predominant conceits about biology
Topology, M a nifolds, Dyna m ical System s, M easure, and B u n d l e s 1 99

or ego-a speculative experimental phenomenology, decentered from historical

anthropocentrism.

Computer Vision at the Sony Computer Science laboratory

When I visited the Sony Computer Science Lab in Paris in 2004, I was most intrigued
to discover a group of engineers who were applying notions from differential geometry
such as Riemannian manifolds and Lie groups to help a Sony (ElBa) robot dog learn
what obstacles are in its field of view. (The robot dog was equipped with a pair of
video cameras for eyes and a set of motors that allowed it turn its head and to move
around the floor.) The computational Scientists, David Philipona (Sony Computer
Systems Laboratory, Paris), Kevin O'Regan (Laboratoire de Psychologie de la Percep­
tion, Universite Rene Descartes, Paris 5), Jean-Pierre Nadal (Laboratoire de Physique
Statistique, Ecole Normale Superieure), and Olivier J.-M. D. Coenen (Sony Computer
Systems Laboratory, Paris), were trying to implement in software and robotic hardware
a crude version of a most subtle idea: that objects of consciousness are invariants of
embodied action and sensing in the world.
Recall that an invariant of a rigid motion is something that stays the same under
the impact (the action) of that motion. For example, if I spin a ball around its axis,
only two pOints on its surface stay fixed under the set of all rotations about the fixed
axis: the north and south poles defined by that axis. An invariant does not have to
be an isolated point, however; if we consider arbitrary subsets of the sphere as putative
obj ects, we could say that each latitude circle about a fixed axis is an invariant of the
set of rotations about that axis. Much more generally, the zeros of a vector field can
be considered the invariants of the flow induced by that vector field. It is crucial to
understand that such vector fields and their substrate manifolds need not be at all
visual or "geometric" in the crude sense that can be drawn on a computer screen.
They may not even be finite-dimensional.
A basic problem in machine perception is to interpret the data transmitted by the
cameras to a software program inside the robot with minimal, ideally zero, a priori
encoded assumptions about the robot's body and the physical/spatial nature of its
ambient. The program can emit signals to the robot's motors that cause clianges in
its field of view which are transmitted back from its cameras. However, the program
initially may not even have information about which of the signals control elements
of its own body and which control elements in its ambient (such as lamps, or objects
illuminated by those lamps), because it should not have a "model" that preencodes a
god's-eye view that already distinguishes between internal and external structure. How
can the robot program infer basic information about the objects in its ambient envi­
ronment based only on its camera data? Philipona, O'Regan, Nadal, and Coenen start
with a few observations: ( 1 ) determinism: repetition of motor signal P yields repetition
200 Chapter 6

of effect on the image K; (2) composability: if P causes a change in image from Image 1
to Image2, and Q causes a change in image from Image2 to Image3, then the composi­
tion of P followed by Q yields a change in the image from Image 1 to Image3; (3) path
continuity: small changes in image can be induced by small changes in the motor
signals; (4) invertibility: for every continuous motor movement P that transforms the
input camera image from Image 1 to Image2, there is another continuous motor move­
ment Q that returns the image back to Image 1 . Q is called the inverse of P. These seem
like very minimal assumptions, and do not encode basic information such as which
signals control a putative "body" and which a putative "ambient." Certainly there is
no encoding about even such spatially fundamental notions as the dimensionality of
the putative ambient environment. But even at this level of generality, we already
have a rich mathematical theory available because conditions (1) through (4) define
what is called a continuous group, synonymous with a Lie group.
Lifting our attention from manifolds to dynamics on manifolds, from things and
stuff to the transformations of things, Lie groups provide a most powerfully expressive
articulation of continuous action. One of the most powerful facts about a Lie group
is that it is both an algebraic group-a technical concept in the mathematical theory
of abstract algebra-and a manifold-a technical concept in Riemannian geometry
(the morphological language of Einstein's general relativity) . It may seem quite hard
to discover any structure in a continuous group at this level of generality, but Lie
theory provides some deep theorems about the existence of substructures in general
Lie groups, and the conditions under which a Lie group admits at least local represen­
tation by matrix groups, where "local" has a precise interpretation in terms of dif­
ferential geometry. This in turn raises the possibility of a machine representation in
software. Of course there is a significant amount of applied mathematics and engineer­
ing to be done, which is the content of these engineers' adaptation of the Lie group
and differentiable manifold formulation inspired by Petitot's approach to perception
and phenomenology. At one key step, to infer information about the ambient envi­
ronment, Philipona et al. 's algorithm uses a dimension-reducing technique called
principal component analysis from engineering, which invokes a version of the prin­
ciple of least action. It is a moment at which the principle of least action, the funda­
mental ontogenetic principle for modern physics as well as for Whitehead and Leibniz,
is encoded into an operational metaprocedure. Is this all? Is this enough? It seems
that Deleuze and Guattari would respond in the negative. But relying on Whitehead­
ian concrescence (as we describe it in chapter 5) also relies inevitably on the principle
of least action. So we have arrived at a conundrum, but a fertile one, I believe.
What are the philosophical implications of proposing to regard obj ects as invariants
of continuous action on a continuous manifold? Let me underline that in no way do
I suggest that we reduce experience to a mathematical model. Nor do I agree with
Petitot's project to naturalize Husserl by confiating mathematics with a computational
Topology, M a n ifolds, Dynamical Systems, M e a s u re, and B u n d l es 20 1

model. Petitot outlines the essential features of the program to find a naturalized
phenomenology:
The key point is the mathematical schematization of phenomenological descriptive eidetics, that
is, the elaboration of a mathematical descriptive eidetics (which Husser! thought impossible in
principle) . For us, to naturalize an eidetics consists in implementing its mathematical schematiza­
tion in natural substrates (e.g., neural nets) .S2

To parenthetically slip in neural nets plays loosely with a very large reduction in kind
from natural substrates of ordinary matter to computational abstraction. My hesitation
dwells more on the reduction than the abstraction. Moreover, there is a large philo­
sophical gulf between recognizing and implementing a mathematical schematization
in a natural substrate. To recognize is to perceive and understand a pattern without
implying that this pattern exhausts or completely fills out the entity in which it is
perceived. The pattern may not even be physically present, even if it arises from physi­
cal morphology-think of shadows. However, Petitot calls for creating an operational
(what Guattari might call a machinic) articulation in matter of the mathematical
schematization. Perhaps the closest classical analog to this would be the realization
of constructive geometry by carrying out geometric procedures in physical material,
such as cutting out an elliptical boundary by swinging a loop of string around two
fixed pegs. This cuts across several ontological strata: consciousness of pattern, math­
ematical pattern, and physical material.
More fundamentally, stated so baldly Petitot's project seems implausibly reduction­
ist unless accompanied by the materiality of mathematical entities, which underwrites
this chapter and book. In short, my quibble with Petitot's formulation of his project
lies not with the deployment of mathematical concepts, but with the parenthesis " (e.g.
neural nets) . " Neural nets are software models, implementing the optimization of a
linked set of differential equations derived by taking the gradient of a system of sig­
moidal (nonlinear) functions. This is a reductive process, in the sense that it does not
register history, affect, or more classically culture and eros, so much as elide them.
Instead, we can deploy these mathematical concepts like the others, as modes of mate­
rial articulation.

The Case for Continua

Exploring the implications of a topological approach to a plenist, unbifurcated ontol­

ogy, I am dealing with the problem of how things emerge and dissolve with respect
to their background. I am using "thing" mindful of four notions. The first is Bruno
Latour's and science studies' notion of things, such as controversies that have left the
lab and have entered public discourse. This is not unrelated to Heidegger's "thing"
performing, gathering the fourfold: earth and sky, divinities and mortals. A third idea
202 Chapter 6

is computer science / machine perception's notion of an obj ect that can be "inferred"
from sensor data. The most evocative sense of "thing" for the purposes of this book
is Alain Connes's proposal that obj ects of consciousness are invariants in a mathemati­
cally interpretable sense. Connes's proposal returns us to Petitot's phenomenological
project, which I interpret as a concretization of Connes's speculation and Rene Thorn's
large project on the topological dynamical study of ontogenesis.
The essence of Heidegger's vase is its being a prop in a performance that gathers
the divine and mundane world together.
Petitot's use of fiber bundles to articulate qualities as extensions, following Husserl,
seems fertile but needs a sustained and well-informed effort to animate it by a more
dynamical, process-oriented story. The key here is to start with Petitot's and Husserl's
speculative proposition that a predicate's extension depends crucially on continuous
variation, not on sequences of discrete points. But to be clear, I would set aside Petitot's
concern with epistemology, and the checking of the theory of experience against a
computational model.
So what, in sum, have we encountered from this double-stranded passage through
computer vision and Petitot's recasting of Husserlian phenomenology? We have the
possibility of a radically decentered, de anthropomorphized concept of phenomenol­
ogy. This removes an enduring reservation concerning Heidegger's phenomenology,
but retains the larger concern with the condensation of subjectivity in the exfoliation
of the experiential world. We have a non-ego-based, number-free, and metric-free
account of experience that respects evidence of continuous lived experience but does
not reduce to sense perception or ego-centered experience. And we have the essential
feature of continuity both as a quality of lived experience and as a mode of descrip­
tion of such experience. Given that, it is hard to exaggerate the radical significance
of fiber bundles as a continuous mode of articulation, reducible neither to alphabetic
language nor to number, and free of the a priori conceit of a Cartesian perceiving
agent. S 3 We have here the seed of an approach to poiesis and expressive experience
that may avoid recourse to stochastic methods, statistics, and informatic sweepings of
ignorance under the rug, and is "nonclassical" in the senses of both quantum mechan­
ics and measure theory.
7 Practices: Apparatus and Atelier

This chapter is concerned with the "hows"-some of the practical and instrumental
matters involved in the kind of artistic or critical work informing or inspired by the
previous chapters. How would you build an apparatus that could even begin to respect
a nonanthropocentric conception of living matter, and yet condition a built environ­
ment that could sustain ethico-aesthetic experiment? How would you do that at the
level of engineering? What sort of organization in what sort of institutional ecology,
and what ecology of practices, would be adequate?
We start with a specific technical application of some of the mathematics, as a way
to build an apparatus for playful improvisation of individually and collectively charged
movement and gesture. This is the sensing and media synthesis system built with
elements of the responsive media installation-event examples populating chapter 3
and other parts of this book. Then we enlarge the scope to the institutional organiza­
tion and ecology of practices that can house the making and evolution of such experi­
mental research and creation.

Ozone, a Media Choreography System

In this section, I give a technical description of a responsive media software framework

that is informed by (1) observing how composers compose a potential event by sketch­
ing metaphorically, and (2) the continuous classical mechanics of fields and motion
actualizing potential fields of metaphor.
This "media choreography" system, called Ozone, is designed to support dense,
continuous play. Each of these terms has a sense taken up in the earlier chapters of
this book. By play I mean unschematized, improvisatory but not random activity
(from the point of view of the performer) . These qualities can be applied not only to
the millisecond scale of producing real-time media in concert with inhabitants' action,
but also to technical ensembles for gesture (d. Doug Van Nort's work), composition
of event (Ozone), and perhaps coding, and to a social nexus in a field of institutions,
such as the Topological Media Lab.
204 Chapter 7

Figure 7. 1
Life in the Topological Media Lab atelier-laboratory.

When we creators gathered to compose the "look and feel" for the Frankenstein's
Ghosts experimental performance or the TGarden play space installation-event, it was
not at all clear how we should go about shaping the environment, especially in a col­
lective circle of peers. We set aside structuring schema such as scripts, scores, and
soundtracks. In general, we cannot assume that the visitors can play or reperform any
musical patterns with any skill. Using a multilinear track-based logic puts a cage on
potential action and leaves little room for improvised expression and surprise. Having
designers make up stories about what a potential visitor would experience in a respon­
sive environment is not so helpful beyond a certain pOint, because every single visitor's
narrative, however skillfully imagined, is just a particular one-dimensional trajectory
through an infinite-dimensional space of intensities and fields. No finite set of these
narrated trajectories could ever add up to a thick description of the environment that
could guide its deSign, much less its construction. But a fictive physics articulated in
the spirit of equations of state could suggest potential dynamics as microcosmologies.
We can breathe life into an imagined cosmology by describing not its exact progres­
sion but its tendencies, a potential field of possible states through which an event
could evolve relative to a set of affective or metaphorical states. Composers can sketch
in their own terms how the event should feel upon entry or exit; some moments of
intenSity or repose, or mystery, or ambivalence and multiplicity; what states could
overlap or blend with what other states; what are one state's tendencies toward evolv­
ing into other states; what sort of physically observable conditions or activities are
associated with a given state; and so forth. This way, designers can fluidly imagine
and revise how the responsive environment could evolve not just from the point of
view of a particular inhabitant's trajectory, but for any and all of the inhabitants in
any condition (see figure 7 .2) .
How can we marshal all the concurrently running and dense media processes (not
prefabricated sound or visual objects!) in concert with both prior aesthetic intent and
contingent ethico-aesthetic action? Since the media textures may be very finely crafted
according to musical and visual design, the system should support the control of richly
structured transformations rather than just "random" sequences of cued media. At the
P ractices: Apparatus and Atelier 20S

F i g u re 7.2
Designers' metaphorical sketch of a field of potential events, produced during a design session
at Banff New Media Institute, 2000. Image courtesy of Sponge.

same time, the dynamics need to be tightly coupled to the gestures and movement
of the players in the room, as well as to the internal state of the system. Traditional
timeline-based narrative sCripts and database representations are too sparse or too
complicated to support the dynamical evolution of fully dense media. Repr� entations
in conventional procedural languages are too low-level to efficiently captureJhe rich,
multivalent or inconsistent semantics of the artists' metaphorical designs. We can
pitch the system's representational structure at an intermediate level between the low­
level response to sensor data plus their statistical derivatives, and the high-level
semantics of the artists' metaphorical talk.
A media choreography system is a set of software (and hardware) frameworks that
extracts features from sensor data tracing what is going on in a physical space, and
creates or modulates ambient media (video, sound, lighting, kinetic material, or
objects) in real time, concurrently with that activity. The mapping from activity to
206 C h a pter 7

media dynamics can follow one of many strategies: (1) fixed timeline, such as cue­
based systems, which impose a fixed, predesigned sequential order on events that can
only be repeated; (2) if-then logic, which tends to be brittle and hard to revise on the
fly; (3) stochastic methods, which provide a degree of unpredictable variation, but
with no ethico-aesthetic sense to the distribution-or rather a very particular one: that
of the random; (4) written scripts, which permit a great deal of interpretation by
persons inhabiting the event, but not by the co structuring software or material. It
should accommodate all the contingent activities and conditions in the environment,
whether due to people, the media, or the environment, as well as any prior intent on
the part of the composers or designers of the environment.
I describe a particular media choreography software system called Ozone, imple­
mented by the Topological Media Lab via five generations of development from the
prototype TGarden system to the present. Ozone has the key feature that the pseudo­
physics and high-level evolution of the climate of the environment can be molded by
means of a representation that is legible to dramaturgical and aesthetic composers
who do not think like programmers. Given that the experience of a responsive envi­
ronment evolves qualitatively like a dynamical system, a composer's expressive design
for such a responsive space could amount to specifying the environment's pseudo­
physics. This way of thinking is quite different from brittle procedural, Boolean, or
data-based programming logic.
A subtle difference between an information-theoretic approach to scripting the
behavior of a system and the Ozone media choreography system's quasi-physical
approach is that the latter bets on a radically modest approach to computational media
as dumb matter. By dumb I mean (1) free of language, even the formal procedural
programming languages that are operationalizations of the logic that I relinquished
early in our experimental work; (2) free of intelligence, as in the cognitivist approaches
of symbolic artificial intelligence; and also (3) free of representations of abstract struc­
tures like hidden Markov statistical models or 3D polyhedral geometry.
A key common feature of the media choreography of this family of play spaces is
that the creators specify not a fixed, discrete set or sequence of media triggered by
discrete visitor/player actions, but rather a potential range-a field-of possible
responses to continuous ranges of player actions. But in this family, behavioral tempers
(or, to use less animistic terms, climates of response) evolve over macroscopic periods
of time (minutes), according to the history of continuous player activity.
The multivalence can be articulated as multiplicity of state. For example, a player
could be described as a soloist or as part of a group. But rather than simply flip
between two discrete player states like Solo and Group, it makes expressive sense to
generalize this aspect of a player to a continuous range of "groupness" between extre­
mal Solo and Group states. One advantage of our physics-based approach is that it
P ractices: Apparatus and Atelier 207

provides a level of abstraction between the sensors and the rest of the system. The
composers can get on with the shaping of experience without freezing on a particular
set of physical conditions in some brittle, intricate code that would have to be ripped
out and rewritten should they change their minds. For example, once groupness is
defined as a concept based on the two metaphorical components Solo and Group,
the composers can begin using it in the media synthesis engine even as the detailed
choice of sensing modality and feature extraction method mutates in the course of
the engineering design and development. A player's groupness could be indexed to
the distance from that player to the barycenter of all the players in the room, or it
could be indexed to a measure of synchrony between that player's accelerometer data
with the other players' accelerometer data. How groupness is related to physical action
or configuration can be decoupled from the software or hardware implementation, and varied
in the rehearsal process. The final choice can be made as the compositional and
rehearsal processes interweave with the engineering production of a responsive
environment.
Assuming a rich, dense responsive environment full of responsive calligraphic
video, responsive sound, and active electronic, sounding, kinetic textiles, all respond­
ing concurrently to activity and conditions in real time, how can a composer hope to
shape such a complex environment without becoming a bureaucrat of media, or a
tyrant of experience? How can we marshal dozens and hundreds of concurrent media
processes creatively? When do some media processes start, and how do they vary in
response to other processes in an event? Based on physical dynamics, the Ozone media
choreography system is designed and built to support interactive spaces that reqUire
the real-time synthesis and coordination of arbitrary streams of video and audio in
response to actions by one or more people. 1
Artistic concerns-which include ethical a s well a s aesthetic concerns-motivate
the following desiderata:
(1) The composer, actor, spectator may be the same body, implying that we focus on
first-person experience;
(2) The primary modes of interaction are not based on (isomorphs of) lin�istic pat-
,
terns, but on continuous fields of matter and media;
(3) The participants are always in a common physical place, setting a very high
demand for sensuous density and effectively zero latency;
(4) The composer composes not specific event sequences but meta-events, or sub­
strates of fields of potential events;
(5) Design for continuous experience with the density of everyday settings.

The second and last of these desiderata again motivate a topological versus discrete
approach to time-based media.
208 C h a pter 7

Technical Features of the Media Choreography System

Given those artistic concerns, I designed the media choreography system to:
(1) Directly accommodate high-level composer semantics for interpreting or respond­
ing to player activity;
(2) Incorporate arbitrary continuous as well as discrete evolution of state;
(3) Support low latency responsiveness to sensor data;
(4) Support synthesis of structured and perceptually dense, continuous video and
audio with plausible response in real time;
(5) Leverage the intuitive concepts of energy-based physics of material dynamical
systems to allow composers to create potential event-landscapes-called state
topologies-in a way that is idiomatically concise and expressive.

In brief, the media choreography system allows the media synthesis system to use
any combination of sensor and metaphorical data to generate meaningful, even com­
pelling, aural and visual patterns on the fly from live or prepared audio or video
textures. By integrating the basic metaphoric components of the composers' language
with sensor data and derived sensor features in a continuous, real-time evolutionary
system, this media choreography framework simultaneously lets the composers think
in terms of evolving metaphorical states while at the same time constituting a potential­
dynamic system that can be evolved by the computer using computational physics
and topological dynamics.
The prototype media choreography system I designed was tested under harsh per­
formance conditions in 200 1 at the Ars Electronica and V2 by the TGarden consor­
tium, which provided valuable feedback for the design of the system. From 200 1 to
20 10, nine distinct responsive environments and installations were built using ele­
ments of the media choreography system.
The central technical challenge is how to make a navigable and playable responsive
media space that has no preassigned interface obj ects nor prespecified gesture/action
sequences. A typical "interactive" installation has a behavior that, however rich in its
basic dynamics, baSically does not meaningfully change its type: an eternal thunder­
storm of particles, for example . But how could we compose a responsive media envi­
ronment whose behavior qualitatively changes in a meaningful, palpable way according
both to the composer's design and to contingent activity?
The media synthesis instruments project video and sound into a room as ambient fields,
continuously changing according to autonomous dynamics and in response to player
activity. As the players interact with the projected sound and imagery over time, they
should be able to invent gestures that meaningfully shape and control the media. This
implies that we avoid prefabricated user interface objects or prespecified gestures but instead
allow the construction of manipulable objects out of the media textures themselves.
Practices: Apparatus and Ate l i e r 209

LAN

aSC/J itte r m atrix asc asc/J itter m atrix \ asc

Figure 7.3
Ozone architecture: the architecture of the current media choreography system. Diagram by
Morgan Sutherland and Sha Xin Wei.

Hardware and Software Components

Our real-time system is written as a set of components, each on a separate host: (1) a
data-routing, statistics, gesture-tracking and media choreography engine written in
MAX and C,z (2) a sound synthesis engine written in Max/MSP and SuperCollider with
externals such as a granular synthesis and DSP analysis, and (3) a video effects engine
written in Max/Jitter and C.3 We chose to implement our real-time media synthesis
instruments using Max/MSP/Jitter and SuperCollider because these are among the
most expressive, popular, and professionally maintained systems currently used as a
lingua franca by media artists and musicians. This gives us the possibility of efficiently
incorporating rich musical and visual imagery and dynamics without writing special­
purpose effects from scratch for restricted laboratory demos.4
210 C h a pter 7

Wireless Sensor Networks

In addition to these core system components, we used auxiliary devices to sense activ­
ity. For example, to track players' locations in the 200 1 version of TGarden, we used
a video camera to follow infrared LEDs worn on the players' heads. This turned out
to be more robust than visible-light vision methods, since the sole illumination came
from projected video under which players are often camouflaged. We also outfitted
each player with two sets of Analog Devices ADXL202e accelerometers, whose output
was mapped to a Compaq iPAQ running LINUX and beamed via 802. l 1b wireless
Ethernet to a fixed computer. In later applications, we used the low-overhead TinyOS
wireless sensor platforms,s with a sensor board containing a magnetometer, acceler­
ometer, photocell, sound meter, and analog and digital inputs for additional sensors.
While convenient and powerful as development environments, these sensor platforms
tend to be fragile for moving bodies. The locus of computation can migrate between
body-borne or fixed hosts as the tradeoffs between platforms, bandwidths, and power
supply evolve. Currently, where possible we use infrared camera-based computer
vision to track movement and form to avoid "instrumenting"-attaching a device
to-the participants.
All software components communicate using Berkeley CNMAT's OpenSoundCon­
trol (OSC) protocol.6

Clothing as I nterface
Each player is assigned an instrument model which is partially parametrized by their
continuous movement. The room/instrument itself evolves over time as it adapts to
the players' more or less expert gesture.
There are several reasons why treating clothing as interface is interesting and fruit­
ful. We can design the garments to test ballistics that augment or constrain player
gestures and poses in ways that can be mapped to vocabularies of motion, much as a
musical instrument's physical features provide idiomatic kinesics that make it learn­
able. Virtual instrument design7 and wearability studies8 indicate that carefully
designed physical constraints are crucial to the playability and learn ability of a gestural
instrument. The composers created the clothing with this in mind.
We started by looking to clothing as the most natural body-borne interface between
a human and his or her environment, and working with fabric and garment compos­
ers who are willing to experimentally extend or constrain the body to provide a variety
of gestural affordances. It is essential for theatrical and psychological purposes that
the costumes' fantastical design inspire the players to move freely and feel encouraged
not to simply habitually repeat but to improvise gestures. It is telling that when pro­
fessional dancers visited the TGarden, after a brief period of habituation they moved
far more surely than amateurs with the moving video projection and music, but they
P ractices: Apparatus a n d Atelier 21 1

Figure 7.4
Gestural instruments based on TinyOS wireless sensor platforms, Ubicomp, 200 3 . Picture by
the author.

tended to do their own disciplined movement rather than listen and co construct their
motions in concert with the room's response.
Allowing arbitrary improvised gesture implies that the body-worn devices should
not impede the players' movement. Therefore, we do not equip the players with head­
mounted devices (HMDs, goggles), cameras, or headphones. We chose to use acceler­
ometers9 with 2 g range because they detect the right range of forces of our gestures
(from free-swinging limbs to sharp chops of the hand), are small enough to be worn
unobtrusively as watch-sized pouches on the body, and could withstand the shock of
dance movement. We use wireless broadcast of sensor data because tethers would
unacceptably constrain the players' motion.

Feature Extraction
Given a set of sensors, one of the key problems to be solved by any interactive media
system is how to reduce the multiple time series of sensor data to extract useful fea­
tures. The Ozone system performs relatively simple statistical reductions: time-based
21 2 C h a pter 7

or sample-based moving-window averages or simple WM (Sobolev-type) norms, and

aggregations across multiple sensor streams and multiple players. An example is the
total angular momentum of the players' motions about the center of the floor.
One could try to perform the sensor data reduction by an automatic supervised or
unsupervised procedure, adapting, for example, a mixture model, or performing inde­
pendent component analysis (rCA) relying on some appropriate entropic measure.
However, in order to achieve meaningful performative, experiential impact, it is
important that the reduction of sensor data, especially that part which is mapped
responsively to the audiovisual synthesizers, be understandable by the players. r
believe that a good way to achieve this is to allow the composers to manually specify
the sensor feature mapping.

Video Analysis
Most of our video analysis is done live using standard computer vision tools such as
optical flow measures or morphological filters from a standard computer vision library
via Jitter, but we have built some moving-window background thresholding and adap­
tive motion extraction extensions.

Audio Signal Analysis

Audio signal analysis10 modules combine common signal analysis algorithms such as
spectral centroid, flux, kurtosis, envelope following, bark amplitudes, mel-frequency
cepstral coefficients, and derivatives of these features that can be used compositionally,
as receptors from other components of the system, and as emitters injecting biased
dynamics into other media elements.
Signal analysis procedures using a priori models can behave unexpectedly due to
unpredictable activity in our improvisatory and experimental environment. For
example, the lighting or acoustical characteristics of the space could change radically,
various stages in the system generate noise, and the media systems themselves gener­
ate feedback, both at the level of reproduction (projections and microphone pickups)
and of analysis. The behavior of the algorithms implemented in a given configuration
can fluctuate conSiderably by design, complementing the system's overall field state­
based behavior.
As an example of a derivative audio feature: a simple "activity" measure calculates
the amount of fluctuation of the spectral centroid of a boundary microphone mounted
on a conference table, by basic prefiltering and onset smoothing. The fluctuation
drives a two-stage cascaded leaky integrator, which advances the system with two
"rates" of activity. The dynamical result of the fluctuation flowing through the two
integrators is usefully richer than one-to-one linear functions of activity.
For another application, the Remedios Terrarium responsive environment, l 1 we
built a simple " dreamer" that functioned on minute (or zero) amounts of audio, "lis-
P ractices: Apparatus and Atelier 21 3

tening" for events, capturing these into buffers, and emitting them audibly at a later
time, via distorted or otherwise transformed playback. The occurrence and quality of
these varied depending on the diurnal state of the system. Storing regular intervals of
fast Fourier transform (FFT) snapshots, we can generate continuous "fabrics" or
"thumbprints" of the acoustic timbre of a space, and these can be presented in a form
derived from the audio and/or can inform other media.
In recent years, we have synthesized acoustic and gesture analysis techniques from
IRCAM into our system, notably a grain-based coding of sound corpora by perceptual
descriptors (CATART), hidden Markov models revised for sparse data with no training
(OMAX), and continuous gesture following (gf)Y This has greatly extended the sub­
tlety of tracking not only acoustic information from ordinary air microphones, but
also camera-based optical information as well. Most promisingly, we can now break a
bottleneck faced by camera-based methods restricted to the low frame rates of con­
ventional video cameras and video acquisition. Where video acquisition rates were at
best on the order of 1 5 to 24 frames per second, digital audio frame rate is typically
44,000 frames per second, or more. Compared as sensors, therefore, a microphone's
temporal resolution is typically 1 , 000 times greater than a camera's.

Time Scales
While the media choreography engine runs in real time, the architecture of the system
effectively operates at three different time scales. In practice, we found three scales of
temporal dynamics to be meaningful to the composers: micro scale of 0 ( 1 / 1 02)-second
sensor data (e.g., denoising at the sensor PIC), meso scale of 0(1)-0(10) seconds:
composer-specified gestural grain (e.g., the rate at which perceptible changes in the
value of a fundamental state like Solo occur), and macro scale of 0(102) seconds: a
"narrative" state, the rate at which the system switches between simplices, which
people can perceive as the unfolding of an event.

Media Synthesis
I have emphasized the core " state engine" of the media choreography system because
of its conceptual implications. But the real-time sound and image synthesis instru­
ments written in Max/MSPjJitter and similar real-time media programming environ­
ments are what create the perceptual richness of the performative event. These
instruments are designed to be controlled by multiple data streams from the Ozone
network such as the "raw" and "cooked" (numerically reprocessed) sensor data as well
as the overall state topology vectors.

Audio Synthesis
The sound instruments include several self-contained designs such as a wind generator
and the polyphonic vocal synthesis engine incorporated into the Meteor Shower
214 C h a pter 7

(2006) and Cosmicomics (2007) installation-events.13 But development is primarily

focused toward a concatenative synthesis framework via MaxjMSP. We prefer concat­
enative synthesis because of its wide range of timbral possibilities, both in terms of
seed sound material (any audio file can serve) and multiple levels of parameter modu­
lation. In fact, every unique configuration of parameters (Le., "preset") of a module
of MSP abstractions constitutes a software instrument with its characteristic sensitivi­
ties, temporal behavior, sonic quality, and transform or synthesis logic. We design our
parameters to be metaphorically meaningful to the designers (Le., " sludgy" rather than
"xyzzI ") . This allows the sound programmers to rapidly create very different sonic
behaviors that can be auditioned by a composer. The behaviors can be copied, modi­
fied, and extended by a composer more familiar with sound design and Max/MSP.
A challenge in working with different types of fast sensor input is that the sonic
dynamics are often tightly coupled to those particular to a given sensor, which raises
the question of which components in the system should be calibrated to which ranges.
A granular instrument's metaparameters are designed to operate within a nominal
range of the continuous interval [0, I], but the instrument does not constrain input
to this range, always allowing for the possibility that a system will attempt to drive
this instrument outside of the expected boundaries.
This decision comes from early experiments involving mappings of various (and
variously conditioned) channels of sensor and video analysis input to about a dozen
low-level parameters, wherein the unpredictability of input dynamics further enriches
the process of composing responsive media. If input constraints are desirable, for
either technical or compositional reasons, they are programmed into a specific
mapping as opposed to being built into the fr amework from which instruments are
designed.
The use of floating-point data throughout all stages (immediately following data
acquisition) of such data "munging" prevents stages that may be very low in dynamic
range from discarding potentially useful information through truncation and round­
off error. 14

Video Synthesis
My general strategy is to treat video not as image (a picture of something) but as
structured light. And if the video is synthesized by our real-time responsive software,
then this structured light behaves in concert with the action or condition of that with
which it interferes. Calligraphic video as palpable light field becomes an alchemical
substance, a shadow of Heraclitean fire. In the lab, we are trying to create real-time
responsive video as a substance that, coupled with computer vision or other more
tangible sensing techniques, can be manipulated as a painterly medium.
Rather than rely on primitive oculocentrism, we tap the user's large pool of corpo­
real intuitions about the behavior of continuous physical material to build interactions
P ractices: Apparatus and Atelier 215

with dense visual textures i n novel and complementary applications. Every person
acquires corporeal intuitions from infancy on, so it seems reasonable to leverage that
sort of lifelong, preverbal capacity.
We are building these calligraphic video interfaces as platforms for research in
gesture along phenomenological lines. Gendlin complements the logical structure of
cognition with felt meaning, which has a precise structure: "Experiencing is 'nonnu­
merical' and 'multi schematic' but never just anything you please. On the contrary, it
is a more precise order not limited to one set of patterns and units. " lS
Moreover, categories may be logically but not experientially prior to instances. This
is a strong motivation for seeking a non-object-oriented approach to manipulable,
active graphics. Human experience is material and corporeal, and is intrinsically struc­
tured as temporal processes. (This motivates our turn to dynamical fluids.) Based on
fundamental work with immune and nervous systems, Maturana and Varela moved
from the discussion of cellular organisms to autopoietic systems, loosely and briefly
defined as continuously self-reproducing sets of processes in an ambient environment,
whose relationships remain dynamically intact across changes of constitutive matter.
Given that, at the everyday scale, experience is continuously composed of tem­
porally evolving matter, we wish to have an experimental platform for creating
objects of experience that do not have to be selected from a preexisting category. For
example, graphic objects in our manipulable system must not appear to the user as
built out of a preexisting set of geometric primitives. It is essential, of course, that
these be manipulable in some improvised way, and essential that these manipulations
be continuous in time, to permit us to study the evolution of material form­
ontogenesis, to use Rene Thom's term.16 We build calligraphic video: video texture
that responds to manipulation by human gesture as interpreted from camera-based
input-as apparatuses in which we can conduct studies of how humans imagine,
create, and perceive dynamical "objects" from fields that are effectively continuous
in time and space. Working with continuous fields of video permits us to construct
experiments in which objects can be formed by improvised manipulation and allowed
to return to general substrates. The manipulations must be as free as possible of
class-based tools or menu structures (else they would imply preexisting logical, func­
tional, or geometric categories) . The video texture substrate may not appear uniform
at all, but it is continuous in space and time. Rather than use arbitrary dynamical
systems to animate the responsive video, we choose to study the structure of corporeal­
kinesthetic-visual intuition via improvised manipulations of media that leverage
corporeal-kinesthetic-visual experience of continuous matter commonly encountered
from childhood.
Practically, our strategy is to treat video as initial data for physical models of mate­
rial like water (Laplace wave partial differential equation [PDE]), smoke (Navier-Stokes
PDE), etc. We borrow the best techniques from computational physics that now can
216 C hapter 7

be executed in real time. Indeed, since video is typically projected onto two dimen­
sions anyway, we may as well forgo 3D graphics and free up computational resources
to compute and present much richer 2D textures.
Over eight years, we have created several generations of real-time video-processing
abstractions in NATO, and now in Jitter. At a time when video was mostly edited off­
line and the tools where designed for such paradigms of use, these video instruments
were designed to respond to live video streams with negligible latency because they
were intended for live performance.
Using the Max/Jitter framework, we can implement processing instruments on
multidimensional dense lattices at video rates, streamed between local networks of
hosts to parallelize the computation. An example of a visual instrument is the Meteor
Shower (2006) which integrates a set of particles across a gravity field due to a 2D
lattice of attractors. Mapping the attractors' masses and locations into a grid yields a
computation speed independent of the number of attractor points. Over the past two
and a half years, we have built an extensive library of CPU- and GPU / Open-GL-based
real-time video instruments. We describe the computational physics and strategies for
parallelizing on different hardware in a technical paper. l?

State Engine Conti n uous State Dynamics

In this section, we present a formal description of the dynamical system which

models the meaningful configurations of people and activity understood by our
system. This model is intended to compactly capture the metaphorical ontology and
dynamical response logics conceived by the composer for each player and for the room
as a whole.
The state engine is based on a continuous dynamical system modeled over a sim­
plicial complex, and coupled to the activity of players in the environment. The sim­
plicial complex represents an N-dimensional metaphorical space, the vertices of which
correspond to elementary conditions imagined by the composers, such as Intrude,
Feed, or Reveal in the TGarden's state topology.
To be clear, the Ozone state engine is quite different from the finite state machine
(FSM) of classical computer science, based on abstract discrete states. It is modeled on
the continuous model of potential fields and material dynamics of physics. In the
Ozone model, states can overlap; the environment can be in more than one basis state
simultaneously. (Here I use "basis" in the sense of a topological vector space and
quantum mechanics.) And equally importantly, the state of the environment can jump
between discrete states but can also vary continuously "over time. "
The instantaneous state o f a player within the system i s represented b y a point in
this metaphorical space, along with a region in a sensor space defined by the informa-
P ractices: Apparatus and Atelier 21 7

tion sensed from the environment plus any features derived from the sensed data.
Each simplex corresponds to a valid combination of elementary conditions; the topol­
ogy of the simplicial complex defines a narrative landscape, defined by the composers,
which conditions the possible evolution of the experience within the performance
space. (For example, a player's evolution through a connected set of simplices might
describe a timeline in which a player passes from some marginally active set of states,
through some transitional region, and into one meant to give rise to a more turbulent
responsiveness.)
The trajectory followed by a player's state is a path in the simplicial complex under­
lying the model. Intuitively, each player's state is treated as a particle with some
inertial mass, evolving according to laws of classical mechanics. N-dimensional forces
are applied to the player's state, using energy derived both from sensors and current
state. The inclusion of energy derived from both the player's activity and the location
in the simplicial complex allows the system to evolve continuously according to pre­
designed dynamics as well as player movement.

Tem poral Dynamics

While the engine runs in real time, the design gives the appearance that it is running
at multiple time scales. This allows the system to simultaneously sustain a sense of
tangibility based on fine-grain temporal response (e.g., for features derived from sensor
data), as well as of global evolution of state (e.g., for the metaphorical state) . For
example, the sensor data changes rapidly in response to the user, which may cause
immediately perceptible changes in the visual and auditory landscape. However, the
values of active fundamental states within the metaphorical space change relatively
slowly in response to the sensor data and the physical simulation, and thus cause
perceptible changes at a slower rate than the sensor data. Finally, when a player state
moves from one simplex to another, a perceptible change in the character of the
output may also begin to occur as a different set of fundamental states within the
metaphorical space become active; however, movement between simplices happens
infrequently compared to movement within a simplex (see figure 7 . S ) . \

Player and Room State S paces

In our system, each player is assigned a total state which is a combination of the player
sensor data and metaphorical state. Similarly, the room as a whole is assigned a total
state; formally the room is treated as the zeroth player. The metaphorical state is a
point on one factor of the model space of our dynamics system, and it evolves con­
tinuously, in a way that is determined by the players' movements in the space relative
to the design decisions about the model structure. By design decisions we mean the
218 C h a pter 7

Room state to p o l ogy

Solo
V = 0. 5
G ro u p-Density
V = 0.4

G ro u p-Sy n c h rony
�(O.7.0.3)
I nsensitive
V = 0.5

V = 0.4

S e n s itive

S l ud g e V = 0.4
V = 0.6
Passive

(0.5,0.5)
A l e rt
V = O.O V = O.O
- - - - .... ,
.
�\ . ... .. ., , '" ;
,

.,
y/3)(t) =
.,
.,
\ .,
,
...
- - - - - -
- V = 0.8 ,
..... ..... ..... �
- - - - '"
- - - - -

P l aye r state to p o l ogy

I ntrude Solo
V = 1 .0

V = -0. 5 V = O.O

G ro u p
Spawn Morph
V = -1 .0
V = 0. 5 V = -0. 5

F i g u re 7.5
Room state topology and person state topology. Diagram courtesy of Maja Kuzmanovic and Sha
Xin Wei.
P ractices: Apparatus and Atelier 21 9

choices made by the environment's composers about the implemented state space
topology and the parameters governing its behavior.
In this section we describe the player metaphorical state space, the sensor state
space, the energetic model that glues the sensor and metaphorical spaces, and the
dynamics governing the evolution of player states. These correspond to the data
structures, parameter assignment, and time-evolution algorithms underlying our
system.
In applications, we model the evental condition or state of each person (player) as
a finite-dimensional topological vector space rp of possible metaphorical states. This
descriptive state space rp is one factor of the complete state space Mp for each player,
the other factor being the space Sp of possible sensed activity for that player within
the environment, the space of sensor data which the system perceives. So
(7. 1 )
the space o f all possible vectors o f data the system maintains for a player p a t any
instant, is represented by a point in rp (the metaphorical state), a point in Sp (for the
sensor data), and their associated time derivatives. The complete space of states is the
product of those describing each player:
Np Np

M = TI Mp = TI rp x sp (7.2)
p=O p =O
where Np is the number of players. The original implementation of this system was
built for five players, but the model has no limit. I prefer to work with three or more
players to break up conventionalized dyadic interaction.
This model provides a representation for the dynamics of the media environment
that captures the high-level semantics of the composers and is at the same time a
representation that can be effectively computed.
We describe both the base and sensor spaces in more detail in the following sec­
tions, and subsequently the parametrized energy model through which these two
spaces give rise to the system evolution. Figure 7 .2 shows a sample diagram from the
preliminary player state topology hand sketched by the composers of th�, TGarden
';
2001 system.

M etaphorical State Space

The base space rp defines the possible metaphorical states that the player p in the
interactive environment may inhabit. A configuration of potential states for the room
is shown in figure 7.S; again, we think of the room in this context as another player.
In our typical applications, the human player descriptive spaces are all identical, and
one may think of the corresponding states as several points on a common space of
potential human physical conditions.
220 C h a pter 7

When rp has more than one connected component,

rp r�) u r�2) U "
= ', (7. 3)
the player state ')'pet) is given by a point on each YP (y�l ) , y�2) " ) . I n this model, the
= . .

substates on each connected component evolve independently of one another, and

so may be treated formally as separate players. As a result it suffices to describe the case
in which rp has a single connected component, which we will assume in what
follows.
Each space rp of metaphorical states is built up as follows. The composers choose
a collection of N fundamental states ni, i 1 . . . n, each named after an elementary
=

condition or scene such as Intrude, Feed, or Reveal. In the player state topology (figure
7.S), the pure state named Reveal corresponds to the condition of the player appearing
to skate across the surface of a magma or ocean, leaving only simple marks on the
apparent surface of the proj ected fluid imagery.
A player state ')'pet) representing a player p at a time t is given by a normalized set
of weights for the mixture of states that player is occupying,
N

Yp (t) = L Aj( t) Vj, (7 .4)

j=!
and the combination of fundamental states which describe it is convex,
N

L Aj (t) == 1 . (7.S)
j=!
The player state ')'pet) at time t determines the metaphorical evolution of that player
through the lifetime of that instance of the system. This set of states is modeled by
overlapping mixtures, rather than by a discrete graph of nodes and arcs, in order to
allow a player state to interpolate, or combine continuously between two or more
states, corresponding to continuous and rich changes in the environment.
At any moment, a player inhabits a mixture of a particular set of "component"
states, so rp is restricted to certain permissible combinations, or simplices ajkl , where
.

(7.6)
is a simplex spanning those neighboring or component states. Think of a simplex as
a span of vertices in some Euclidean space. If it spans three vertices then the simplex
is a two-dimensional triangle; if it spans four vertices, then the simplex is a three­
dimensional block (the tetrahedron whose body is the interior of the span of these
four vertices), and so forth for any higher dimension. A player state may occupy a
positive mixture:

(7. 7)
Practices: Apparatus and Atelier 221

only if the corresponding simplex (hjkz kn is contained in rp. At any moment, the player
state describes a pOint on the simplex spanned by the states it is inhabiting. This player
descriptive state (or the set of such states) is what is evolved by the dynamics engine,
and rp is just the connected union of the simplices of the system, in other words the
polyhedron of the simplicial complex.
The composers or designers of the responsive environment specify the set of fun­
damental states ni and metaphorical associations to each, with cooccurrence relation­
ships determining exactly which mixtures of fundamental states, or simplices ajkl ,
exist in the system.18
Transitions can be defined at simplex boundaries. This set of boundary conditions
between simplices in the complex determines a graph g({a}) between the simplices
contained in rp. When necessary, we elevate g to a weighted graph to allow a mecha­
nism for mediating situations in which more than two simplices share the given
boundary, and make it directed in case composers desire asymmetric transition rela­
tions between neighboring simplices. The weighted, directed graph g is represented
by a matrix of floating-point values. Each entry g ({ai , aj }) of this matrix is zero if the
boundary from ai to aj does not exist, or if the transition is forbidden, and greater
than zero otherwise.
To summarize, the domain rp of descriptive states }pet) for each player consists of
the polyhedron of a simplicial complex built out of vertex representatives of the states
of the system, and determining which of those states may be simultaneously active,
combined with the set of boundary relations gels}), describing a pseudo-narrative
topology determined from design decisions regarding which states meet in a particular
composition.
A player's trajectory }pet) for time tE [til td is a path in rp determined by a dynamic
that we describe in the sections that follow.

Sensor Space
As indicated earlier, the player state space Mp is made up of the descriptive space rp
outlined above, over each point of which lies a space Sp of possible sensor feedback
that the system has about player p. The sensor data is used to drive the dynamics, in
the manner described below, in response to the movements of the players within the
environment. The sensor data for each player p consists of a vector of real-valued
parameters s/1[t], obtained from hardware sensors in the room and their derivative
features. Typical applications use very simple derivative features, but the model sup­
ports any features that can be represented as a time sequence of vectors of floating­
point values.
The parameters are updated in real time, at a rate that represents the movements
at a time resolution sufficient for the media synthesis components of the system. This
resolution frequently exceeds the requirements of the dynamics model itself, because
222 C h a pter 7

the effective integration time of the dynamics is significantly longer than those of the
media synthesis components.
When the human narrative spaces also coincide (as has always been the case in our
implementations), one can think of these states as particles moving on a common,
piecewise-linear domain
(7.8)
In cases where the human player states respond to similar sensor data, we have
sometimes found it useful to think of there existing a single sensor space in which
the sensor data of each player takes its values. One may then consider the state space
as consisting of Np copies of rcommon with a uniform player sensor space Sp'

Energetic Model
For each player Pk, we engineer an evolving energy landscape over the state topology,
letting the player state move as a massive particle on rp, which evolves according to
the laws of classical mechanics.19 It is this movement that is recapitulated in the
changing character of the output and responsiveness of the associated media synthesis
instruments.
The potential portion of this energy arises through the coupling of the sensor data
acquired from each player to the fundamental states in the system. To a given point
Y == Yp (t) on the base state space rp and sensor data vector s == sp(t) of Sp is associated
an energy given by:

U[y] = L Hr ( y , Vi ) Hs (S), (7.9)

VkEa(r)
a sum over the sites Vi of the current simplex O'( y) . In this sum, Hr gives the
energy dependence on the pOSition y relative to the pure state sites Vi, while Hs is
designed to give the energy of the player sensor data vector s relative to data assigned
at Vi' A wide variety of energetic models based on equation 7 . 9 are possible. Those
which we have implemented consist of a sum over pairs of factors having the
general form
U[y] = L f(Ad {e-flEk [Sl+gs¢k + gvVk } (7. 10)
Vk Ea(r) ,

where f(Ak) is some function of the weights of JP relative to the states Vk. We describe
below in more detail the r- and S-dependent contributions that we have found
useful.
Figure 7 . 6 illustrates the simplicial model of state structure and parameters. The
simplicial model is constructed from the data indicated. JP[t] is the current state. sp[t]
is the player sensor data vector. The data associated with each elementary state are
the nominal sensor value 11, the variance, the static potential V, and the local scale of
the sensor contribution to energy rp.
P ractices: Apparatus and Atelier 223

�, cr, <I>, V

1« t)- - - - - - _

F i g u re 7.6
Simplicial model of state. Diagram courtesy of Yon Visell.

Sensor Coupling, Hs
Our sensor energetic coupling for the kth player has the form:

(7 . 1 1)

The state-dependent contribution I/Ik controls the local scale of the sensor contribu­
tion to the energy, and the static potential Vk is included to give the media choreog­
raphy system a background dynamics independent of the sensor activity, in a way
that is capable of lending it dynamic tendencies independent of the player sensor
data. The coupling constants gs and gv control the global relative scale of these
contributions.
Ek[S] is the energy of the kth player's sensor data vector S relative to data at Vk' And
f3 is a coupling constant for the model dependence on Ekt controlling how sensitive
the system is to the player's sensor activity.
The local sensor coupling contribution Ek is determined in conjunction with a
model parameter assignment procedure performed at the time the environmental
dynamics are being designed. The composer assigns a nominal vector of Sell S?r values
f.1ak and variances O'ak which are chosen to correspond with the metaphorical descrip­
tion of state Vk. (In case the player spaces rp are identical, these data are typically
chosen to be the same for all players.) Then we take a quadratic dependence
( 7 . 12)
because it represents the leading-order dependence near a generic potential with a
minimum at the given sensor mean:
N,

Ek [s 1 L ( f.1ak - S) ( f.1k{3 - s{3,p (t » / O'a{3,

= ( 7 . 13)
a{3= l
224 C h apter 7

where Ns is the dimension of the space of sensors Sp'

The mean sensor vector Jla represents a set of sensor values which, to the extent
that they agree with the player's activity, are meant to draw the player state }p[t] toward
the location where they are assigned. We have typically taken the variance (fa/3 to be
purely diagonal. Choosing values for the variance s allows the composer to assign the
relative sensitivity of each state to different sensor vector components (sensor chan­
nels) . A variance which is large relative to the real variation seen in a channel will
leave the state relatively insensitive to sensor data in that channel.
If one desires to associate more than one representative nominal sensor vector and
variance per site, one can have a weighted sum of such contributions to Ek• Because
the generalization is not difficult, we restrict discussion here to the single summand
case.
The r-dependent contribution to the energy at a player state position }p[t] is given
by restricting to the current simplex and choosing any potential function on it. We
compute in convex coordinates A; on the simplex, and refer the standard embedding
of the N-simplex in RN as the constraint surface

N
�>'k = l , O :o; Ak :O; 1. (7 . 14)
;=1

The constraint of convexity means that the N-dimensional components of any

vector fields on the simplex should sum to zero, in order that they remain tangent
to it.
To determine which potentials may be promising, it is useful to consider the force
each would give rise to, using the formula
F = -V U (7 . 1 5)
(the dynamical results of which are explored in the next section) . If we want the force
to be continuous across simplex boundaries, we can decide that the normal compo­
nent should vanish along each. One could also consider configurations in which this
normal component attains a constant value on each boundary, and assign these
boundary forces in a consistent manner, but the zero case is certainly the Simplest,
because it reqUires no intersimplex gluing considerations. The consistent application
of gluing conditions can be obstructed by the topology of the model.
One solution is to take the force due to Vk to be polynomial in the coefficients �
and independent of the other coefficients:

( 7 . 1 6)

for some constants em.

The reqUirement that the force F be tangent to the simplex at each pOint, or in
other words that its components Fk sum to zero, can be assured everywhere on the
Practices: Apparatus and Ate l i e r 225

simplex in the simplest case by setting Fk C(Ak - 1 / N), where N is the number of
=

states in the simplex. Such a force is obtained from an inverse potential,

(7 . 1 7)
If the Hs term were location-independent, one would have a combined simplex
potential given by

U[ r] - A(t) · A(t) A(t) · ns

_
(7. 1 8)
N

with ns ( I , I, . . . , 1 ) denoting the normal vector to the simplex, and A[t] the vector
=

of weights for y. The latter term can be thought of as a Lagrange multiplier enforcing
the constraint of tangency.
Dual to the last example, one may consider the quadratic potential
(7. 1 9)
where Ak[t] are the weights giving the component along the pure state Vk of the player
state y. This is a harmonic potential centered at the pure state site �[t] I, and the force
=

it gives rise to is restorative and proportional to the displacement from Vk. This
force does not lie tangent to the simplex, and one must project out the normal portion
in RN .
A class of forces which are manifestly tangent to the simplex are those computed
from the Euclidean distance on RN,

(7.20)

where ek is the unit vector, which is 1 in the kth component and 0 elsewhere. The
associated distance vector lies on the line between the player state and pure state, and
as a result any central potential such as
(7 . 2 1 )
gives rise to a force which lies tangent to the simplex. We have for example taken
p -I, which gives rise to an inverse-square force law.
= \,
The last two potentials give rise to forces having discontinuities acros� simplex
\

boundaries. The force, and consequently the momentum, have such discontinuities,
but the player state trajectory is continuous. We will describe the dynamics further in
the next section.
The environment can simply tend to drift to a steady-state local minimum, thus
providing a closure to an experience corresponding to narrative closure in a conven­
tional interactive scenario. And in the simplest case, time-based behavior can be
incorporated through time dependence of the parameters of the system, which might
be accomplished by treating a clock as a channel of virtual sensor data.
226 C h a pter 7

The pseudo-physical model, like any physical model, evolves by minimizing energy.
In order to see explicitly how a state evolves, we treat it as a particle with an associ­
ated mass and compute the force acting on that state. We do this in the next section.

Dynamics
In this section, we describe how the model evolves the player states in response to
the player sensor statistics. Evolution proceeds by integrating the first-order equations
of motion (equation 7 . 1 5 and equation 7 . 1 6), including contributions from the gradi­
ent of potential and kinetic energy terms.
Using a total potential energy U[y] of the form described in the previous section,
the force is computed from the gradient of U,
F[y] = -VU - � Yp (t), (7.22)
where we include a damping force depending on a coefficient Xif and the time deriva­
tive Yp[t] of the player state, which is useful, for example, for suppressing oscillatory
modes of the system. The force law in convex coordinates is given by

(7 .23)

For example, with an inverted harmonic potential Hr, one gets (temporarily setting
the background couplings gs, gv 0 for simplicity) a force proportional to
=

Fk (t) = (Ak (t) - 1 / N) e-f3Ek [Sa(t)] - �ik(t), (7.24)

with Ek computed as in equation 7 . 1 3 .
We then take F{, the force tangent t o the simplex, obtained by projecting out the
( 1 , 1, 1, . . . , 1) component (perpendicular to the simplex) . The dynamics are deter­
mined by Newton's second law, F[y] = my. We evolve each player state by means of
the first-order versions of this, in components:
. . 1
Ak � Ak + - F{ (t)dt (7.25)
m
Ak � Ak + ikdt , (7.26)
where dt is the integration time step and m is the mass of particle representing the
current state.
Framing the dynamics this way gives, in addition to the parameters assigned by
the composer at the time of system initialization, several real-time controls over the
evolution of the physical model: the mass and damping of each player state, m and
x, the coupling constants g., gv of equation 7 . 1 1, and the sensor sensitivity {3.
When channel-specific sensor sensitivity is required to be adjusted in real time, one
factors from each diagonal component of the variance vector
P ractices: Apparatus and Atelier 227

(7.27)
a global factor O"Z'0d which may be varied, and likewise with the other local parameters
of the system.
The palpable magnitude of certain of these parameters is constrained by the desired
time scale for evolution for the autonomous and non autonomous motion on rp. The
potential energy is the composer-imposed background energy landscape assigned to
meaningful states, eliciting evolution even in the absence of sensor data.
For example, the typical time scale governing the motion of a player state of mass
m in a harmonic potential of magnitude gv Vk is

T - �g:Vk . (7.28)

In this case, one chooses m, gv,

and each potential Vk
such that the autonomous
processes of the system have the desired meso/macro time scale behavior. Param­
eters governing the motion in response to sensor data must be adjusted separately
such that the typical time scale for that response has the desired character, and
it is easier to do so empirically once the system is set up. The result of this pro­
cedure is to constrain the set of ranges of the dynamical parameters to what is
useful for the system's intended response. This, however, is very sensitive to small
variations.
One key point: These equations are used to implement a dynamical system in the
computer, but the topological dynamics can be defined without explicit numerical
constants and differential equations.

Experimental Work
The early exercises, studies, and installation-events by Sponge20 dealt with particular
questions in performance research: how to make events that were experientially as
powerful as works of avant-garde theater but without resorting to verbal/written lan­
guage, erasing the distinction between actor and spectator, and relying on thick,
physical/computational ambient media. TG2001 as built by FoAM and Sponge was an
installation-event that marked a transition and a bifurcation from per\o rmance
research into a strand of public installation-events and a strand of studio-laboratory
research in the Topological Media Lab (TML) . After leaving Stanford for Georgia Tech
in 200 1 , I started the TML to take stock of, and strategically extend, some of the
technologies of performance according to a particular set of ethical-aesthetic heuristics
inspired by continuity, human performance (e.g., the violin), human play (e.g., in
water and sand), and nonelectronic matter like clay, smoke, or rain. I wanted to make
responsive media synthesis engines and gestural instruments, and choreography
systems that would allow participants to experimentally costructure, not " interact"
228 C h a pter 7

with, coevolving ambient life in the "real time" of perceptually concurrent action and
the specious present.21

Methodology versus Experimental Phenomenology

One of the first questions I was asked by colleagues from the sciences was: How do
you know that this works? How do you measure progress in this domain? What is
your methodology?
A negative response would be: "This is an activity incommensurate with your
domain and your norms . " It is difficult to fit methodologies for measuring utility­
oriented applications to the composition and evaluation of expressive technologies,
especially those based on statistical survey methods. As Barbara Hendricks said about
the design of playgrounds: there is no average user.22
More constructively, we can observe that the domains of performing arts, installa­
tions art, and cinema all have their own systems of evaluation, mediated as much by
the opinion of expert nonpractitioners as by popularity measures like attendance and
market measures like box office receipts. The role of the critic, one who is informed
by the history and conceptual ambitions peculiar to the context of practice surround­
ing the obj ect of criticism, is well established with both practitioners and audiences.
Even if that critic's judgment is contested, so long as the critic's responses are relatively
coherent and stable, one can use them to triangulate the position and value of a piece
of art with respect to one's own criteria. Peter Brook, a celebrated living director whose
career spanned the Royal Shakespeare Theatre, Artaud, the Theater of Cruelty, and
Grotowski, argued for the essential role of critics in the healthy ecology of theater.23
Such a community of cultural beacons is well established as an evaluative network at
least as accepted as a jury system composed of so-called peers. One problem with peer
review is that such a system tends to average against innovations, perturbations far
from equilibrium. One practical task in an emerging community of art practice could
be to establish a critical practice alongside the material practice.
Human subj ects committees have been designed to prevent abuses of subj ects in
experiments in psychology and medicine, influenced by controversies in the wake
of World War II like Stanley Milgram's experiments in obedience to authority.24 But
then do such committees comprise the creeping edge of a Foucauldian discipline
emergent among institutional experimentalists made docile? How else would one
design phenomenological (not statistical or empirical) experiments but in the mode
of art? However, to be interested in the phenomenological is not the same as to be
a phenomenologist (Le., an adherent of Merleau-Ponty, Heidegger, or Husserl), which
would be a scholastic specialization. To be interested in the phenomenological is to
be interested in the relational aspects of experience in embodiment and in the
essences as they emerge in the course of lived experience. And most importantly it
is to be sensitive to situation or context and how they may be bracketed. Human
P ractices: Apparatus and Ate l i e r 229

subj ects research committees are an example of local rationality that masks global
irrationality, just as "health" as a category in the market of commoditized medical
care and risk management can ignore actual human well-being. Meant to protect
experimental participants from being subjected to unethical procedures, "institu­
tional review boards" or " human subjects committees" cannot govern a priori the
essentially boundlessly unpredictable processes of creative invention in theater, cho­
reography, or art making in general. When such committees do try to govern creative
work, they must permit an unconditioned range of action adequate for creative
design, or enforced rule-governed accountability tends to yield uninspiring work, or
there is a disconnect between what is reported and what is actually done. In any
case, this is clearly a situation of governmentality in institutionalized cultural
production.

Technical Research

I approached the branching family of play spaces represented by TGarden, txOom,

tgvu, and trg as phenomenological experiments of a certain kind, as events based on
gesture and movement rather than on language, for people face to face in a common
place, playing and improvising meaningful microrelations without language, in thick
responsive media. I saw those play spaces as opportunities for ethico-aesthetic play,
to adapt Guattari's concept of the coming into formation of subjectivity; as places to
engage in biopolitics, radically dispersed into tissue and molecular strata, and reaching
far beyond the computational media arts, meeting with experimental impulses in
dance, movement, textiles, musical performance, experimental theater, but also the
most speculative initiatives in urban design.
In his epochal essay on a revolutionary Theater of Cruelty, Antonin Artaud listed
a mise-en-scene of theater in which all the theater arts-costumes, scenography, light­
ing, sound, body movement-would act in concert and with agency equal in power
to the dramatic text. The Topological Media Lab has created its responsive media and
instruments as conscious extensions of an Artaudian spectrum of technologies of
performance.
However, the TML's broader aspiration is to create apparatuses for conducting
speculative, critical, social, cultural, phenomenological experiments. Terry Winograd
once commented that a phenomenological approach (versus human-computer inter­
action's nonrigorous appeal to the statistical empirical) draws insights from experi­
ments of one, from singular experiments.
The ambition here is to conduct even the most philosophical speculation by articu­
lating matter in poetic motion, whose aesthetic meaning and symbolic power are felt
as much as perceived.2s It is one thing to do philosophy or science of dance, and quite
another to do philosophy informed by movement. And it is a further challenge to
articulate movement-informed philosophy not in written text but in movement or some
230 C h a pter 7

F i g u re 7.7
IL Y A showing body gesture related to smoke, 20 12.

other temporal medium. In order to prepare the ground for such a challenge, I shift
the emphasis from representation to speculative, corporeal experiencej hence the
Topological Media Lab's emphasis on technologies of performance, live event, and
real-time responsive media-on alchemical matter.

Alchemical M atter

To let people play immersed in media with evocative, responsive qualities, we could
have them step into a warm pool of water laced with honey, so why use computational
media? Computing the quasi-physics allows the creators to inject a physics that
changes according to activity and local history, and responds in ways that resemble
yet are unlike any ordinary matter. This is analogous to the alienation effect of theater,
but not at the level of whole bodies (characters, actors, spectators, plot). Instead, what
continuous, dense, topological dynamical systems afford is a microfine alienation
effect at the level of substrate media such as calligraphic video, gestural sound, and
kinetic fabrics imbued with uncanny physics.
Practices: Apparatus a n d Atelier 23 1

Indeed it would take a lot of work to build up to macroscopic obj ects and actions
from relatively homogeneous textures and simple dynamics. However, I would say
that it is no more difficult or complicated than the enormous amount of hard-earned
psychosocial knowledge, narrative apparatus, and literary skill needed to render a
character in a novel or play from the raw material of alphabetic text and grammar.
Such textural, alchemical techniques seem strange and unidiomatic for all of us who
have been trained to the aesthetics and logic of whole bodies and macroscopic human­
scale objects like words, props, characters, and theatrical or game action.
The Ozone state engine evolves through a rather sparse topological landscape with
few valleys and peaks, whereas the visual and sound fields are synthesized as densely
and temporally finely as possible, and as necessary to sustain a rich experience with
millisecond-scale dynamic response that we do not attempt to compute using the
slower state engine. The reason for decoupling the dynamical metaphorical state
engine from the media instruments is in fact to decouple the evolution of the behav­
ioral response "climate" from the dynamics of the visual and sonic textures, which
should be as rich and tangibly responsive to the players' actions as possible. It seems
artistically and compositionally useful to keep these dynamiCS decoupled from one
another.
My concern in the context of this chapter is precisely with the possibilities that a
microphenomenology-free of ego and anthropocentrism and indeed free of fixed, a
priori objects-can offer toward fresh and refreshing improvised play. Aesthetically,
this play should take place immanently in as dense an ambient medium as that of
ordinary life. So the best approach would be to start with ordinary matter and real
fleshy people in common space, and judiciously augment the everyday matter with
just enough computational matter to give the event a strange and marvelous cast. This
approach, which I nickname "minimax" design (maximum experiential impact for
minimum computational technology), resonates with Grotowskian poor theater's
choice of a minimalist technology of mise-en-scene relative to cinema, a minimalism
which in fact is constitutive of its magic.26
The apparent inefficiency of such highly engineered virtual reality environments
is in fact endemic not only to "bottom-up" simulations but to all simu�tions. As
Maturana and Varela pointed out, to be as dense as life, a simulation of an aJtopoietic
system can never operate any faster than that autopoietic system, and can at best run
at the speed of life. So much for the cybernetic fantasy of mastering and replacing the
lifeworld by a transcendental, superior simulation of life.

Rhythm as Structured light = Calligraphic Video + Theatrical lighting + Domestic

lighting
Classically, typography differs from calligraphy in two important respects. In a typo­
graphic alphabet, every instance of a letter form is mechanically identical, whereas a
232 C h a pter 7

manually lettered form varies with every instance: identity is a limit rather than the
instance. Also, typography is mechanically produced, whereas calligraphy is manually
performed, and in fact breathed. Although it is not often taught explicitly, a calligra­
pher learns to time the rhythm of her breathing to the strokes, and so the pace and
meter of her breath echo in the inked letter forms.
Calligraphic video refers to computationally synthesized moving image that fluidly
responds to gesture. I suggested that video should be taken out of the screen, out of
the box, treated not as a framed obj ect but as structured light projected onto physical
bodies. Different species of structured light have distinct textures and latent responses
to perturbation. My approach was to borrow from computational physics and treat
each frame of video as the initial data for a lattice computation approximating the
dynamical equations for some physical substances. For example, Yoichiro Serita carried
out this idea treating video with the heat and wave operators. (See figure 7 . 7, showing
calligraphic video as used in the IL Y A video membrane.)
Using these computational lattices that numerically simulate the dynamical equa­
tions of physical materials imbues the video with analogous responsive qualities.
Projecting such video textures allows people bathing in such structured light to
nuance the video textures as if they were physical material. This leverages the deeply
embedded intuition that we lay down from infancy with the materials of the world,
an intuition that may not be articulated in any language. Moreover, this lattice com­
putation scales well with increasing numbers of people. In fact the computational
complexity is O(number of lattice cells), but is essentially a constant with respect to
the number of people in the field represented by the lattice. This is the same strategy
followed by astronomers, who model galaxies with tens and hundreds of thousands
of stars by smoothing the distribution of stars into a continuous distribution density,
and then model the physical motion by hydrodynamical equations of a plenum
rather than the combinatorially intractable, multidimensional graph of pairwise­
interacting particles.
A low-resolution display is a set of isolated "pixels" spatially scattered across a space
in some arbitrary, though sometimes intentionally determined, pattern. These so-called
pixels could be the set of all the LEDs winking in your house, or a set of ceiling fans,
or sewn in strands of glowing fiber sewn into a carpet.
What are needed are abstractions to operate these low-resolution displays, not as
regular rectangular arrays of a million pixels, but amorphous, scattered, and shifting
sets of isolated flecks of light, puffs in air, or curling polymer. Whatever the medium,
these sets can be articulated more expressively by the extensive concepts and opera­
tions of measure theory than by classical geometrical concepts and operators. The
richest descriptions we have of such pOint sets draw not from geometry (whether
Riemannian or otherwise) but from measure theory. There exists computational
P ractices: Apparatus and Atelier 233

support for this at a primitive level, but not yet incorporating the more expressive
mathematical articulations that we would needY

Rhythm as Gestural Sound

Similarly, sound can be nuanced, rather than triggered as playbacks of fixed recordings
by the movement of a body or a thing. Typically, in an "interactive" installation, a
specific movement by the human triggers the playback of a recorded sound. Even
though the sensing of the movement may be a sweep of the hand through a laser
beam or some more exotic mediation, the result is still limited to the playback of a
piece of sound, whose audible quality does not vary in the course of the playback
from the way it was edited. This treats sound as an object-a compact body in naive
space-time, which is often how a visually oriented artist first approaches the use of
sound. Yet another symptom of a visualized and object-oriented approach to sound
is heavyweight sound "spatialization" systems that focus on placing sound objects in
a virtual three-dimensional space, or flying them along spatial trajectories as if they
were physical compact bodies rather than rhythms permeating a continuous material
field. However, a sound that is synthesized or varied over the course of its running
time according to some parameters fed by the nuance of a live, coordinating move­
ment can be gesturally modulated. Think of a piano versus a violin. After the piano's
hammer falls, the note resounds in a way that is pretty much independent of how
the pianist lifts her or his hand after removing it from the key. Because the violinist
maintains continuous contact with the wood and strings and bow, however, she or
he continuously nuances the note as it is sounded. This continuity of contact with
gesture and consequent continuity of the nuancing of the sound characterizes what
I call gestural sound, as contrasted with triggered sound. Gestural sound is analogous
to calligraphic video.z8
One approach to creating electroacoustic instruments is simply to electronically
mimic the analog instrument, with momentarily amusing results. Another is to mime
the forms of performance, for example to create an "opera" or a "symphony" or a rap
performance, but with electronic instruments in place of conventional ones. While
this may comfort those who need to measure progress by triangulating from shore­
bound landmarks, these imitations strongly bind the creation of fresh sonic experi­
ence. Fifty years after John Cage oriented us to the experience of "organized sound, "
we need not simply repeat the instruments or the genres of sonic practice as we
explore acoustically mediated experience, except as exercises to build confidence in
technique.
My approach has been to treat sound as a medium for exploring distributed
agency, superposition, and alinguistic time-saturated patterns. Creating "instruments"
that map movement to sound permits my collaborators and me to experiment with
234 C h a pter 7

Figure 7.8
Navid Navab at a sonified table, Topological Media Lab. Photo by Jerome Delapierre, 2012.

intentionality, agency, and affect via gesture. Concretely this has demanded creating
sound synthesis instruments that are not mimetic of any conventional instrument.
Why only imitate something that is already done quite well and far more subtly,
sustained by centuries of performance practice and social embedding? Thinking of
sound as substrate instead of well-schematized patterns implies making sounds rather
than music or speech. Even sonic patterns that happen to be those of speech or music
can be treated as texture rather than semantic representations, with fresh results.29

Rhythm as M ovement: Dance, Movement Art

As Arakawa and Gins proposed in their provocative book A rchitectural Body, the body
finds its limits where attention alights. We can pose this more generally as a phenom­
enological problem of how the body is conjured in the wearing of a space's material.

S u m mary

Across all these streams of inquiry, we study how to richly condition streams of media
in concert with the activities of the people in a common built environment, such that
the activities (to use an overly anthropocentric term) of the media and the people
Practices: Apparatus and Atelier 235

co structure one another and evolve over time according to prearranged strategies and
latent predilections, contingent activity, and memory of past activity. I appeal to
continuous dynamical systems on several grounds:
( 1 ) People's experience of the world is continuous.
(2) People have sedimented huge amounts of experience with the physical world, so
we should leverage it by using quasi-physics models.
(3) I wished to see how we could move away from egocentric and anthropocentric
design.

The Atelier-lab as a Transversal M achine

On the macro scale of sociodynamics and cultural-epistemic diffusion, my long-term

interest in the TGarden and its sibling responsive play spaces extends beyond the
actual events themselves to the mixing of ideas and conflicting ideological commit­
ments from different epistemic cultures. It could be liberating to practice our arts and
sciences in a more reflexive way. 30
Two decades ago, Felix Guattari pointed to the heterogeneous machines around
us: material, semiotic/diagrammatic/algorithmic, corporeal, mental/representational/
informatic, libidinal/affective. Guattari's Chaosmosis asked how we could construct
machines that act transversally across those machines. In the decade since 2001 , the
Topological Media Lab31 has been working as an atelier-laboratory transversal to com­
puter science, performing arts, and architecture and the built environment, to generate
insights and techniques in the domain of new media and responsive environments.
The atelier was motivated by the questions: How can ordinary actions in everyday
environments acquire symbolic charge? What makes some environments enlivening
and others deadening? Reflexively, we ask: To what extent can we instantiate labs or
ateliers for the creation of apparatuses for ethico-aesthetic improvisation? This section
describes institutional, sociotechnical, political and economic issues around running
such an atelier-laboratory as an alternative social economy complementary to postin­
dustrial, "knowledge-based" economies.
The big methodological moves are (1) avoiding a priori schema, (2) working with
material, collective, environmental situations, and (3) moving from nouns to verbs,
from things to transformations of things. This includes, by reflexivity, moving from
working with fixed (Le., transcendentalized) concepts to putting concepts in play. It
motivates an approach via ontogenesis. Staying close to the material and collective­
environmental implies making "thick" experiments in the "wild. "
The TML prototypes what I call an atelier-lab, an open space in which affiliates can
pursue art research without having to constantly defend individual projects in the
institutional language of disciplines and granting agencies, or in terms of the market.
236 Chapter 7

The atelier-lab's raison d'etre is not to be a facility for the production of art, although
it does produce media art and software.32
It is not a studio warehousing gear and technicians. It is a lab, building apparatuses
impelled by certain streams of inquiry. But it is not an engineering research lab, even
though it accesses the technical expertise necessary to invent solutions to any required
technical and mathematical depth. Its engineering results are published in the relevant
professional contexts. It can produce media/installation/movement art and techno­
scientific work that are legible and valued in their home disciplines, which is different
from asking that these artifacts be evaluated as "interdisciplinary" work according to
some less rigorous standard.

Stream s of I nquiry
Like other interdisciplinary "labs, " the atelier-lab is a transversal machine for the
production of knowledge. However, it differs from many in two respects: ( 1 ) it meets
disciplines not in a pOint, but thickly; (2) it provides a place for its affiliates to reorient
their approach to their production of art and knowledge.
Although it adopts no homogeneous method or discipline, the TML in particular
approaches process-based articulation from the perspective of continuous, material
experience. Its methods generally sidestep models or representations, because it is
concerned with noncognitivist experience33 and unbifurcated ontology.34 Its philoso­
phy of technology draws on continuum; it processes intuitions rather than algebraic
schemas and discrete states.
Although the atelier-lab is not a production facility for works of art, it does create
poetic installation-events as a side effect of its research.35
People come to the atelier-lab adept in some technique, such as video editing, real­
time sound programming, real-time video programming, physical computing, archi­
tecture, dance, or interpreting Gilles Simondon and Felix Guattari. The atelier-lab
hosts apprentices and expert practitioners to realize experiments reqUiring collective
effort, drawing from the practices of the art studio, the engineering laboratory, or the
preindustrial atelier. Two mottoes flexibly inform the work: ( 1 ) Minimax-maximum
experiential impact for minimum technology inserted into a situation; and (2) Art all
the way down-crossing the boundary between art and craft to open up black-boxed
technology, to expose and rework conceptual framing assumptions that are normally
tacit when used off-the-shelf by artists, scholars, or scientists. Even naturalized pro­
cesses like physics of materials, computation, and social structure may be put in play.

Art Research versus Art Practice

What I s Research in Conte m porary Art?

Research in the arts is quite different from research in engineering, which in turn is
different from scientific research. It is more akin to the humanities in its attention to
Practices: Apparatus and Ate l i e r 237

the particular rather than the systemic. However, it creates knowledge via aesthetic as
well as critical inquiry, and engages material and embodied experience as well as
concepts.
Like other modes of research, art research generates portable knowledge: insights
and how-to's learned in the context of one art research project can be applied in a
different one by another artist. Like research in other domains, art research has its
own archive; but whereas historians use textual archives and anthropologists materials
gathered in fieldwork, art research's "body of literature" is the corpus of prior works
and the critical commentaries surrounding them. Like other research, art research is
open-ended: one cannot declare in advance what the "deliverable" will be. If one
already knew the answer, one would not need to do the research.
Art research is not the same as art practice. Not every artist shares her working
knowledge with her peers, nor need she. Art practices range widely. A large part of
their vitality comes from their autonomous ways of making.
In art research, experienced artists mentor less experienced artists as potential peers,
not just as hired hands. Art research is reflection upon practice. Its fruits are not pre­
sented in galleries, theaters, or other exhibition venues, nor are they directly or neces­
sarily for art production. Mentoring in art research has the quality of individual
mentoring in the humanities. Art research generates questions, opens up frames of
reference, and rigorously investigates questions concerning value (as opposed to fact),
desire, and imagination, questions that transform or break genre and even the frame
of art.
Art research can amplify social and cultural commentary but along aesthetic and
poetiC as well as critical dimensions. Rather than promoting a particular methodology,
it can draw general knowledge from the creation of things or events. An invaluable
aspect of art research is that it rigorously investigates the cultural and human imagi­
nary in the way that philosophy investigates social and individual knowledge: by
constructing precise and memorable questions about what may perhaps have been
taken for granted.

What Is Practice in Contemporary Art?

The strength of contemporary art practice lies in how diversely art is made today.
Contemporary art practice includes (1) obj ect making by artisans based on inherited
folk knowledge, (2) commercial work in all media by professionals in global or global­
izing markets, (3) objects and performances created for galleries, private collections,
or venues sponsored largely by private wealth, (4) works subsidized by public money
for public display, (5) nonreproducible events or objects made by collectives for par­
ticular places and situations . . . I focus here on North American art practices situated
in contemporary economies. To practice art necessarily includes the question of how
one makes a living doing such creative cultural work. The situation in Quebec and
Canada has unique qualities.
238 C h a pter 7

Artists work as individuals, in collectives, or in institutions like universities. The

academy brings both benefit and handicap. On the one hand, it provides shelter from
the commodity market and the imperative of adhering to conventional forms of art
or ways of making art; on the other hand, institutional funding norms and perfor­
mance metrics based on publication and exhibition can leaden the work.
Some North American artists make a living by selling their own work; others by
work related to craft expertise in service to the design of other creative projects; still
others from work largely unrelated to the skills relevant to their art practice. Some
more established artists incorporate and direct teams of executants or subcontractors,
and may not be engaged in material production themselves. Their work is largely
conceptual, directorial, and managerial. Some of the more prominent artists work
essentially on the model of for-profit corporations. Some artists choose to work in
collectives or networks outside the formal economy and often outside North American
or western European cultural spheres. These collectives exist in a hybrid of gift and
public-subsidy economies. Some artists do not make the majority of their income from
their art work. They may be employed in public cultural institutions, advertising,
schools, or in service sector j obs. In every era, artists whose experimental practices do
not fit into institutional categories of art do not expect to sustain themselves from
their work.
Many artists in Montreal and Quebec show their work in conventional galleries,
thanks to provincial and federal support and to an active community of curators and
audiences. Academic artists also diffuse their work via teaching, writing, and curating.
They participate in an alternative cultural economy, reflecting on personal work and
exchanging insights with academics and creative practitioners in neighboring
disciplines.
A large portion of art is produced by people who may not identify themselves as
artists.36

Public Institutional Context: Research Creation in Quebec, Canada

In 2000, the province of Quebec's Fund for Research in Society and Culture (FQRSC)
pioneered a program to fund research by university-based artists on an equal footing
with sciences and humanities. The FQRSC called the activity that it intended to fund
recherche-creation, which it characterized as:

les activites ou demarches de recherche favorisant la creation ou l'interpretation d'oeuvres lit­

teraires ou artistiques de quelque type que ce soit. Dans Ie cadre de ce programme, I'interpretation
est analogue a la creation et ne peut etre comprise comme une demarche intellectuelle d'analyse
d'une oeuvre ou des realisations d'un createur.37

This program later inspired a national program (2004-2007) at the Canadian Social
Sciences and Humanities Research Council (SSHRC) . The SSHRC had a programmatic
motivation for its research-creation fund:
Practices: Apparatus and Atelier 239

Alternatives: An environmental scan was conducted to identify similar programs in Canada and
abroad. Aside from initiatives by the Fonds quebecois de la recherche sur la societe et la culture,
there is no comparable program in terms of total investments in research/creation projects
( $ 1 3 . 4 million), size (an award value of up to $250,000 per project), scope (nearly 1 00 individuals
from a wide range of artistic disdplines funded during the five-year pilot phase), and tenure of
funding (three years) . Survey responses echoed the lack of comparables, but dted provindal
government, university, and federal government sources as potential (though not equivalent)
resources.38

The Topological Media Lab in Montreal has become home to a rich nexus that over
six years has sustained some 60 graduate students, artists, and scholars working in
creative research that may be described as transversal (in Deleuzian terms) or refractive
(in Karen Barad's) . The TML is unique in its equally strong emphasis on depth in three
areas of practice: text-based experimental and speculative philosophy, artistic/poetic
expression, and technical/engineering craft. Most laboratories or studios still privilege
either textual readings, engineering, or studio art.39

People
How do people affiliate with an atelier-lab? Not being part of any department, the
Topological Media Lab does not offer courses. Students come by word of mouth from
all disciplines: computational media arts, electroacoustics, fiber arts, philosophy, cul­
tural studies, communication, anthropology, architecture, design, French or English
literature, computer science, electrical engineering. Affiliates enter at all levels: as
undergraduate students, master's students, PhDs, postdoctoral fellows, experienced
scholars or artists.
Students come once their classroom studies are largely over. However, exceptional
students have entered at the beginning of their undergraduate studies. For some, the
atelier serves as an oasis in a program, discipline, or world that seemingly has no place
for them.
Others come after a first career whose institutional or disciplinary bounds they wish
to exceed, attracted by productions and documentation that demonstrate the aesthetic
and genre-croSSing work of the atelier.40 Researchers with ethnographic int�est in the
TML's ecology of practices have taken up multiyear residenciesY The observed regards
the observer.
Prospective affiliates are told that this is not a short affiliation. It takes a year to
get to know people, to know oneself in the milieu, to understand not just how to but
why to do things a certain way.

M odes of Understanding i n Practice

Over time, different modes of understanding in practice have become important for
the sort of art research that the atelier-lab sustains: unlearning, apprenticing, taking
240 C h a pter 7

a leading role, directing, writing. Not everyone experiences these modes of under­
standing in the same sequence.
Unlearning involves relinquishing conceptual frames: letting go of cognitivism, the
computational equivalence42 conceit, commodity and utility rationales, market logic
predicated on scarcity calculus, or networks, as adequate models or topoiY It also
implies unlearning practices: letting go of solipsistic habits, procedural programming,
and ego therapy or ego announcement, as modes of art practice.
Apprenticing means working as a volunteer on experiments or projects defined by
faculty or experienced affiliates. Some exercises may be defined as stand-alone projects
on which no TML research or production depends critically. The atelier-lab assumes
that people arrive with some expertise. After some preliminary orienting work, a
person may act as an understudy to an experienced affiliate responsible for a particular
aspect of a collective project (a "leading role"). Every core creator (developer, in engi­
neering ontology) in the atelier-lab should be able to cover for another via j ob rotation,
and develop the capacity to work collectively in depth. Through their apprenticeship,
affiliates learn how to move beyond single-ego expression, as they absorb the accu­
mulated atelier-lab's knowledge and technique.
This does not always work in practice. The gap between talking about a work ethos
and effective practice cannot be bridged by merely assembling sociologists and artists.
The atelier-lab's respect for expertise is coherent with a working ethos of not disciplin­
ing the apprentice to act in a transdisciplinary way. What is required is mentoring,
which must constantly adjust to varying research contexts, individual aspirations, and
microsocial dynamics . This is an inefficient way to produce artifacts, but a good way
to produce persons.
An affiliate who has demonstrated commitment and reliability takes on a leading
role in collective projects. Some become resident experts in techniques codeveloped
with the experimental apparatus.
There is a significant jump to the responsibilities of directing and writing. Directing
means inventing some research question or a vision that exceeds any one project, but
also having some idea of how it might be realized. It is difficult to discover a vision
that is not merely derivative, a permutation-what Georgina Born calls decorative "
II

art-science. It is even more challenging to find questions and voicings that evoke and
warrant collective attention and energy. A director enrolls fellow creators to help
realize his or her experiment or work. Discussions with experienced members of the
atelier-lab and its Director determine whether a particular proj ect constitutes an
atelier-lab's proj ect, drawing on collective energy, material, and capital resources.
Writing and publishing is another mode of understanding that fits uneasily with art
practice and art-based graduate studies. There is no easy road leading from the inven­
tion, design, and development of apparatus or experiment to critical and theoretical
discourse in writing.
P ractices: Apparatus and Atelier 241

Synchronic Description of Roles

Over the past ten years four prototypical roles have evolved: apprentices (newbies),
research assistants, research staff (researchers), and peer artists or scholars. Apprentices
are often undergraduate students who volunteer in an existing project. Unlike fabrica­
tion work for studio artists, art research constitutes not merely a set of work-for-hire
relations, but an intellectual, affective, and symbolic ecology. Research staff are paid
to maintain and build group knowledge across generations of students and projects.
Graduate research assistants design, manage, and carry out research projects. Experi­
enced affiliates take the initiative to deepen and extend specific research responding
to the general themes of the atelier-lab. They work on each other's projects and take
turns directing work. Mature artists or scholars propose proj ects to be hosted at the
atelier-lab. They form an international network sharing an ethos, aesthetics, and
philosophical inquiries that substantiate and extend the atelier-lab's research themes.
The atelier-lab partly models theater production, in which all the crafts combine for
the purpose of making an event or an installation.
The model includes heterogeneous practices and an explicit set of roles ranging
from gofer and understudy to master craftsperson (e.g., in costume or lighting design)
and event designer (e.g., directors, composers, and choreographer) .
Between 2001 and 201 1 , the TML hosted more than 1 00 "highly qualified profes­
sionals" (HQP), interdisciplinary PhD, master's, MFA, and undergraduate students.
However, institutional class does not correlate neatly with roles in the atelier-lab.
Whereas roles in projects are legible to funders, less project-oriented work-say, an
MFA's poetic occupation of space using DIY electronics, or a doctoral collective's read­
ings of philosophy-is hard to sustain, and is perpetually distracted by funded projects
or external commissions. What gets privileged by projects normed along engineering
or some social scientific methodologies is work that can be cast into problem, method,
solution, rather than poetic or rigorous investigation.

What Does an Atelier-lab Produ ce?

Although the atelier-lab produces media works and software, its primary "products"
are experiments (events) and more experienced people. It's inadequate to think of the
atelier-lab in terms of projects instead of areas of inquiry motivated by synthetic ques­
tions or visions. These streams of inquiry are patterned by seminar and reading groups
and visitors.44
Organizing activities around evolving yet historically connected areas of inquiry,
rather than being driven by calls for proposals, avoids Brownian-motion jumps from
application to application, or being herded toward the mediocre centers of the Gauss­
ian distributions that reflect social, academic, industrial, or cultural fashion. Projects
like the TML's WYSIWYG sonic weaving, Ouija intentional and collective movement
experiment, and Memory + Place seminar/experiment exemplify what I call thick
242 C h a pter 7

F i g u re 7.9
Plant Life Support System in the Topological Media Lab. Illustration by Jane Tingley.

transversality, in which the intersection between the project and a given discipline on
one hand is a substantial constituent of the project and, on the other, represents a
substantial contribution in the intersected disciplines.
Projects often culminate in, or are punctuated by, workshops4s with internal team
members, the atelier-lab's local constituency in the university and the city, and prac­
titioners and theorists from around the world.

Political Economy, Knowledge and Reputation Capitals

The TML is not a production facility for individual art proj ects: it is a place for build­
ing sketches and experiments with larger ambition and impact, which requires the
collective talent, expertise, and energy of a small team.

Building Social Capital

The " N + 1 /1 ethic: A small group of N peers aspires to create N + 1 works, in each of which
the collaborators are by turn the conceptual lead/responsible agent, and including at least
one additional project that the collaborators would not have accomplished on their own.
Citation: Scholarly communities over the past 700 years have developed a precise
and refined citation practice to trace their flow of works and acts of imagination by
name and moment in a non-Maussian gift economy. Knowledge circulates and grows
in these small moments and denominations: the phrase, the paragraph, three lines of
handy code, a fabric switch materializing a different thought about connection, the
hypothesis of a theorem, one video segment providing the texture to seed a synthesis.46
Practices: Apparatus a n d Atelier 243

Affiliates credit peers and mentors for the ideas they offer or the prototypes they
demonstrate. Acknowledgments will be scaled to the significance of the contribution
relative to the final work. I require practitioners to adopt the citation practice of the
scholarly community. This is not an argument for or against the notion of originality;
it is about rewarding and publicly acknowledging help and, conversely, being tangibly
rewarded, in terms of social capital, for giving knowledge to a colleagueY
Coauthoring papers simultaneously constitutes deeper critical engagement between
mentor and student, and rewards all parties with public credit.

Dissonances and Assonances

Art research is not art production. Atelier-lab research investigates phenomena and philo­
sophical questions, in the mode of art "all the way down"-all iayers of "craft" are
subject to artistic interrogation. Experiments are installation-events to be encountered
corporeally. They should be built not only to standards of engineering or scientific
lab work, but to the much more stringent standard of the performing arts: when it is
show time, no one can wait for the computer to reboot. In-house installation-events
that are regarded as experiments are not built for an art gallery or a theater. They are
built neither for an audience (an epideixic relation) nor to determine an empirical
truth (an apodeixic relation of sorts) . This surfaces another ethical stance of the work
of the atelier-lab: we creators should (wish to) inhabit what we create for others. The
erasure of distinctions between composer, performer, and spectator as bodied agents
is not only an academic, analytic, or political issue but an ethical one as well.
Engineering research is not technology development. Unlike many applied engineering
or tech-art labs, the TML starts with art research questions and philosophical ques­
tions, and then tries to build the apparatus in which to experimentally explore those
questions. Practical questions emerge out of making installations that in turn are
inspired by conceptual and ethico-aesthetic aspirations. Some of those practical ques­
tions or problems yield conceptual and frame questions that exceed the particular
event. These become themes that can motivate long-running research initiatives. The
atelier-lab does not take a given piece of technology from the market and then try to
find artistic applications for it, as defined by commodity markets. Instea�, it works
more like a low-budget version of a high-energy physics lab, in which fundamental
research questions motivate the improvisation of new technologies that in turn inflect
theories and experiments.
Contrary to how many humanists and artists understand engineering as an aca­
demic discipline, it does not reduce to coding, or soldering sensors into microproces­
sors, any more than painting reduces simply to mixing pigments. Engineering research
includes systematic familiarization with the literature and with contemporary disci­
plinary practice; finding new methods, new algorithms, new configurations of devices,
new theories; and finding skilled people to solve problems in their discipline. The sort
244 C h a pter 7

of basic programming needed by other labs can and should be done by exceptional
undergraduate students, but the real work is to design the behavior of machines, to
understand something.
Project � experiment � design � line of inquiry. Unlike a stream of inquiry, projects
have definite scope, beginning and ending dates, audience, a deliverable. Projects are
expected to succeed. Usually a project has a team whose members are formally identi­
fied and charged with specific areas of responsibility. In an experiment, a negative or
dis confirming result is just as useful as a positive one. Design does not have to have
a hypothesis generating knowledge. A stream of inquiry may endure for as long as the
question remains vitally unanswered: streams persist across generations of students.
Projects define, periodize, chapterize the work, sometimes as an artifact of competitive
funding programs and grant cycles. Sometimes what starts as a project may become
a (set of) research line(s) . There is no average case; no hard-and-fast category. They
evolve as the atelier-lab evolves.48
"Accountability" and "transparency. " As Born and Barry point out, the logic of
accountability can often lead to less, not more, innovation or creativity.49 A university
that only a generation ago was structured as a teaching institution for the working
class had a bureaucratic machinery inadequate to competing for and administering
public grants. Its response to the logics of accountability and transparency led to a
metastasis of internal micro-oversight and a highly inefficient distribution of book­
keeping work to individual researchers, and even graduate students. Lacking experi­
ence with administration of research, the university did not keep an adequate amount
of overhead for central services and infrastructure. Consequently, research staff and
research dollars were squandered, when information was in fact centrally available
and could have been managed centrally. This has led to unsustainable demands for
hyperdocumentation, in which for every dollar spent on a research grant, another
dollar is spent on metadocumentation.
Throwing money at academic artists without adequate understanding among
faculty about what new modes of work are afforded by "research creation" yields only
larger studios and more hired hands doing work-for-hire work. More monitoring and
reporting does not yield deeper, more innovative, or more life-changing or practice­
changing work. Refining Born and Barry's analysis of the logics of interdisciplinarity,
there is accountability to private sponsors, as well as accountability to the public in
the guise of the state. These accountabilities have quite different form. Accounting
measures as presently instituted can readily measure only formal features of the work,
such as numbers of presentations at specific venues or professional societies. The same
limitation holds for measures of people. To be valued, every professor in an art school
need not be a paragon studio artist and teacher and researcher. The ostensibly neutral
administrative practice of adding to the criteria of teaching and learning a criterion
called "research" encourages faculty to further divide their energies, and mimic alien
modes of practice. Under perceived pressure when the administration declares that
Practices: Apparatus and Atelier 245

the strategic function of the university includes research, an artist acting according to
some externalist conception of a "researcher"-whether modeled on imagined labora­
tory science or, less commonly, on the imagined humanities-may willfully detach a
label for a concept from its context of practice and scholarly genealogy but not rec­
ognize or respond to all the conceptual commitments and caveats attached to it,
because she or he is not familiar with the scholarly fields of discourse and practice
associated with that concept.
To sum up, much institutional friction drags research activity into merely going
through the mechanical motions of research activity---content-free "zombie research"­
a mechanical permutation of dematerialized, evacuated concepts, together with a
jumbling of technologies or technical works with no attention to epistemic frames.
Institutional program. A master's degree involves mastering a field and its literature,
not an extra year of programming. A PhD is not a longer, funded MFA. Nor should a
PhD modeled on the humanities target narrow, technically bounded research assis­
tantships or allow itself to be driven by grants. The lab should define its research
agenda around areas of inquiry. It welcomes affiliates to an open space in which to
develop and extend those areas, in coordination with their own growth. Projects and
proposals for projects emerge out of this. Out of the projects we develop proposals to
available funding bodies at the appropriate times. Hexagram, ClAM, and our home
institution's internal seed grants subsidize this most significant cambium of explor­
atory research. 50
Accountability to private sponsors. Premature attempts to convert research creation
into intellectual property or even commodity tend to muzzle the articulation of fresh
ideas. Hexagram members who were entrepreneurial artists or had some training in
engineering, but no actual experience in industry at a strategic level, advocated "intel­
lectual property" as a way to make the network successful in the eyes of its board
drawn from industry. The Hexagram research-creation network's "Funding Competi­
tion Evaluation Criteria" (2007) explained the foundation's context for research/
creation in the following terms:
Hexagram's mission is t o support innovation i n the field o f digital content throu �h research,
creation, training, experimentation, production and dissemination activities based �n original
forms of communication and expression driven by new technologies. In short, the Institute's
mission is to stimulate and enhance research and creation artistic activities using new technolo­
gies . . . [pooling] a critical mass of university researchers . . . . Hexagram also serves as a bridge
between university research and a variety of non-university-related communities and sectors; it
promotes the transfer of research in new media and technology at the local, national and inter­
national levels . . . . Hexagram's obj ectives are:

• To foster the development of content in media arts and technologies through systematic
exploration and experimentation using new technologies;
• To enable researcher/members and managers to . . . [discover] ways in which digital technology
can be used to enhance communication and artistic expression;
246 C h a pter 7

• To encourage contemporary artistic practices based on . . . needs . . . in . . . film and digital televi­
sion, interactive games, performing arts, and interactive multimedia (educational and cultural) . . . .

• To . . . train multidisciplinary professionals to understand . . . new media and manage innova­

tive projects in order to contribute to the development of an industry in this field;
• To stimulate interdisciplinary research and . . . collaboration between people from . . . artistic,
scientific, technological and engineering [fields] .51

Hexagram specifically justified its support from the private sector by valorizing com­
mercializability and the transfer of results to the commercial sector. However, it
defined the notions of commercialization and transfer extremely broadly (at least by
US standards) :
Enhancement (recognition and validation of the research) has a broad meaning within Hexa­
gram. Research is enhanced when its results are presented outside of the workshop or laboratory,
when they are commercialized, once the research is introduced and shared with the community,
when it becomes the basis of subsequent research or when the research is modified for com­
mercialization . . . . Some project components created in Hexagram can be used by other researcher/
members, organizations, outside individual users . . . or by companies . It can be code fragments,
material components . . . , specific technological expertise . . . , transposing an object which was
initially created for artistic purposes into a design product, tools or databases created . . . . The
transfer can be done directly, or through seminars (workshops, . . . ). [Each possibility] represents
an opportunity to network with outside partners.52

The "Hexagram Fund Guidelines" (2005-2006) laid out the criteria of innovation and
of transferability potential:
a) " Innovation" is defined to include:
• the development of an innovation or framework for artistic or technological content in rela­
tion to a specific technology . . . ;
• the creation of new tools (software, hardware, interfaces);
• a novel combination of existing technologies;
• the production of artistic content using one or more technologies that have never been used
in an artistic context; and the development of new uses for existing technologies.

"Transferability" will also be taken into account in project evaluation and is understood to mean:

• the degree to which research activities are aimed at meeting . . . identified needs in the sectors
targeted by [Hexagram]; or
• the degree to which the essential aspects of the project can be included and incorporated into
research in similar or different fields (inside or outside Hexagram and the university setting) .53

However, naive expectations of generating income from "intellectual property" and

patents ignored the lessons from the 1 990s and 2000s that most academic research,
even marked by patents, generated little profit for the host institutions. 54
Authorship. There is a structural conflict between the need to establish oneself and
the need for amplification of one's work by recruiting collective strength. The basic
P ractices: Apparatus and Atelier 247

disincentive to collective work is that institutions evaluate individuals on individual

CVs, not collectives.
The thermodynamic equilibrium problem. The patient building up of a rich practice,
thoughtful reorientation, and multipolar relations in a protected sociotechnical space
can be undone when the rate of diffusion between labs is high. When a student
achieves some technical mastery, she also becomes employable by neighboring or
copycat labs. Implementing a work-for-hire project for a research program with nor­
matizing thought is easy to justify by the student and the poacher. This happens when
the technically proficient student is just beginning to develop the judgment that
informs the radical ethico-aesthetic work of the atelier-lab.
The atelier-lab can avoid poaching by remaining under the radars of utility and
technocultural fashion. But the very norms informing the institutional policies gener­
ous enough to sustain substantial art research sustain poachers as well. Lateral migra­
tion to average labs gives average students good exposure to average practices and
normatizing conventions. However, this leads to a fragmentation of attention and
effort, making it almost impossible to realize substantial or substantially novel work.
The exposure to Gaussian normalization/normatization yields what I call a thermo­
dynamic equilibrium problem: too high a rate of diffusion of people and ideas across
organism boundaries makes it proportionately difficult to create significant, novel
forms that stand out against noise or uniformity.
The conundrum is that in order to attract funding one needs to hew to the Gauss­
ian mean and expose oneself to mass norms and the consequent thermodynamic
equilibrium problem. Since its transplantation to Montreal, the TML was able to
build-as a planned side effect-an apparatus for philosophically motivated ethico­
aesthetic experiment. This was validated in 2009 when a philosopher (David Morris)
approached the TML to collaboratively build a noncognitivist experiment on memory,
place, and identity, taking into account the substantial alternatives enabled by Merleau­
Ponty and Husserl. Using the transversal techniques available in the lab, the memory­
place group's researchers from philosophy and computational media are building an
experimental process more attuned to the phenomena and reflexively aware of how
apparatus and protocol can preschematize observation. This work is extremely slow,
and yields little glamour in the intersected disciplines.
When imitation leads to zombie work, or makes zombies of organisms-that-person
(to borrow from Arakawa and Gins), ss when the ratio of drones to dancing bees grows
too large, perhaps it is time to swarm. But to what fields and ecosystems should a hive
swarm? Perhaps we can ameliorate the thermodynamic diffusion problem by imagin­
ing and realizing weedier ecologies. There is little we can predict. Therefore we may
as well experiment and improvise in the maelstrom of intersecting ecologies, and
discover what works for ourselves. This may be a tactical attitude more akin to de
Certeau than to aikido.
8 Refrain

Sometimes, more often in recent years, I've taken to asking students and colleagues,
"Why do you do what you do?" Although that question is not the same as "Why do
we live?, " it is not unrelated, because I think how we live would be part of my own
response to the question of why we live. The quality of life is perhaps a more fruitful
question than the meaning of life, so popular in an earlier era, more enamored of
epistemology. It could be a phenomenological question about the experience of life,
but I would like to answer it in a poetic way in the context of contemporary and
emerging technologies of performance, where performance is construed generously
beyond the domains of performing and performance arts.
One may aspire to do philosophy in the mode of poetry again, a Laozi multiply
transposed. But didn't Plato throw out the poets from the Republic because they oper­
ated in the realm of the fictive imitative, thrice removed from the truth, and therefore
were not to be trusted with the proper affairs of the polis? I wrote this book as an
exercise in philosophy in the mode of art, trusting that it can be done, that it matters
not only what we say or do, but how we say or do it. I've wagered that both truth
effects and ethico-aesthetic passions can be accommodated in the same breath, the
way mathematicians construct truths. Mathematicians are not Scientists, because their
theorems do not claim anything about the " real" world. Therefore they do not write
under the sign of empirical truth. Mathematicians prove theorems true or false within
J�
propositional systems that they themselves construct. Therefore their co tructions
are works of imagination. Writing neither under the sign of truth nor of fiction, math­
ematicians create truths via imaginative processes that can be regarded as poetic
processes.
It is in this spirit that I propose exploring some questions refined from crude, con­
crete, and technical craft, refined over the years into what would typically be consid­
ered philosophical questions. But, together with a set of fellow artists, engineers, and
scholars, I have explored those questions via a hybrid of material and phenomenologi­
cal experiments that have been built in the Topological Media Lab and by affiliate art
groups, notably Sponge and FoAM.
250 C h a pter 8

Questions of craft can become refined into questions of philosophy, and questions
how become questions why as well. Questions of philosophy in turn can provide
heuristics, though never blueprints or methodologies, for craft. The most compelling
reason for refining technical challenges into philosophical questions is to accommo­
date value. Given that we can engineer A, B, or C, the question we ought to answer
first is why A, B, or C? Such an ambition places this project in the area of the critical
studies of media arts and technology. However, the proj ect of constructing a genealogy
of topological media embodies a more radical ambition, which is to produce matters
of value as well as matters of fact. To make sense of how we may approach the pro­
duction of matters of value occupied the central chapters of this book.
Recall that the topological is about the continuous, about proximity and connect­
edness without metric quantity. The topological is the second most primordial mode
of articulating the multitude of which Michel Serres writes in Genese. (Set theory is
more primordial, but too bare to be adequated to life, witness Badiou's herculean
efforts.) Topology is, in a strict sense, immeasurably richer than the graphs and net­
works favored by engineers and their social scientists! Topological media for me is a
set of working concepts, the simplest set of material and embodied articulations or
expressions that allows us to engage in speculative engineering, or philosophy as art,
and to slip the leg irons and manacles of grammar, syntax, finite symbol systems,
information and informatics, database schemas, rules and procedures. I claim that
topological media as an articulation of continuous matter permits us to relinquish a
priori obj ects, subjects, and egos and yet constitute value and novelty.
In this sense, I think of the material, corporeal, technical, experimental work with
the Topological Media Lab as an art practice, deeply informed by practices of engineer­
ing, mathematics, and philosophy and some lives of activism.
In 1 995, I formed a faculty seminar called the Interaction and Media Group at
Stanford's Humanities Center, with professionals and colleagues from the humanities
and a few sciences. One key aspect of some members' works since then has been to
construct conj ectures as environmental installation-events so that participants in our
conversation could encounter them palpably and speculatively in events that suspend
certain assumptions (a scientific attitude), yet retain all the affective density of lived
experience (an artistic attitude) .

From "What Is the H uman?" to "How to H u m a n ?"

In order to experimentally investigate and refashion the fold between nature and arti­
fice, signs and matter, ego and other, twelve years ago I wagered that we must create
a responsive medium as a continuous amalgam of computational and physical matter
that is accessible to our craft: projected light, organized sound and video, fabric, cho­
reographed bodies, speech, software. It was in order to build the apparatus and the
Refrai n 25 1

F i g u re 8. 1
Sentient ocean matter. Still from film Solaris, by Steven Soderbergh, 2002.

techniques but most importantly to mentor and host a new sort of experimentalist
that I formed the Topological Media Lab in 200 l .
What w e have put i n question are certain categories such a s technology-synony­
mous with digital or computer technology-the cogito, as well as the body. In order to
understand such ontological or phenomenological categories, our strategy has been
to experimentally transgress those categories' boundaries rather than assume them a
priori.
It may help to compare this with the modern investigation of intelligence. The
Enlightenment's formation coincided with a fascination with the boundaries of the
human, indexed by such quasi-objects as Wolfgang von Kempelen's chess-playing
automaton of 1 7 70, the Mechanical Turk. ! In the first age of the electronic computer,
one of the grand challenges computer scientists set for themselves was to build a
computer that could play chess better than any human. Such a specialized quest was
j ustified on the grounds that exceeding the cognitive limit of the human in this
dimension could yield insight into the extent and even the structure of human cogni­
tion engaged with this sort of puzzle solving. It is characteristic of contemporary
science to be singularly obsessed with cognition. And it seems that even in the less
explicitly anthropocentric work of software engineering, we tend to enshrine ourselves
in what we create. My alternative concern lies with the conditions of possibility for
ethico-aesthetic play: poiesis, to use an ancient word in sympathy with Felix Guattari's
chaosmosis and Isabelle Stenger's cosmopolitiques .
In a parallel but more substrate and materialist mode, I propose to bracket the
human in order to understand not so much the "what" but the "how" of human
experience: I would not ask "What is a human?" but, to borrow Ann Weinstone's
252 C h a pter 8

F i g u re 8.2
The Mechanical Turk, 1 7 70.

phrase, instead "How to human?" Sidestepping this "seW-absorption permits us to

address the question without tautology. How? My strategy has been to bracket, to put
in play, beliefs and disbeliefs about the body, synchronic structure, and objects.

Bracketing Body, G rammar, Cognition

One conventional limit of the human is the fleshy body, so let us put that in play.
But how could we bracket the body phenomenologically, and what are the conse­
quences of such a bracketing? To put the concept of the body in play is not to deny
or to hide the body2 but in fact to pay attention to its framing condition. The concept
"body, " of course, is motivated by my and your fleshy bodies. However, I do not
restrict it to humans and animals, but use it mindful of Whitehead's entities and actual
occasions.
In general I find it helpful to imagine the world, as I said earlier, not as a vacuum
raisined with corpuscles but as a plenum of varying density. With such a field-based
approach, the body becomes a local density whose boundary is implicitly and provi-
Refrai n 253

sionally defined by contingent anticipation or imagination and by the expectations

formed in the course of contingent performance.3 Of course it follows that these densi­
ties and boundaries vary over time, from moment to moment, and from disposition
to disposition.
A set of pedestrians' or dancers' limbs moving in tandem could form a body, as
could a group of voices momentarily syncopated. What we ought not assume, however,
is an invariant deterministic mapping from physiological data to metaphor. Although
an invariant mapping may be a necessary working notion for neurologists and lin­
guists and engineers, we need not and should not, as poets or as phenomenological
experimentalists, assume a discernible deterministic relation between physiological data
like heartbeat, galvanic skin response, or breathing rate and macroscopic aspects of a
performative event, like emotion, mood, or narrative entity. Pragmatically, what we
learn from neurophysiology and the principled scientific study of neural phenomena
is that the data are simply too complex and polyvalent to plausibly map to any simple
linguistic token of an emotion or some human behavioral state. A smile could correlate
with amusement, embarrassment, confusion, or the rictus of death. A spike in the nervous
signal of a muscle could correlate with an equally great variety of putative "causes. "
But beyond such pragmatic concerns, there is a more fundamental conceptual issue.
Such a mapping would be merely a trace of the physical other, which is not identical
and may have only accidental relation to the embodied phenomenal experience.
As an aside: It is true that an artist may intentionally impose a mapping, but the
art of a responsive environment such as what I have built with the Topological Media
Lab lies in the fashioning of a substrate, not merely a particular obj ect in a particular
event. Turning from art to experimental research, in order to unmoor ourselves from
preconceptions of body and embodiment and to free the actors' tissues from pre de­
signed "mappings" of cause and effect, our responsive environments should provide
extra modalities of media in addition to the ordinary tissue of the performer-player.
Put more strongly, a responsive environment could put in play the Newtonian distinc­
tion between living tissue and inert matter. For the present, the modalities are primar­
ily those of gesturally modulated light, sound, and sensor-augmented fabric. If you
move, your skin shrugs over the bones of your hand not in a dialogic response to your
action, but as the locus of intentional imagination fused with the physics of muscle
and bone. In the same way, we create our calligraphic video, sound, and fabric not as
pre carved masks or prosthetic devices, but as expressive tissue that can be charged and
recharged with latent, potential responsivities to gesture and movement. Continuity
of media and body, whether effected by techniques of camouflage and projection or
by haptics and sensors and active cloth, leaves open the boundary of the performing
body in the way that helps us as experimentalists in performance research to explore
j ust such a bracketing of the body.
254 C h a pter 8

Now, having suspended the body in this sense, what if we suspend the cogito, even
the subject as well? How could we do this? Since 1 99 7, my strategy has been to suspend
a technical reliance on representations that are built from linguistic structure (canoni­
cally rule-bound systems of lexicon, grammar, syntax) . This eliminates the majority
of games and so-called interactive art as apt apparatuses for playful ethico-aesthetic
research.
Deferring such representationalist presumptions and models allows us to see how
subjectivities emerge under the dynamics of copresent play, and what becomes of
agency. As composers of responsive environments, we can ask where we should locate
the causal agency of a human-machine system.
Our typical model of interaction has been of humans and their proxies engaging
in an action-reaction ping-pong. And interaction design, even in its most enlightened
mood, has been centered on the human (viz. " human-centered design"), as if we knew
what a human was, and where a human being ends and the rest of the world begins.
The chess-playing Mechanical Turk, in fact, turned out to be powered by a human
hidden inside the box. Three hundred years later, I suggest, engineering and computer
science operate largely on the same conceit of homunculi, of putting software proxies
for Homo sapiens into our machines, from the ENIAC to the fictive Hal 9000 in 200 1 ,
to the agents of Sim City and the customer call center's speech recognition program
that can interpret telephoned speech as well as John Searle's Chinese Box. This anthro­
pocentric conceit is not confined to engineering, of course. Look at Bill Viola's beauti­
ful series of video works, The Passions. If we really take seriously the challenge to pursue
art all the way down, and if we are willing to put in play, in suspension, all the puta­
tive atoms, objects, and subjects of the world, then I ask you this question: To whom
do you owe allegiance: Homo sapiens rex, or the world? How one responds to that
question has deep implications for how one works as an artist, a scholar, or a technical
individual (to use Gilbert Simondon's term) .
Apart from the totalizing and dematerializing power of the Judeo-Christian God
and of informatic and logico-linguistic schemas, essentially the only ethico-aesthetic
choice in the West has been to start with the self, with Homo sapiens. We witness the
disastrous global ecological and economic consequences of this choice. However,
given topology as a way, even a rigorous and precise way, to articulate living, nonde­
numerable, dense, nondimensional, open, infinite, and continuous matter, one has
the option of choosing the world instead. I use these adjectives precisely for their
intertwined technical and poetic values. But of course this is not a cure-all, a recipe
for success, just as Deleuze and Guattari warned us at the end of their chapter on
smooth and striated spaces: "Never believe that a smooth space will suffice to save
,,
US. 4 It's an approach to design, a way to imagine and think about living in the world,
how to shape experience, a disposition with respect to the world, rather than a meth­
odology or a technology.
Refrai n 255

Figure 8.3
Bill Viola, The Passions, video installation, 2003.

Political Economy, I n stitutions

Of course all this work does not take place in a vacuum. I accepted the Canada
Research Chair in New Media in 200S in order to transplant the Topological Media
Lab from a techno scientifically respectable institutional setting to a political economy
that could support a much more sustained and experimental practice of media arts,
what is more and more explicitly not art or technology but experimental philosophy
and philosophical experiment cast as installation-events.
One of our mottoes for the TML is "Art all the way down"-meaning not taking
for granted any black box around technology, and reserving the right to critically and
materially investigate any border between concept and craft, between art and engi­
neering. This implies being able to open up technologies to artistic intervention. As
technologies I include the algorithms, the choice of programming method, the hard­
ware, in our case the computational technologies underlying contemporary media.
But thanks to the atelier's institutional embedding in Concordia University's Hexagram
256 C h a pter 8

research network in Quebec, we have been experimenting with the institutional and
organizational structures as well.
Chapter 7 reprised some of the organizational and the external socioeconomic
conditions under which we have been able to pursue this work. Any experimental
work relying on technology, or any work that coarticulates emerging technologies,
especially our pervasive computational technologies, necessarily contends with politi­
cal economics of government funding agencies, corporations, and universities. That
is part of the framework that the lab has leveraged in order to build the apparatuses
that focus on how events may be charged by responsive media in contingent and
composed situations.

Apparatus, Responsive Environments

Turning from the institutional context to the experimental apparatus, I have built a
series of media choreography systems with the teaching/research atelier over the past
decade. Currently named Ozone, the gesture-sensing and media resynthesis system
produces responsive sound and video with behaviors that evolve in the course of play.
It enables composers to distribute agency in a much more fine-grained way through
the different components of the media architecture, but it evolves with the activity
of the human participants as well. Indeed, this mode of conditioning an environment
to coarticulate an event challenges media composers to relinquish fully a priori
control, to accommodate contingent activity of the participants yet shape event
potentials yielding experiences that feel more engaging than accident or pastiche.
In my view, one condition for an artistically compelling experience in a responsive
environment is that it should not induce puzzle-solving behavior. The mechanism
should be completely obviOUS, or completely transparent. I prefer to create installation­
events in which participants may have compelling experiences without having to
think about how everything works. This cognitive response has become almost inevi­
table among experienced consumers of interactive art, because that is how we have
come to expect to play with a machine. But puzzle solving is a poor substitute for
theater or any thick form of life. More fundamentally, puzzle solving ferociously rein­
scribes only cognitive acts, and a particularly reduced set of such acts at that. This
eliminates most games and rule-based "interactions" from our set of techniques for
building an experimental apparatus.
The TGarden is an example of a responsive environment in which people can play­
fully improvise gestures, and collectively or individually create affectively or symboli­
cally charged patterns out of fields of varying light, sound, fabric, or bodies. The media
synthesis processes develop continuously according to a field-theoretic, magical
physics without propositional logic, schema, or symbolic computation. The textured
Refrai n 257

video and sound fluidly evolve according to autonomous processes as well as in

response to the players' activities. The continuous shaping of the responsive media
follows definite, composed metaphorically described tendencies that give a character­
istic potential to the experience in a particular aesthetic, performative event. One
might say that the potential dynamics created by the composers of such a responsive
environment are a collective social gesture eliciting a collective response from the
ambient social world, rather than a predetermined set of calls and responses it la
Disney imagineering.
My point is that such a responsive environment can host more than an art
installation-event; together with its makers and players it can serve as an apparatus
for an experimental investigation of subjectivation, in Guattari's sense (see Chaosmo­
sis) . In order to conduct this exploration in the mode of experimental performance
research, we focus our attention on the amplification of metaphorical gestures by
copresent humans performing in a shared responsive medium imbued-by computa­
tional means-with alchemical, responsive properties.
Almost all the events for which we have been making our apparatus over the past
decade and a half share the following common conditions: ( 1 ) the participants are
copresent in the same physical space; in principle you can stop the action by touching
a coparticipant; (2) inhabitants improvise collectively or individually significant
gesture; (3) these gestures act continuously in a thick event that is richly augmented
by media, evolving in concert with activity.
The question has been how events in everyday settings can assume symbolic charge
of heightened theatrical events. But in shifting from the explicitly marked theater, we
shift from explicit action sequences to conditions of possibility for topologies of
potential action.

A M aterials Science

Motivated by such situations, I ask "How to human?" but in what I term a more
substrate material mode. In fact, what is suspended is not just bodies but subjects and
objects generally, all part of a century-long resistance to an ever-present tendency to
make the immanent and contingent appear fixed and transcendental, to make humans
god. Put positively, my approach recognizes objects but focuses attention on the pro­
cesses by which they come to be. But these processes are material, hence the attention
to substrate.
Indeed, I sometimes characterize the empirical practice of the Topological Media
Lab as a form of materials science, taking the term into much deeper waters. Adopting
the more modest spirit of making a textile rather than a j acket, one can ask what
would play the analogous role of X in the following relations:
258 C h a pter 8

JACKET : TEXTILE : : PERFORMANCE-EVENT: X

In the case of sound, what is the substrate, or stuff, X? What is the stuff of gesture?
More radically, what is the stuff of event?
I claim that the stuff of event in our present state of technology would have to be
material which for our working purposes I characterize as the amalgam of matter +
energy + affect, the hybrid, dynamical, responsive fields out of which particular narra­
tive objects and event sequences emerge. These fields constitute what I call the sub­
strate. The TGarden technology constitutes not a particular sense making (gestures),
nor an event action sequence like a stage play or a game, nor even a generalized lan­
guage, but the substrate to a continuous range of performance.
I should emphasize that I do not wish to use " substrate" in its ordinary sense of
being prior to or more foundational than its objects or events, but in the sense of the
physics of fields. The substrate is constitutive of the objects and events that form in
it; in other words, the substrate and its contingent objects occupy the same ontological
stratum. So objects do not emerge out of the substrate, objects emerge in it. The sub­
strate is immanent in its dynamically forming and dissolving objects.s What this offers
performance is an alchemical technology for poetic matter. Such technologies of, for
example, gesturally nuanced real-time video and sound synthesis, and of responsive,
sensate and luminous electronic fabrics comprise contemporary amplifications of the
technologies not of representation but of performance.6
What is the medium of gesture in this extended dynamical setting but continuous
and open material, that is, a topological medium?7 We use topological media not to
represent some abstraction, but as the substrate of performance and physical action
itself, an expressive tissue amalgamated from gesturing flesh and resynthesized video
and sound. Where Grotowski challenged actors to use their own bodies as their expres­
sive medium, in studio-laboratory work I take as my challenge creating computation­
ally mediated matter for expressive presentation.8 Analytic sciences and philosophy
may be less attuned to this nonrepresentational use of matter because matter, whether
ink and paper or fabric, has tended to be regarded as part of dumb nature, the object
of mere craft (not art) . Literary theory and until recently cultural studies might gain
analytic purchase on matter only so far as it could be traced as linguistically signifying
matter.9
Matter, topologically construed and topologically constituted, may serve as the
substrate of poetic expression.

M aterial Practice

Can the material process of making things collectively be radically non denumerable,
countless, noncomputable, nondimensional, infinite, and yet remain also immanent,
Refrai n 259

embodied, and continuous? Can we make play spaces that evoke not puzzle-solving
behavior but sustained ethico-aesthetic play, and marvel, vertigo, or elation? Yes, in
fact, if we make structures and obj ects contingently out of the material substrate,
activated by dynamics sensitive to fields of intensities and tensions such as what I
suggested in chapter 4 or others yet to be discovered. The material fields I provide a
way to shape are both continuous (topological) and dynamic, saturated with and
constituting time.
To respect the open, unbounded lifeworld, such a space should not be useful or
therapeutic. In fact, that was Guattari's point about psychoanalysis, too, that its
purpose should not be to help the participant construct a narrative analogous to the
hermeneutic objective of classical psychoanalysis-"This is what the patient's phobias
/ psychoses / dreams mean"-nor to effect a cure-the "therapist's" analytic stance
with respect to his patient: "You are sick. We will fix you."
Why not just enclose a volume of ordinary space 'and repeat some experiments like
the action art of forty years ago? Why not pursue an Artaudian project with Grotow­
ski's via negativa of theater reduced to the fleshy but intensely disciplined body? With
such techniques, a responsive environment could be charged with latent magic, a
heightened potential for charging gestures with symbolic power. Such an environment
could become a theater for the alchemical ontogenesis of hybrid matter, not a space
for cognitive games, inducing puzzle-solving behavior, or simply a bath of raw qualia.
An alchemical theater would avoid having "users" and " system" building models of
each other. (In the human, such models would be cognitive models.)
As a personal practice, responding to a certain blend of American and Chinese
pragmatism, I try to pair negative critiques with constructive alternatives. So let me
offer a parable about topology in lieu of a full discussion which would be yet another
seminar series. Allow me to suggest a reverse allegory and use a piece of the world to
stand in for some concepts of the topological that I have introduced in this book.
Here is a patch of sod like the one that I cut out of the earth under a tree outside
the RIXC building in Riga, Latvia. Representations, words, are like blades of grass,
individually well formed, discrete. I can pull up this piece of sod and turn it over to
reveal the root structure underneath. Yes, there is a network of roots as we � n plainly
feel running our fingers through the dirt. However, I draw attention past the blades
of grass and their contingently formed roots to the dirt and the moisture in between
the roots. It's the continuous, nourishing, dark, loamy stuff in between the discrete
structures that materially constitutes the Earth. This moist earth is always and every­
where in continuous transformation. Our discrete structures, our words, syntax, gram­
mars and schemas and methodologies are the blades and at best the roots. And yes,
they are our best ways to grip the earth. But though they are a common supraindi­
vidual resource, they are not transcendental. They can only take form in and draw
meaning from the earth, and become earth when their life cycle is finished.
260 C h a pter 8

F i g u re 8.4
Grass SOD oF RHIZOME. Photo by the author.
Refrai n 261

Archimedes said, " Give me a place to stand, and I shall move the world. However,
II

what if there is no place to stand inside a bubbling chaosmotic soup of infinite infla­
tion? To what extent can we alchemically open and critically transform all of moder­
nity's black boxes-its conceptual fulcrums for good or ill-such as "market,"
"machine," or "human, " if we do not have a place to stand in this age of globalized
empire and permanent war? Is there any possibility of an immanent resistance for us
not as nondocile bodies, but as resistive and desiring tissue? Yes, I believe, yes, if we
take reality already as an amalgam of the potential and the actual, dematerializing,
for example by becoming fictive, and rematerializing under the incessant quickening
action of our imagination. This affords openings for life in the mud-filled interstices
of our technology.
All the modes of articulation I have suggested in this book come with their tech­
nological complements, just as certain modes of languaging come with their comple­
mentary grammar. Working with substrate enables us to work with material matter =

+ energy + affect without a point locus.

If we use "implication in the sense of enabling, this removes the necessity and the
II

imperative, replacing those with the sense of permitting and sustaining as partial
actions (analogous to partial obj ect) to be defined only in the event. (It is exactly in
this sense, for example, that we can modulate the experience of a potential state topol­
ogy in the Ozone media choreography system. The potential topology conditions but
does not determine actions or event sequences: there are no forced moves in the media
or by the participant. And the space of possibilities is open and dense.)
This combination of material substrate with a nondeterministic logic permits us to
work expressively in a textural, non-Archimedean way that can be " scale-independent. II

My mode of articulation for this is not the pattern of graphs, but the modes of con­
tinuity, proximity, density, and continuous transformation, in other words the pri­
mordial mode of (point set) topology.

Concerted Ontogenesis and Costructuration

Seeing in terms of substrate and dynamics recalls a motivation from a conference

attended by Stengers, James Williams, and a younger generation of scholars interested
in the relation between Deleuze and Whitehead: the possibility of theory adequate to
life. Any theory must accommodate dynamical, material patterning, Le., a material
ontogeneSis; in my processual interpretation of ontogenesis, the formation of obj ects
in continuous material field.
Having developed this notion of the substrate, we can address three related ques­
tions about the phenomenology of performance. (To be very clear, my understanding
of phenomenology is somewhat heretical because I relinquish the Cartesian project
threading Husserl and Heidegger.)
262 C h a pter 8

(1) How can people coordinate transformative and compelling experiences without
relying on conventional linguistic categories such as verbal narrative? The technical
analog to this is: How can people create sense together in a responsive environment
without appealing to grammatical structures?
(2) How could people improvise meaningful gestures collectively or singly in an envi­
ronment that is as alive as they are, an environment that itself evolves over time as
a function of its inhabitants' life?
(3) How could objects emerge continuously under the continuous action of inhabit­
ants in a responsive space? Recall that this question itself arises from a critique of
technology that we encountered earlier, a skepticism of psychologism, behavioralism,
and representationalism, of which genetic determinism is a corollary.
As we saw in the TGarden's improvised nonverbal events, people can shape and
create sense in their material, ambient substrate without recourse to any linguistic
representation. But here is a problem. Suppose we have a more conventionally con­
structed computational media environment in which the software is coded with
conventional procedural programming languages, implementing decision and Boolean
logics. But suppose the intermediating layer of media is built with nongrammatical
texture. Is the person improvising and operating with the patterns of procedural logic,
or with the nongrammatical texture?
Moreover, this improvisatory signification can be responsive and collective. As for
the second question, I argue that the continuity and density of the substrate, and
costructuration that permits infinitesimal variations from a point, lend themselves to
easy improvisation of significant gesture. People can improvise gestures as they already
always have in continuous media like water or snow. And third, objects can be rein­
terpreted more contingently as variations in local denSities, concentrations, or even
as invariants under some thick set of continuous transformations.
Let me present some examples of call and response, and of concurrency. The typical
sensor-based approach to interactive media models a processing chain passing from
human movement through sensor hardware (modality), sensor data, denoising, feature
extraction, mapping logics, synthesis, spatialization, and digital-to-analog sonifica­
tion. The engineering approach is to design a sequence of hardware devices and
accompanying software that process the chain of data from corporeal action to media.
Out of respect for the musicians' decades of experience performing in ensemble,
initially the electroacoustic instrumentalists merely augmented the sound as conven­
tional instrumentalists improvised material. But over three years, the conventional­
instrument musicians learned to play through and with the processed and synthesized
sound, and the electroacoustic instrumentalist-programmers learned not just to process
what the microphones picked up, but to actively play in concert with the other instru­
mentalists. They progressed from electroacoustic programmers and sound designers
Refrai n 263

to performers with coequal responsibility in a j oint performance. Specifically, sound

artists and musicians such as Tim Sutton, Navid Navab, and Julian Stein fashioned
instruments for live performance out of extensive real-time computational sound
analysis, modeling, and resynthesis toolkits. What twenty years ago would have taken
minutes to analyze "off-line can now be done on the fly. For example, !ReAM's
II

OMAX uses signal analysis methods to construct, on the fly, new voices out of
"similar" segments from the given audio input. Most importantly, a host of similarity
measures can be prepared by the programmer in advance and used to construct mul­
tiple voices that the performer can modulate in live performance together with the
other musicians. These resynthesized voices can be quite distinct in timbre or other
sonic features, yet have a discernible relation to the other musicians. In fact the elec­
troacoustic instrumentalist plays in concert concurrently with the rest of the ensemble,
and can even take action to musically tilt the performance together with his or her
partners. From the perspective of the performer, concurrency is as familiar a mode of
collective articulation as call and response.
The question of agency appears as an analytic distinction, but disappears under a
more symmetrical, unbifurcated concept of costructuration. One of the most success­
ful fusions of experiment and research creation that the TML has realized, in my
opinion, is the Ouija movement experiments. Ouija was a set of phenomenological
experiments legible to philosophers but simultaneously also structured improvisation
exercises legible and familiar to trained dancers or actors. Some of the conceptual
questions were: How can dancers make movements with sense sans grammar? How
can a group of people in the same space cocreate an event in concert with a live,
responsive environment? How does a movement-object emerge continuously out of
a continuously distributed volitional or affective field?

Quickened M atter

So what are the implications of regarding experience as inhabiting the world as quick­
ened matter? I suggest that materiality and lifelikeness of obj ects can be taken as effects
of process, rather than predicates on obj ects. Nevertheless, obj ects are not epiphenom­
enal, because they and the processes under which they emerge as invariants are
immanent in the magmatic substrate that constitutes the world.
Furthermore, inhabited in such a mode, this world is as rich as we can imagine
it-rich, not complicated, in the same way that Simon don characterized the difference
between machine memory and human. Richness entails value, and for me what's
valuable is not denominated solely by use value or exchange value but is suffused with
value, with the power to inspire passion.
In such quickened matter, we can create obj ects, and therefore contingent fulcrums
as needed or deSired, because objects emerge in the substrate in the course of the
264 C h a pter 8

F i g u re 8.5
Navid Navab and instrumentalist (Coaticook) .
Refra i n 265

F i g u re 8.6
Oui j a experiments in collective gestures . Still from video.

F i g u re 8.7
Complicated versus rich lifeworlds. Photos by the author.
266 C h a pter 8

world's magmatic, topological dynamics powered by the imaginary. This profoundly

motivates continuous, field-based modes of articulation in a dynamical plenum, rather
than object-oriented or ego-oriented design of objects in vacuum. Let me just indicate
the motivation for continuity (or in a word, topology) in three points: philosophical,
experiential, and technical. First, I conjecture that the topological is the primordial to
the temporal in a sense for which Heidegger seems to have been reaching in his later
work (such as the Metaphysical Foundations of Logic, 1 9 78) . Second, pace quantum
mechanics and its vulgarization as the digital episteme, my experience of the world
at the basic level is continuous. As I walk toward you, you do not flicker in and out
of existence, at least not in my ordinary experience. As I move my hand and flex my
fingers, they do not jump from place to place and moment to moment, but move in
nuance continuously. And third, as I have argued elsewhere, despite what we have
been given as the ground of technologically mediated life, our built world can be rich
rather than complicated. Our task as engineers and architects is to discover how. And
that is another aspect of the research creation of the Topological Media Lab.
It is often the case that poetry can leap over the bridgework patiently established
by philosophy. In 1 938, Dylan Thomas wrote:
The force that through the green fuse drives the flower
Drives my green age; that blasts the roots of trees
Is my destroyer.
And I am dumb to tell the crooked rose
My youth is bent by the same wintry fever.

The force that drives the water through the rocks

Drives my red blood; that dries the mouthing streams
Turns mine to wax.
And I am dumb to mouth unto my veins
How at the mountain spring the same mouth sucks.

The hand that whirls the water in the pool

Stirs the quicksand; that ropes the blowing wind
Hauls my shroud sail .
And I am dumb to tell the hanging man
How of my clay is made the hangman's lime.lO

Ethico-aesthetic P lay

Words, as categories, are handles on the "blooming, buzzing" cloud of experience (to
borrow William james's description); they get their grip via concepts shooting sense­
making tendrils through the loamy earth. In languaging, the earth reorients us. And
conversely, in languaging we grip experience, we manipulate it. And we clarify or
rationalize it with words, but at the expense of washing clear the dirt and leaving
Refrai n 267

these handles swinging clean but free of any friction. This frictionlessness, resulting
from the clarification afforded by systematization and dematerialization, also evacu­
ates sense and value.
Poets by and large do not rely merely on neologism: to say something "new, " they
use, in fact, the very same vocabulary that we all use in our everyday language. Poets
do not have to invent new words a la Lewis Carroll. Fresh poetry from old vocabulary­
continuity allows infinite degrees of freedom and, even in a preCise sense, a larger
order of infinity than the infinity of n to n + 1 . This additional aspect of continuity
is what yields dynamic novelty in Whitehead's or Bergson's versions of ontogenetic
process.
Revisiting Archimedes on the lee side of the twentieth century, we can no longer
expect to be given the fu lcrum of the world or of language. But neither shall we need
to. Nor should we, need we, seek their origins, pace Derrida's particular use of Husserl's
essay on geometric intuition. In place of those modernist and postmodernist projects,
one can begin to see a more immanent mode of resistance and weedy generation in
the muddy interstices of our technologies of performance: the mode of play. Play can
articulate the make-believe, the as-if, making fictive, becoming other than what is the
case, the exfoliating art that drives the green fuse all the way down and up again. But
in recent years, play has been harried by many who would classify it, barely escaping
the nets of those taxidermists who would like to stuff play into the carcass of game.
What our play spaces could offer us are not allegories of other worlds, whether cos­
mological, political, religious, or psycho-fictive, but events affording playful processes
that open life up to more life. Let me close by suggesting a few senses of play that
may merit but also escape more careful consideration. There's the play of water lapping
against the side of the boat, making the lazy slapping sound that evokes sunlight and
fish in the clear water just beyond the reach of your fingers. There's the play, the
empty space, between the teeth of interlocking gears, without which the entire assem­
bly of gears would lock UPi the teeth guarantee discrete synchrony, but it's the gap
that allows movement to be born. And yet that gap is never a vacuum, because the
world's structures are always and everywhere part of the substrate magma of the world.
There's play in the sense of continuous, infinite-dimensional variation from any given
trajectory, that articulates arbitrary degrees of novelty. And there's play as the endless
deferral of definition, a passionate sense making that develops ever more virtuosity in
reenchanting the world.
Epilogue: Conceptual Tactics

This book does not presume to explain what the world really is made of or how the
world really works, or what it really means to be human. It does not "argue" but gives
a sense of how one might regard the world with a certain as-if. Inspired by the tactics
of a Zhuangzi against the logicians and Confucian order, a de Certeau or the situation­
ists vis-a-vis their city, and Grotowski's nonperforming performance laboratory, I've
collected a few conceptual tactics over the years, a set of orienting tropisms, what
Stengers and Whitehead have called lures for feeling and thinking. They are particu­
larly elaborate lures, informed by political, artistic, and technological practices, but
they are not recipes or methodologies. (Parenthetically speaking, methodology comes
after the practice becomes a process that no longer generates knowledge.) These tactics
seem to recur with enough salience to be worth recording. Just as the final chapter of
Deleuze and Guattari's Milles plateaux constituted anything but the answers-at-the­
end-of-the-book for their reader, let me offer these tactics as a measure against the
development of any methodology or school of practice or theory.
We have seen enough leadening in the wake of inspired work (of Marx, Grotowski,
Freud, Heraclitus, Christopher Alexander, Deleuze, Guattari) to make me feel more
than a little concerned about how readers may "apply" this book in their own work.
You have been advised.
( 1 ) Dynamical thinking, topological dynamics, is not so much a metaphysIcs (which
would again be making truth claims) but a style, a way of thinking and making that
is sensitive to ethico-aesthetic poiesis.
(2) Use any formal structure, any form, any theory, any representation, but as a trellis,
not a carapace for thinking.
(3) Turn nouns into verbs.
(4) Instead of making a theory that makes propositions of the form "X is Y, " try
making a theory that turns presumed identities into disequalities, "X > Y" or "X s;:; Y. "
This is not merely a distinction but a local gradient. Recognize that the vectoriality
of the gradient is itself part of your construction, so there you are not making a total­
izing claim.
270 Epilogue

(5) Avoid monocausality, reductionism, looking for the primitive, the Uf-explanation,
the originary cause or event. A category or predicate P so universal that everything is
P is useless. If you define a category S or a predicate P, see if as many things of interest
are not in S as are, or do not have predicate P as do.
(6) Aim for richness and multiplicity, which is not complexity.
(7) In art, use the concepts to transform not just the appearance but the making as
well. Be dissatisfied with allegory.
(8) Instead of making identities, use a modified form of implication, meaning not
"necessarily-Ieads-to" but "enables. " My suggestion that we use implication in the
sense of enabling removes the necessity and the imperative, and replaces those with
permitting, scaffolding, trellising, and sustaining as partial actions (analogous to partial
obj ects) to be defined fully only in the event.
Afterword

Arkady Plotnitsky

Sha Xin Wei introduces the argument of this book as follows:

I argue for an approach to materiality inspired from continuity, field, and philosophy of process,
based on ethico-aesthetic as well as technoscientific grounds. This project investigates what could
be implied by continuous, or more precisely topological, approaches to media and matter in the
concrete setting of installation-events. Another motive is to explore the ethico-aesthetic conse­
quences of topologically creating performative events and computational media, drawing from
the critical studies of science and technology. This project is a philosophical investigation that
is conducted in a poetic mode of installation or event-based art and technology.

This statement reflects a rich network of ideas and fields that this book aims to explore
and sometimes to connect. For some of these ideas (such as materiality and mathe­
matical concepts) or fields (such as installation art and the critical studies of science
technology) are still commonly seen as heterogeneous and distant, or linked only
superficially, even though alternative views of the relationships between them have
been advanced for as long as these relationships have been thought about. Xin Wei's
book expressly follows this alternative thinking, in part by taking its inspirations from
thinkers such as Alfred North Whitehead, Gilbert Simondon, and Gilles Deleuze and
Felix Guattari, who taught us to think of and look for unexpected ways of connecting
seemingly disparate ideas and fields.
I would like to consider three cases of such connections, which are, in my view,
central for the book's project. Xin Wei's statement just cited reflects each of these
cases, as does his title, Poiesis and Enchantment in Topological Matter, which too conjoins
concepts rarely brought together. I will be especially concerned here with poiesis,
topology, and matter, and their relationships or conjunctions in Xin Wei's book.
Although I will have less to say about enchantment, it is, I would argue, essentially
linked to all three conjunctions I discuss here, beginning with the conjunction or even
fusion of materiality and topology, given that " enchantment" is, according to Xin
Wei, "the transmutation (not transubstantiation) of material, " understood, topologi­
cally, as "dynamical plenum, " a crucial concept for him.
272 Afterword by Arkady Plotn itsky

I will begin, then, with the conjunction or fusion of materiality and topology,
which, I would argue, fundamentally grounds Xin Wei's project in conceptual and
ontological terms, as his statement just cited indicates by stressing his attempt to
approach materiality on the basis of the idea of continuity and hence, as I shall
explain, of topology. (As I shall also explain, the concept of "field, " also invoked by
Xin Wei, in effect yet another form of dynamical plenum, belongs to the same con­
figuration of topology and matter, and reflects the same approach.) For this reason,
and because the relevant mathematics, crucial to Xin Wei' argument, requires a more
detailed explanation for a lay reader, the fusion of materiality and topology will be
given a special attention here. The second case I shall consider relates philosophy, art,
mathematics and science, and technology, in the plane of what Xin Wei calls topologi­
cal media, a plane that is also given an ethical dimension. The third case, in effect
correlative to the second, links theory and practice, both in general and in the case
of the book's proj ect itself, as "a philosophical investigation that is conducted in a
poetic mode of installation or event-based art and technology. "
Xin Wei's fusion of topology and matter belongs to the line of thinking that his­
torically emerged, beginning especially with Leibniz, in the wake of Newton's physics,
which firmly separated matter and space or time. Newton's physics, and with it the
notion that matter and space or time are fundamentally separate entities, had domi­
nated science and intellectual history as a whole until the rise of a different thinking,
Leibnizian in spirit, concerning the subject in the late nineteenth century. This new
thinking, initiated by Faraday and Maxwell's electromagnetic theory, became espe­
cially crucial for Einstein's so-called general relativity theory, his non-Newtonian
theory of gravity, based on earlier (by about half a century) geometrical ideas of Bern­
hard Riemann. The rise of mathematical topology as a study of spatiality and con­
tinuity emerged around the same time as well, with Riemann, again, making major
contributions. Working with this history, especially via Whitehead (whose philosophy
was significantly influenced by Riemann's geometrical and topological thinking and,
especially, by Einstein's theory), leads Xin Wei to the fusion of materiality and spatial­
ity or, again, spatiotemporality in his concepts of topological matter and topological
media. Crucially, these concepts are developed within the philosophy of process or
becoming, in terms of dynamical plenum, again following especially Whitehead. It is
also important that this "fusion" brings together time or space and matter, or topology
and matter, without dissolving either in the resulting mixed medium.
Following the original ancient Greek meaning of topos (place), the term "topology, "
when used apart from its mathematical meaning, refers to the architecture of spatial­
ity, as when we speak of the topology of a given place or space-landscape, cityscape,
and so forth. Xin Wei does use the term in this general sense as well. His primary use
of the term, however, derives fr om the mathematical discipline of topology, which
deals mathematically with the structure or, again, architecture of mathematical spaces,
Afterword by Arkady Plotnitsky 273

rather than with measurement, as geometry does, a difference reflected in the etymol­
ogy of both terms (topo-logy vs. geo-metry) . Xin Wei also sees the practice of topology,
in mathematics itself and beyond, in broader terms of a philosophical or even poetic
endeavor, a form of pOiesis, pursued in what he calls topological media. He defines
"topological media" as "a set of working concepts, the simplest set of material and
embodied articulations or expressions that allows us to engage in speculative engineer­
ing, or philosophy as art, and to slip the leg irons and manacles of grammar, syntax,
finite symbol systems, information and informatics, database schema, rules and pro­
cedures." One could speak of topological media even in mathematics, specifically in
topology, all the more so given the use of digital media in contemporary mathematical
research. However, as understood by Xin Wei, topological media is a much broader
conception, as is his concept of fusion of topology and matter, in which the concept
of topological media is grounded.
The question of the relationships between materiality and spatiality or temporality
(customarily seen in science and philosophy in spatial terms, as spatiotemporality, or
at least in relation to temporality) has a long history, extending even to the pre­
Socratics. Restricting this history to modernity for the moment, matter and space or
time were firmly separated by Newton's physics, as noted above, based on the concepts
of absolute space and absolute time, and, following Newton, by physics and, by and
large, philosophy, or even by culture at large. This separation is also part of what Bruno
Latour, in his We Have Never Been Modem, calls the constitution of modernity, which
is, more broadly, based on a related separation of nature and culture, especially poli­
tics. 1 Latour's work is an important reference for Xin Wei, as part of "the critical studies
of science and technology" he refers to in setting up his proj ect. In physics, the view
of space and matter as irreducibly separate entities had reigned until in the end of the
nineteenth century, when Faraday and Maxwell's work revealed the difficulties of
maintaining this separation in physics. These difficulties led to the concepts of ether
and field, both of which may, in part by virtue of their continuous nature, be seen as
a kind of topological matter, or dynamical plenum. While the concept of ether had

eventually proved to be unworkable, that of field has remained crucial to physics ever
since, including relativity and quantum theory, and it remains dominilnt now,
although quantum theory also fundamentally relates this concept to discontinuous,
quantum objects and phenomena. The separation of matter and space was not entirely
unambiguous in Newton's argumentation either (at least Newton had difficulties in
rigorously maintaining it), but it became the dominant way of thinking of the physical
world, viewed as governed by Newton's mechanics. Thus, planets were assumed to be
moving around the Sun, but relative to the empty (absolute) space or time, rather
than only relative to the Sun, as in Einstein 's theory of gravitation, the so-called
general relativity, which was introduced around 1 9 1 5 and rethought the relationships
between space (or time) and matter in terms of the gravitational field.
274 Afterword by Arkady Plotn itsky

We still tend more often than not to separate matter and space, although we should
know better, at least as concerns the limits within which this separation is possible,
which often become important for a rigorous scientific and philosophical argument,
and sometimes even in daily life. The difficulties of this separation have not of course
gone unquestioned, beginning with Berkeley and especially Leibniz, who was also one
of the founders of the mathematical discipline of topology, or at least was among
those who anticipated it. Descartes, too, was unwilling or unable to separate matter
and space in this way. Newton juxtaposed his physics to that of Descartes on these
grounds as well (there were others) . The split of matter and space, or time, was essential
for Newton's project and argument. It was immensely successful in physics before it
was questioned beginning with Maxwell and Faraday's electromagnetic theory, and
ultimately abandoned with Einstein's general relativity theory, based in geometrical
ideas of Riemann, arguably the most original and profound mathematician of the
nineteenth century. Riemann was also one of the founders of modern geometry and
topology; some of Xin Wei's key concepts derive from Riemann's geometrical and
topological thinking. Riemann also anticipated some of Einstein's physical ideas, in
particular in his thinking of the relationships between space and matter in terms of
fields. Einstein's theory was a crucial event in the history of a very different kind of
thinking, in physics and philosophy and beyond, thinking that sees matter and space
or time as inextricably linked.
Xin Wei's project belongs to the history of this alternative thinking, and both
Einstein's and especially Riemann's ideas are crucial for the book. It is working with
this history that primarily leads Xin Wei to the concepts of topological matter, a more
general fusion of materiality and spatiality or spatiotemporality, and then topological
media, again a techno-poetic concept, grounded in the concept of topological matter.
The concepts of topological matter and topological media are developed within the
philosophy of process or becoming, following Whitehead most especially. In a way,
these concepts j ointly form a version or recasting of Whitehead's famous title conjunc­
,,
tion, "process and reality. 2 In any event, the topological matter and topological media
are dynamically spatiotemporal, just as is Einstein's general relativity theory, one of
Whitehead's inspirations in developing his philosophy. As Xin Wei argues, "White­
head's core strategy [in Process and Reality] was to recast relativity theory as [a more
general] ontological theory. " Xin Wei's book aims to extend this recasting to a yet
broader field of topological matter and topological media, an extension that, in par­
ticular, accommodates digital media.
The second main conjunction that defines this book's project relates philosophy,
art, mathematics or science, and technology. This conjunction takes place in what
Deleuze and Guattari call the plane of immanence, in this case configured via or even
as topological media, and in the process given an ethical dimension as well. It is true
that this book is generally more concerned with mathematics than with the sciences.
However, because it is equally concerned with a conjunction of topology and matter,
Afterword by Arkady Plotnitsky 275

it unavoidably brings into play physics and other sciences, such as neuroscience, as
well as technology, in particular digital technology. The book's engagement with the
latter is, I should note, greatly helped by Xin Wei's expertise and intimate knowledge
of this media, including its use as artistic media, rethought in topological and specifi­
cally continuous terms.
The third main conjunction enacted by the book is that of theory and practice, a
conjunction that in effect arises from and, reciprocally, gives rise to the conjunction
of different fields of practice. This is especially the case by virtue of the role of Xin
Wei's understanding of the nature and functioning of poiesis, from mathematical
thinking and practice to "topologically creating performative events and computa­
tional media, " invoked in the passage cited above. But then, Xin Wei argues, math­
ematical poiesis is material and performative, too. All poiesis, all creative making (the
original ancient Greek meaning of poiesis) is material and performative, taking its place
in one or another form of topological media.
Now, in order to better situate Xin Wei's project, I would like to offer a sketch of
the history of the problematic of spatiality (including as spatiotemporality) and matter
and their relationships, beginning with what one might call Euclidean thinking.
Although modern, Newton's separation of matter and spatiality outlined above is a
product of this type of thinking as well, in part by virtue of taking its inspiration from
and conceptual grounding in Euclidean geometry. The key aspects of Euclidean think­
ing are found already in Plato and Aristotle and can even be traced to the pre-Socratics.
Thus, they predate Euclid's Elements, which codified them in what is now known as
Euclidean geometry. Although the Elements do not expressly speak of space qua space,
the corresponding concept of space, which belonged to a kind of philosophical topol­
ogy (rather than geometry), was current in contemporary philosophy and thus must
have influenced the author (or authors) of the Elements. The corresponding concept
was only given a proper geometrical form with Descartes, who also gave this concept
its coordinate form, which proved to be so crucial for physics.
Euclidean thinking conceives of space either as a physical (exterior) entity or as a
corresponding phenomenal entity (an interior image of the exterior), a single (flat)
container of its subspaces, surfaces, lines, points, and so forth, or of phenomenal or
material objects, which such mathematical entities idealize. Inspired by Euclid's geom­
etry and Newton's physics, Kant, in the Critique of Pure Reason, saw this type of phe­
nomenal representation ( Vorstellung) of space, along with time, as given a priori to our
phenomenal representational intuition (Anschauung), rather than as derived from
experience or, to begin with, as corresponding to the actual constitution of nature.3
One might also speak of a certain representation or visualized "image" of space, which
is arguably closer to the meaning of Anschauung in Kant. Kant's view concerning the
a priori Euclidean-like character of spatiality, or temporality (his analysis of time
follows the same line of thought), has been challenged on several grounds from the
moment it was introduced and throughout subsequent intellectual history. The subject
276 Afterword by Arkady Plotnitsky

has been investigated at length from the nineteenth century on, beginning (to name
the most prominent nineteenth-century figures, often in dialog with Kant), with
Fechner, Wundt, and Helmholtz. Helmholtz's investigation of the subject occurred
under the impact of Riemann's ideas, following the 1 868 publication of Riemann's
Habilitation lecture on foundations of geometry (originally given in 1 854),4 and Helm­
holtz also made important mathematical, physical, and philosophical contributions
to geometry which influenced Einstein, among others. Although twentieth- and
twenty-first-century developments in psychology, neuroscience, cognitive sciences,
and other fields have advanced our understanding of spatial perception and thinking,
and have established new grounds for pursuing the questions just mentioned, many
of these questions remain difficult and are far from being resolved. For example, it
remains unclear whether our neurologically grounded phenomenal sense of spatiality
is ultimately Euclidean-like or not. Some current neurological research suggests that
it may not be, but this research is far from conclusive.
These challenges notwithstanding, the Euclidean view of space, found in mathe­
matics, science, philosophy, and art, and in our daily life, may be seen at least in part
as derived from a certain phenomenal intuition of spatiality, even if this intuition is
acquired through experience, possibly early in life, rather than given a priori. In other
words, we do appear to have Euclidean-like spatial intuitions, at least within certain
limits, and some of the artistic practices described by Xin Wei and grounding his theo­
retical argument may be seen as exploring these limits. These intuitions must in part
have prompted Kant's view of space as given a priori to our phenomenal intuition,
but, I would argue, only in part, since Euclidean and Newtonian concepts of space
played their roles in shaping his view, rather than only confirming it. These intuitions
must have also shaped, again at least in part, Euclidean thinking from the ancient
Greeks on, and one might see this thinking as giving more refined and specifically
mathematized form to our daily ideas concerning space, time, motion, and so forth.
As understood by Euclidean thinking, space is an infinite three-dimensional con­
tinuum of points. The continuity of space means, roughly, that space contains no
isolated points. Certain configurations in space may contain such points or, in the
case of discrete configurations, consist of such points. A rigorous mathematical concept
of continuity and its significance for our understanding of spatiality had to wait until
the nineteenth and even the twentieth century to be more or less settled. We may not
be able to do better than "more or less" in either case. The continuum of space was
assumed to be flat, corresponding mathematically to Euclidean or Cartesian geometry,
until the introduction of non-Euclidean geometry and especially Riemannian geom­
etry in the nineteenth century.
Time, from Plato and Aristotle on, has usually, even if not always, been conceived
on the same model of continuum, most commonly, especially in physics, of one­
dimensional continuum: as a line extending from the past to the future via the
Afterword by Arkady Plotn itsky 277

present. There have been alternative models of temporality, especially historical or

cosmological temporality, such as cyclical models, models that suspended the flow of
time altogether at the ultimate level, and so forth. These models, however, are less
germane here, except from certain multidimensional or scrambled models (on which
I shall comment presently) . Besides, most of these models still relate to and often
depend on the linear model of time under discussion. The impact of this model is felt
in Bergson's and Whitehead's thinking concerning temporality, even though these
thinkers also complicate and even challenge this model, in part by giving it material
dimensions, by defining time by topological matter, which one may index here by
Bergson's famous title Matter and Memory.s These dimensions are far from absent in
Husserl either, although his phenomenology, specifically that of time consciousness,
remains more bound by the traditional model of temporality, as Derrida showed.6
Derrida also showed that this model ultimately governs Heidegger's view of temporal­
ity, expressly advanced in juxtaposition to this model, in particular in Aristotle and
HegeL ? In any event, once seen as an effect of materiality, rather than as preceding or
being in an unconditional juxtaposition to it, temporality (phenomenal, psychologi­
cal, or historical) also acquires pluridimensionality, in the manner of musical chords.
Xin Wei's argument takes advantage of this pluridimensionality and extends both
Bergson's and Whitehead's (or others') topologically continuous model of temporality
accordingly. I can, however, only mention this aspect of Xin Wei's argument here, or
its relationships with the work of such thinkers as Heidegger, Freud, Lacan, and
Derrida, who questioned continuous temporality, or spatiality, and who may, in this
respect, be contrasted to Bergson and Whitehead, with whom Xin Wei's argument has
manifest and deliberate affinities in this regard.
Until the late nineteenth century, it was three-dimensional space that had been
commonly understood by " space, " either in phenomenal or physica� terms. While
non-Euclidean geometry made us realize that physical space may not be Euclidean, it
did not change this underlying view, although four-dimensional mathematical obj ects
were introduced around the same time as well. Riemann, the founder of the modern
mathematical conception of spatiality as manifoldness (Mannigfaltigkeit) or manifold
\
(the currently standard English mathematical term), also restricted the apP'J.- cation of
this concept to space by defining space as a three-dimensional manifold, possibly
\

curved, despite the fact that as a mathematical concept space can have any number
•
of dimensions, can even be infinite-dimensional. On the other hand, Riemann only
saw the view of space as a three-dimensional manifold as a hypothesis, one of the
"hypotheses that lie in the foundations of geometry, " to cite the title on his Habilita­
tion lecture, where his concept of manifold and his new thinking concerning spatiality
and geometry were introduced.8 Physics, he argued (prophetically anticipating Ein­
stein), was to decide what kind of manifold space ultimately is, flat or curved, discrete
or continuous, three or more dimensional, and so forth. In speaking of space, like
278 Afterword by Arkady Plotnitsky

most of his predecessors and contemporaries, he appears to have had in mind primar­
ily or even exclusively physical space, which may be phenomenally represented
mathematically, although he also appears to have thought (in this regard following
Kant) that however accurate this representation, it may not be established with cer­
tainty. The term "space" acquired a broader mathematical scope shortly thereafter, in
part under the impact of Riemann's ideas concerning manifolds, which, as I said, can
be mathematically defined as having any given number of dimensions, including an
infinite one. Riemann also allowed that the physical space may ultimately be discrete,
in which case it would have the topological dimension zero, although, given the very
small scale of this discreteness, we would not perceptually experience it. To accom­
modate this possibility, he introduced the concept of discrete manifoldness, different
from that of continuous manifoldness, with which I am primarily concerned here
because it grounds Xin Wei's view of spatiality or topology, or geometry, and the
relationships between them and materiality.
As indicated above, one of the notable features of Xin Wei's book is its exploration
of the experience and concept of continuity, via the topology, rather than only the
geometry, of manifolds. Discontinuity has, arguably, been more prominent in recent
(and some no longer recent) theoretical discussions of some of the key problematics
considered by the book. In this regard the book expressly follows and takes its inspira­
tion from thinkers of continuity and plenitude (again, as dynamical plenum), such as
Bergson, Whitehead, Deleuze, as against thinkers of discontinuity and rapture and of
unavoidable " subtraction" from plenitude, such as Nietzsche, Freud, Lacan, Levinas,
Derrida, or Badiou. Indeed, the history of twentieth-century philosophy is often one
of the oscillations between these two lines of thinking that extend from Kant (on the
side of discontinuity) and Hegel (on the side of continuity)-decoupling for the
moment the earlier history of these oscillations, from the pre-Socratics on. It is not, I
hasten to add, a matter of giving an unconditional priority to one or other of these
lines of thought, and some of the thinkers just mentioned (in a way all of them),
beginning with Kant and Hegel, oscillate between these lines of thought as well. Rather
it is a question of the theoretical and strategic preference in any particular case, and
Xin Wei explains his preference in this regard from the outset, in part by noting that
thinking in terms of discontinuity has received more attention in recent decades. This
has actually changed more recently, in part because of the prominence of Deleuze and
the resurgence, in part via Deleuze, of interest in Bergson and Whitehead, a develop­
ment that Xin Wei's book reflects as well.
The history of modern topology and geometry (and thus this book) owe much to
Riemann's concept of manifoldness, in particular, again, continuous manifoldness.
This is hardly surprising given how important, even indispensable, this concept has
been to modern mathematics and physics, Einstein's relativity in particular, and phi­
losophy, mostly indirectly, although this impact is pronounced in Bergson, White-
Afterword by Arkady Plotnitsky 279

head, and Deleuze and Guattari. For simplicity, and following the current mathematical
idiom, I shall use the term "manifold" and " space" more or less interchangeably, as
both referring to Riemann's concept of continuous manifolds. Also, I shall for the
moment only consider (as did Riemann and as, for the most part, does Xin Wei) those
manifolds that are not only continuous but also differentiable or " smooth, " which
technically means that one can use differential calculus in studying them; hence, the
corresponding mathematical field, extensively discussed by Xin Wei, is differential
geometry. Intuitively, this means that a movement from point to point can be smooth,
proceeding along a smooth line-without sharp angles, breaks, or jumps-and that
the shortest possible path is always smooth. A smooth two-dimensional surface, say
a spherical surface, provides a good model of a Riemannian manifold. The shortest
(the technical term is "geodesic") path between any two points is the shorter arc of
the great circle connecting these points (or half of the great circle between two polar
points) . Overseas flights generally follow this arc td save time and fuel. On the surface
of an ideal mathematical sphere this great-circle path is always a smooth path, while
on the actual surface of the Earth it may not be. By contrast, a plane's geodesic trajec­
tory is essentially smooth.
A (smooth) Riemannian manifold or space is an interconnected conglomerate of
locally (technically speaking, infinitesimally) flat Euclidean spaces forming neighbor­
hoods of its pOints, without the manifold as a whole being Euclidean, except in the
special case of Euclidean space itself of the corresponding dimension. In the case of
the sphere, for example, one can imagine small circles on the surface around each
point and project each such circle onto the tangent plane to this point, to get a regular
flat circle on this plane. If the first circle is very small, the difference between the two
circles becomes very small and can be neglected, allowing one to treat each circle
drawn around a point on the surface as flat, Euclidean. Usually, one considers "open"
circles by removing the boundaries, or, once this picture is properly generalized, open
neighborhoods. Given that we define them around each pOint, such neighborhoods
may multiply overlap. This, however, does not pose a problem, because we can estab­
lish proper rules for governing this overlapping and for (smoothly) moving from one
neighborhood to another. Such a "covering" or "mapping" (which, like '('neighbor­
hood, " are technical mathematical terms) may itself be multiple. Deleuze anil Guattari
offer a fine philosophical description of Riemannian manifolds ("Riemann space" in
their terminology), via Charles Lautman, which further suggests that Riemann's
concept is easily generalizable to a more general philosophical concept, with a rich
potential field of application. They write:
Riemann spaces are devoid of any kind of homogeneity. Each is characterized by the form of the
expression that defines the square of the distance between two infinitely proximate points . . . .
"It follows that two neighboring observers in a Riemann space can locate the points in their
immediate neighborhood but cannot locate their spaces in relation to each other without a new
280 Afterword by Arkady Plotnitsky

convention. Each vicinity is therefore like a shred of Euclidean space, but the linkage between
one vicinity and the next is not defined and can be effected in an infinite number of ways.
Riemann space at its most general thus presents itself as an amorphous collection of pieces that
are juxtaposed but not attached to each other. " It is possible to define this multiplicity without
any reference to a metrical system, in terms of the conditions of frequency, or rather accumula­
tion, of a set of neighborhoods; these conditions are entirely different from those determining
metric spaces and their breaks (even though a relation between the two kinds of space necessarily
results). In short, if we follow Lautman's fine description, Riemannian space is pure patchwork.
It has connections, or tactile relations. It has rhythmic values not found elsewhere, even though
they can be translated into a metric space. Heterogeneous, in continuous variation, it is a smooth
space, insofar as smooth space is amorphous and not homogeneous. 9

The architecture of Riemannian manifolds is, then, defined by the multiplicity and
connectivity of local spaces and therefore local mappings, and ways of connecting
these spaces and maps, rather than being defined as a set of pOints. In other words, a
Riemannian space is defined as a conglomerate of local spaces and the (contiguous)
connections between and among them. Lautman's description, cited by Deleuze and
Guattari, is inflected by Einstein's relativity, via the idea of "observers in a Riemann
space, " whose curvature is defined by matter. I shall return to Einstein's theory pres­
ently. First, I shall summarize those features of the concept of manifold that are espe­
cially important for Xin Wei's argument.
First, it is more flexible and ultimately more effective to think of (the continuum
of) space as a manifold "glued" or "quilted" of subspaces, such as those covering each
point, rather than as something continuously constituted by points. "Points come
later, " topologists sometimes say, " spaces come first, " making topology or geometry
the " sociology" of space. To begin with (although this is less germane to Xin Wei's
argument) , the development of the concept of continuum as (continuously) consti­
tuted by points poses formidable and even insurmountable mathematical difficulties,
manifest throughout the history of this concept, from Georg Cantor's introduction of
set theory on. Indeed, it does not appear possible to rigorously formulate the concept
of the continuum of pOints, and Leibniz was perhaps first to realize the grave difficulty
of doing so. The concept of manifold (of any dimension, beginning with dimension
one in the case of a line, straight or curved) allows one to circumvent some of these
difficulties. It also allows one, and this aspect of the concept is important for Xin Wei's
book, to explore more effectively the possibilities of mathematical pOiesis, given that
one can compose ("glue" or "quilt") a manifold differently out of neighborhoods. Our
phenomenal intuition of continuity does not depend on a rigorous mathematical
concept of the continuum as constituted by points, a concept that, once considered
fully rigorously in technical terms, has little in common with this intuition. On the
other hand, this intuition can help us form philosophical concepts and philosophical
underpinnings of mathematical concepts, such as those considered by Xin Wei, many
Afterword by Arkady Plotn itsky 281

of which are linked to or based on Riemann's concept of a continuous manifold as

composed of subspaces or neighborhoods. Xin Wei also takes advantage of the fact
that in many cases that he considers, this architecture is defined by various forms of
materiality, thus again making this architecture that of topological matter, and in some
instances, such as those dealing with "event-based art and technology, " of topological
media. The idea of defining a given space in terms of a sociology of spaces (it can be
extended to spaces other than subspaces of this space) can be effectively used in
examining a great variety of spaces-physical, phenomenal, cultural, political, and so
forth-and of the interactions among them. Consider geography as an example,
simple but far from unimportant, by virtue of the fact that this example extends to
other cartographies (such as those of Michel Foucault) : the meaning of a physically
spatial region or even point depends on what kind of map (physical, economic, politi­
cal, or other) or atlas of maps is associated with this region or point.
Second, once Riemann's perspective is in place, Euclidean (flat) spaces, such as the
two-dimensional plane or the standard three-dimensional space, are merely particular
and indeed very special cases of manifolds. Most mathematical manifolds, of any
dimension, are not Euclidean; and modern geometry and topology consider manifolds
or spaces, called topological spaces, that need not be Euclidean even locally. Moreover,
one could define geometrical properties (such as curvature) of two-dimensional sur­
faces internally, rather than in relation to their ambient three-dimensional Euclidean
space. Thus, a two-dimensional sphere is a Riemannian manifold of positive curvature,
whether one considers it embedded in the three-dimensional space or an independent
space. At the same time, this independent, intrinsic nature of curvature allows us to
ascertain whether a physical space, such as ours, is actually curved or flat, either locally
or globally. This shift to the independent, internal determination of a given space,
specifically as possibly non-Euclidean, was crucial in the history of non-Euclidean
thought in mathematics, physics, and philosophy. For it follows that, rather than
having a single all-encompassing space or geometry, as a container of all other possible
" spaces" (which are thus merely configurations in this single space) or geometries, we
have, even mathematically, an irreducible multiplicity of spaces, each entering on
equal footing. The pre-Riemannian conception of non-Euclidean geonietry only
changed a possible geometry of space by allowing it to be curved, but not the view of
space (mathematical or physical) as a single all-encompassing space, absolute space,
assumed as a physical space by Newton or as a phenomenal space by Kant. 1O It may
be noted that, while there is only one Euclidean space and geometry in any dimen­
sion, there are an infinite number of non-Euclidean spaces and geometries in each
dimension. In any event, one now confronts the irreducible multiplicity of spaces and
geometries even in three dimensions, let alone when we move to other dimensions.
A multiplicity of manifolds, even a manifold of manifolds (this concept has a strict
technical meaning in mathematics), takes geometry over, and lays grounds for the
282 Afterword by Arkady Plotn itsky

field of topology and its role in geometry. This architecture can, again, be generalized
well beyond mathematics, a generalization that has a tremendous and far-reaching
philosophical potential, with implications and applications in many fields. Xin Wei's
book reflects and explores this potential as well, including in the areas rarely consid­
ered from this perspective previously.
Third, in part as a consequence, since the geometrical properties of space are
defined internally, the question becomes what defines them, specifically in the case
of physical space, since mathematically one can legitimately consider any geometry.
Riemann's answer was that properties and behavior of matter, such as gravity, define
the geometry of space, possibly as curved, even as having variable curvature. Riemann
was the first to grasp this possibility on both counts-the role of physical matter and
the role of variable curvature, as against previous conceptions of non-Euclidean geom­
etry only dealing with constant curvature-although it was Einstein who rigorously
connected space and matter (gravity) in general relativity theory.
As indicated above, this view extends from Leibniz's analysis of the idea of space,
advanced against Newton's concept of an absolute space. Relativity follows the Leib­
nizian view of space or time as defined by rather than as containing matter, and,
correlatively, implies the Riemannian manifoldness of space and time, as against
Newton's physics grounded in the Euclidean understanding of space and time (as
absolute space and time). There is no such space or time in Einstein's theory, in part
because there is no empty space preexisting matter. Within certain idealized limits the
concept of empty space, vacuum, has its place in physics, specifically in special relativ­
ity, which was introduced by Einstein earlier (in 1 905) and which deals with the
propagation of light in the "empty" space in the absence of gravity. This theory still
dispenses with absolute space or time, even in the absence of gravity, although not in
the absence of matter, since light and hence electromagnetic properties of matter are
crucial to the theory. Space or time thus exists only in relation to ponderous bodies
or other forms of matter (such as an electromagnetic field), the gravitational forces of
which determine the curvature, generally variable, of this space. We witness this effi­
cacy of matter in measuring rods and clocks, which create effects of space and time,
rather than being merely places in space and time which they then measure.
Indeed, Einstein's theory may be seen as doing even more conceptually, as again
anticipated by Riemann as well: it establishes a physical fusion of matter and space,
and also time, or space-time, in effect establishing space-time as, in the language of
Xin Wei's book, a kind of topological matter. This type of vision of space, time, and
matter was arguably announced by Hermann Weyl's classic on Einstein's relativity,
Space-Time-Matter, the hyphens of the title suggesting a fusion of all three, a fusion
that brings them together again without dissolving them in the resulting medium. l l
From this vantage, one can, reciprocally, think either o f geometry and topology from
matter or, closer to Xin Wei's way of proceeding in the fields he explores, of matter
from geometry and topology.
Afterword by Arkady Plotnitsky 283

The conceptual architecture of manifolds could be transferred into our understand­

ing of the relationships between materiality and spatiality or temporality, and of each
concept itself. Deleuze and Guattari even credit Riemann's introduction of the concept
of manifold with "mark[ing] the end of dialectic and the beginning of topology and
typology of manifolds. 12 They clearly have in mind the extension of Riemann's
II

concept well beyond mathematics and physics, and this extension clearly reaches to
the relationships and sometimes fusion of spatiality, temporality, and materiality well
beyond physics as well. In the last hundred years, we have seen this conjunction or
fusion at work in philosophical discourses and artistic practices, upon which Xin Wei's
book builds in its own exploration of this fusion.
The appearance of this type of conceptual architecture beyond mathematics and
physics has sometimes been due to a more or less direct impact of Riemann's and
Einstein's ideas and related developments in mathematics and physics, as in Deleuze
and Guattari or earlier Bergson and Whitehead, or to their more indirect and mediated
impact. More significantly, analogous and even isomorphic conceptual architectures
may also emerge, more independently, under the pressure of particular problems that
one confronts in a given field. Indeed, the latter is a decisive, even the most decisive
factor, even for figures such as Bergson, Whitehead, and Deleuze and Guattari who
were directly influenced by Riemann and Einstein. In the case of Deleuze and Guattari,
too, a certain set of problems, such as that of smooth and striated spaces (physical,
cultural, aesthetic, or other), considered especially in A Thousand Plateaus, required
the type of conceptual architecture in question, for which Riemann's concept of
manifold, again supplemented by Einstein's ideas, provided a helpful mathematical
model. It was, however, only one of the models they used to approach this set of
problems, since they also offered physical, maritime, mUSical, and other models­
potentially, to paraphrase their book's title, a thousand models, or even a Riemannian­
like manifold of models, each a kind of Riemannian-like manifold.
By the same token, as Deleuze and Guattari would have argued as well, it is not
merely or even primarily a matter of using the corresponding mathematical or physical
metaphor, but instead of establishing a rigorous concept, which they always defined
as a response to a problem, or set of such concepts. (Their use of the teOV "model"
refers to such concepts or sets of concepts.) Such concepts need not be exact in the
way they are in mathematics or, usually via mathematics, in physics or elsewhere in
science: they are, in Deleuze and Guattari's language, anexact (more qualitative) yet
rigorous concepts. The role of such concepts is crucial for Xin Wei's book, not least
because such concepts may also emerge from mathematics and indeed, as Xin Wei
rightly argues, are operative in mathematics itself or, it follows, in physics and else­
where in science.
Topology plays a particular role in this type of thinking, both within mathematics
itself and in extensions of the exact conceptual architecture of spatiality, or topological
matter, to other fields, as Xin Wei's book aims to effect. Before I address this extension,
284 Afterword by Arkady Plotn itsky

I should comment further on the nature of topology as a mathematical discipline,

given the role of it in Xin Wei's book.
While both geometry and topology are concerned with space, they are distin­
guished by their different ways of studying it. Geometry deals primarily with measure­
ment and thus also with curvature (since we often need to measure distances along
curved paths) . Topology, by contrast, disregards measurement as well as scale, and
deals only with the architecture of space qua space, for example with the essential
shapes of figures. Here distances or curvatures are generally irrelevant. It is only con­
tinuity (as connectivity) or conversely ruptures of continuity that matter. Insofar as
one deforms a given figure continuously (Le., insofar as one does not separate pOints
previously connected and, conversely, does not connect points previously separated),
the resulting figure is considered the same. The proper mathematical term is "topologi­
cal equivalence . " Thus, all spherical surfaces, of whatever size and however deformed,
are topologically equivalent, although some of the resulting objects are no longer
spherical in geometric terms. Such spherical figures are, however, topologically distinct
from the surfaces of tori, since these two kinds of surfaces cannot be transformed into
one other without disjoining their connected points or j oining the disconnected ones:
the holes in tori make this impossible. This is sometimes expressed by saying that,
rather than measuring distances like geometry, topology "measures" (counts) the
number of holes in a spatial object. Topology defines a given space via its so-called
open subspaces, such as open neighborhoods around each point. A good model of an
open subspace in the two-dimensional case is, again, a circle considered without its
circumference. On a two-dimensional surface one can think of a neighborhood of a
given point as a small circle (without boundaries) around this point, as described above
in the case of Riemannian spaces such as a sphere, except that now we disregard
measurement, while in Riemann's geometry measurement and its locally Euclidean
nature are crucial. Topologically, any open (without boundaries) part of the sphere's
surface will do, and the sphere itself can be deformed in any conceivable way, as long
as we keep its connected points connected. As will be seen presently, the underlying
topological architecture of manifolds is crucial in turn.
Topology is mathematical, exact, not by virtue of mathematizing spatiality by
measurement, as geometry does, but by virtue of relating the architecture of spatial
objects to algebraic or numerical properties of algebraic or arithmetical obj ects. The
number of holes in a given object, such as the surface of a sphere with no holes in it,
that of a torus which has one hole in it, or the surfaces of pretzel-like figures each
with several holes in them, are the simplest example of this kind of relation. The idea
was introduced by Riemann as well, and was used by him to classify two-dimensional
surfaces of this type as manifolds, which was part of his move in mathematics and
beyond to the topology and typology of manifolds (celebrated as a major philosophi­
cal move by Deleuze and Guattari) . Earlier anticipations of this way of thinking are
Afterword by Arkady Plotn itsky 285

found in Leibniz and, especially, in Leonhard Euler. However, Riemann's topological

thinking was decisive in the development of the mathematical discipline of topology,
especially for the work of Henri Poincare (who was probably most responsible for
establishing topology as an independent mathematical discipline) and others in the
late nineteenth and early twentieth centuries.
At the same time, however, topology arguably engages more readily than geometry
with anexact concepts (since measurement and hence geometry are generally more
quantitative, and thus more likely to derive from exact concepts), although, as Xin
Wei's argument makes clear, this engagement is unavoidable in geometry and in all
mathematics. Hence, topology has a greater potential for transferring such anexact
concepts elsewhere. However, geometry and topology are often interlinked at exact
and anexact levels alike. For one thing, ontologically, topology precedes geometry, at
least in terms of phenomenal mathematical ontology, but elsewhere as well, given the
architecture of topological matter. Any geometrical space, Riemannian or other (while
a Euclidean space is only a special case of a Riemannian space, not all geometrical
spaces are Riemannian), is first a topological space. Before a Riemannian space is
composed of local Euclidean spaces, it is composed of local topological spaces, upon
which a Euclidean or Cartesian superstructure is built, but we can think of such topo­
logical spaces or of course topological spaces in general apart from such a superstruc­
ture. The field that specifically studies the topological aspects of differential, such as
Riemannian, manifolds is known as differential topology, which, somewhat surpris­
ingly, receives little notice in Xin Wei's book, compared with differential geometry,
for example. In any event, this precomprehending of geometrical spaces by topological
ones is crucial, both in mathematics or physics and in other fields, such as those
explored by Xin Wei, which is why the subject is important, if sometimes implicit, in
the book.
Thus, for example, the currently standard view is that the space of the observable
universe is globally flat on average (local perturbations created by stars or even galaxies
can be disregarded on that scale) . It is, however, not inconceivable that on yet bigger
scales, which we cannot observe (either now or ever), the universe may be something
else, say a three-dimensional torus. In this case, the flatness of the observed universe
is merely a "local" effect, large as it may be relative to a human or even galactic scale.
We cannot visualize a three-dimensional torus, but think of us as flatlanders living on
a two-dimensional torus, so big that we cannot get anywhere or even observe beyond
our local region, which we can only experience as flat. My main point is that topol­
ogy, or topological matter, may affect the geometry and the physics of the universe.
To give another, less cosmologically charged example, due to a famous theorem of
Poincare, the " hedgehog theorem, " a flow of liquid on a sphere cannot be free of
singularity, whereas it can be on a torus. The name "hedgehog theorem" is due to the
fact that the theorem also tells us that we cannot "comb" the ideal spherical hedgehog
286 Afterword by Arkady Plotnitsky

(with a hair growing in each point of the surface) . We cannot make each hair exactly
tangent to the surface, since at least one hair will stick out, thus revealing at least one
pOint of singularity. The implications for the physics, the topological matter, of flows
are qUite significant as well. In fact, to return to the cosmological scale, some of the
current cosmological theories suggest that in fact the universe is only flat and three­
dimensional on an observed scale, while on a very small scale it has in fact a very
complex topology (and hence geometry) of nine dimensions, ten if one counts time.
While the macro-scale topology is primarily defined by gravitational matter, the micro­
scale topology is primarily defined by and defines quantum matter. Thus, topological
matter is multi structured and multistratified even in physics.
These stratifications are ever more multiple and complex when we deal with other
forms of topological matter, as does Xin Wei's book, which may be seen as an explora­
tion of these multiple stratifications, from mathematics itself (and specifically topol­
ogy) to technology to art to ethics, while bringing these fields together in turn. I would
like now to discuss this bringing together, via Deleuze and Guattari's argument,
advanced in What Is Philosophy?, concerning the differences and relationships among
philosophy, art, and science, including mathematics.13 Deleuze and Guattari do not
expressly address technology or ethics; but both (more manifestly ethics), and their
relationships to philosophy, art, and science can be considered in conjunction with
their argument in that book. I would also like to link this problematic to their concept
of the plane of immanence, as an entity in which, I would argue, these relationships
ultimately originate. This last argument is only implicit in What Is Philosophy?, which
expressly associates the plane of immanence with philosophical thinking, but else­
where Deleuze extends this concept to the immanence of thought and even of life
itself, most especially in "Immanence: A Life . " 14 As is suggested by its name-the plane
of immanence, in effect a kind of Riemannian manifold of immanence (and the term
"plane" may reflect our limited capacity to go beyond envisioning low-dimensional
topological objects)-this concept is topological in character. In addition, especially
once given materiality, as I would argue it must be, this concept can be linked to Xin
Wei's concept of topological matter and topological media.
Deleuze and Guattari define thought itself as a confrontation between the mind,
indeed the brain, and chaos, a confrontation, they also argue, especially and even
uniquely characteristic of philosophy, art, and SCience, again including mathematics.
On the surface, such a view of thought is hardly surprising: much of our thinking in
the conventional sense (that of mental states and processes) may be understood as
this type of confrontation. Deleuze and Guattari, however, have something more in
mind, which gives a greater originality and force to their conception. For one thing,
they understand chaos as "defined not so much by its disorder as by the infinite speed
with which every form taking shape in it vanishes. It is a void that is not a nothing­
ness but a virtual, containing all possible particles and drawing out all possible forms,
Afterword by Arkady Plotnitsky 287

which spring up only to disappear immediately, without consistency or reference,

without consequence. Chaos is an infinite speed of birth and disappearance. il l S This
is an unusual conception of chaos, which does not appear to have been previously
used in philosophy (it appears to originate in quantum theory) . It gives a particular
form to the interaction of thought and chaos, as it were, extracting more permanent
or graspable forms of order from more constantly and speedily disappearing forms of
order, which is quite different from most other conceptions of such interactions,
usually grounded in the idea of chaos as disorder. It could be argued (I think Xin Wei
would agree) that any topological media in his sense involves the same type of inter­
action, which takes a form of poiesis, in the broader sense of making ordered entities
(philosophical, mathematical, and scientific concepts and theories, works of art, digital
software, and so forth) . Secondly, and more crucially, Deleuze and Guattari see chaos
not only as an enemy but also as a friend of thought, its greatest friend and its best
ally in a yet greater struggle that thought must wage, the struggle against opinion.
[The] struggle against chaos does not take place without an affinity with the enemy, because
another struggle develops and takes on more importance-the struggle against opinion, which
claims to protect us from chaos . . . . [l1 he struggle with chaos is only the instrument in a more
profound struggle against opinion, for the misfortune of people comes from opinion . . . . But
art, science, and philosophy require more: they cast planes over chaos. These three disciplines
are not like religions that invoke dynasties of gods, or the epiphany of a single god, in order to
paint the firmament on the umbrella, like the figures of an Urdoxa from which opinions stem.
Philosophy, science, and art want us to tear open the firmament and plunge into chaos. . . . And
what would thinking be if it did not confront chaos?16

The struggle against opinion, thus, gives thought an ethical and political dimension.
Another one emerges along the lines of the relationships between what Deleuze and
Guattari call minor or nomadic practices, as juxtaposed by them to major or state
practices, including, arguably especially pertinent to Xin Wei's project, such relation­
ships in mathematics. While major mathematics aims to maintain and enforce math­
ematical opinion, it has other functions as well. In particular, any form of major
mathematics often, indeed nearly always, emerges from some initially minor mathe­
matics, and on occasion it may become minor mathematics (although tlii s is rare) .
Deleuze and Guattari make this argument in the case of geometry in late eighteenth­
and early nineteenth-century France by j uxtaposing a minor or nomad projective
geometry of Gerard Desargues, who was engaged in a kind of mathematical poiesis,
and a major or state geometry of Gaspar Monge and his school enforced in and by
the Ecole Polytechnique, then a leading science and engineering institution in France. l ?
Although Xin Wei does not pursue this line of argument, it may be suggested that,
especially by virtue of dealing with anexact m athematical concepts, the mathematical,
such as topological, poiesis in his sense is more characteristic of minor or nomad math­
ematics, and it enhances the role of minor mathematics within major mathematics.
288 Afterword by Arkady Plotn itsky

In addition, mathematical poiesis also has a potential to counteract mathematical

opinion, and, as in other domains of life, misfortunes of mathematics do often come
from opinion. Xin Wei does invoke Deleuze and Guattari's juxtaposition between
major and minor science in the context of the thematic of the smooth and the stri­
ated, for which Riemann's manifolds (which are, essentially, globally smooth but
locally striated) serve as a primary or even the primary mathematical model. Boulez's
music provides the main mathematical model, and the language of the smooth and
the striated is due to Boulez. The discussion of music, including in the set of contexts
I am conSidering now, forms a particularly engaging and innovative part of Xin Wei's
book. As I said, the type of argument for the major and minor could also be made for
science, and it was made by Deleuze and Guattari for several other fields, as in their
discussion of Kafka's work as minor literature, which they also see as a jointly aesthetic
and ethical-political phenomenon. IS My main point here is that mathematical pOiesis
has a clear bearing on the ethics and politics of mathematics, and hence on how we
do the philosophy of mathematiCS, which, especially as the institutional philosophy
of mathematics, rarely deals with anexact mathematical concepts and mathematical
poiesis, or with minor mathematics.
To return to the argument of What Is Philosophy?, the confrontation between
thought and chaos takes a particular form in each respective domain: a creation of
affects and planes of composition in art; a creation of functions or propositions and
planes of reference in science (including, again, mathematics); and a creation of con­
cepts and planes of immanence in philosophy. Deleuze and Guattari give a special
sense to the term " concept, " or introduce a special concept of "concept, " by defining
it as a multicomponent entity, uniquely arising in each conceptual invention in
response to or posing a particular problem, rather than being a generalization from
particulars. It may be noted that in effect concepts in this sense are equally found in
mathematics and science, or even in art, and not only by way of philosophy's interfer­
ence with this field, although the latter is important as well. The specificity of the
workings of thought in each field makes them different from each other, and part of
the book's project is to explore this specificity and this difference. Deleuze and Guat­
tari go so far as to argue that "the brain is the junction-and not the unity-of the
three planes" through which art, Science, and philosophy, each in its own way, cut
through chaos.19 In other words, art, science, and philosophy, or at least something
that neurologically defines each as a particular form of the confrontation between
thought and chaos, are seen as more immediately linked to the brain's neural func­
tioning than as mediated products of thought. Cultural mediation does play a role in
the specific forms of practice each endeavor may take and ultimately shapes it in the
form we are familiar with, as for example, in the case of mathematics, especially tech­
nical mathematics (things are somewhat trickier in this regard in philosophy and art) .
Afterword by Arkady Plotnitsky 289

But this mediation acts upon the neurological architecture defined by the three cor­
responding planes of thought or defining these planes.
This is not the place to assess the scientific validity of this remarkable conjecture,
except for noting that, from this perspective, the brain may be seen in terms of a
particular neurobiological form of topological matter, for example via the work of Jean
Petitot. Petitot uses topological ideas and, in part for that reason, figures significantly
in Xin Wei's book. (Deleuze and Guattari, too, invoke Petitot's ideas on several occa­
sions in their work.) "The brain is the mind itself, " Deleuze and Guattari say, a plane
of immanence of thought or, again, a Riemannian manifoldness of (the immanence
of) thought.20 Nor shall I comment here on the necessity to separate these endeavors,
either along these particular lines or otherwise. There are, as Deleuze and Guattari
explain, good reasons to do so within certain limits, either for the corresponding
functionings of the brain or, especially, for the end cultural activities that we name
philosophy, art, and science. Also, while interactions between them in question here
occur in the same plane of thought, this plane does not simply dissolve them, since
their separate identities do emerge on this plane as well, as effects of these interactions.
Thus, "a creation of affects and planes of composition in art" is still, rightly, seen as
artistic activity by Xin Wei (and more may be said on the role of " affect" in his argu­
ment), even when this creation takes place in other fields. I might note in passing
that Xin Wei's concept of enchantment can be related to Deleuze and Guattari's theory
of affect in art, developed in What Is Philosophy? My main concern at the moment is
these interactions, again, as fusions without dissolutions. Indeed, for Deleuze and
Guattari they are just as significant, perhaps ultimately more significant, as the dif­
ferentiation among these fields, which compels them to develop a more complex­
heterogeneous, yet interactive-manifold of thought in relation to which each of these
fields is positioned. As they say:
The three planes, along with their elements, are irreducible: plane of immanence of philosophy,
plane of composition of art, plane of reference or coordination of science; form of concept [in
philosophy], force of sensation [in art], function of knowledge [in science] ; concepts and con­
ceptual personae [in philosophy] , sensations and aesthetic figures [in arts] , figures and partial
observers [in science] . Analogous problems are posed on each plane: in what sense and how is
the plane, in each case-what unity, what multiplicity? But what to us seem more important
now are the problems of interference between the planes that join up in the brain.21

These problems are perhaps more important even for understanding "in what sense
and how is the plane" in each of these endeavors, which thus are also the result of
such interferences from the outside. Xin Wei's book compels us to think along these
lines as well, even in the case of mathematics, let alone in conSidering philosophy,
digital technology, music, dance, theater, or installation art. Beyond its interference
with matter, and thus physics, biology, and neuroscience, or technology (digital
290 Afterword by Arkady Plotnitsky

technology in particular), any given mathematical field, such as topology, is as much

a product of conceptual, including philosophically conceptual, thinking or aesthetic­
like composition as it is of the workings of functions and other technical concepts.
Indeed, most of even the simplest mathematical concepts are already products of
interferences between mathematics and philosophy. There are, Xin Wei's argument
also makes apparent, in part following both Whitehead and Deleuze and Guattari,
important interferences not only between topology and physics, but also between
topology and music, topology and digital technology, topology and dance, and so
forth. Geometry, which as we have seen is often in a (more mathematical) interference
with topology, often enters these interferences as well, as do in fact or in effect other
mathematical fields, certainly algebra and analysis, and the same may be said about
physics, neuroscience, and other scientific fields.
From this perspective, Deleuze and Guattari's concept of the plane of immanence,
defined in Riemannian terms of manifoldness in the first place, should be extended
beyond philosophy to all thought. Indeed it is in effect so extended by Deleuze himself
in his later essay "Immanence: A Life." This extension implies the incessant interac­
tions between this plane and the plane of reference and coordination in science, or
the plane of composition, the plane of poiesis, in art. It also implies the existence of
the material stratum of this plane, which makes this plane a form of topological
matter, as, again, Petitot's research and other recent work in neuroscience (or Deleuze
and Guattari's view of the mind as the brain) suggest; or a form of technological­
topological matter, once technology, such as digital technology, enters the picture.
But then, has any thought ever been possible apart from one technology or another,
any more than apart from matter? Was it not this impossibility that compelled Derrida
to speak, in Of Grammat% gy and other early works, of all speech or even thought
itself as writing, in a special technomaterial sense he gives to this term, rather than
as something that exists first as a pure thought that can then be put into writing (in
the usual sense) ?22
It also follows under these conditions that, as I have suggested from the outset, the
relationships between theory and practice becomes reconfigured by bringing both
together via interferences among philosophy, art, and mathematics and science. Xin
Wei's book clearly advocates this type of refiguring. It might be argued that, insofar
as the book especially explores and argues for the poiesis of this theory-praxis in the
plane of immanence of topological media, it is ultimately art that takes the most
significant role within the spectrum of activities explored and practices pursued here.
That may be questioned; but this emphasis is an occasion for the book's readers,
including practitioners of philosophy, mathematics and Science, and other fields
outside art, to think of the possible role of art and its poiesis, or, to return to Deleuze
and Guattari's idiom, the role of the plane of composition in these fields. Conversely,
or rather reciprocally, it would be equally impossible to unconditionally separate art
Afterword by Arkady Plotnitsky 291

from either philosophy or mathematics and science; and I do not think that, while
this reverse traffic is given less attention in the book, Xin Wei would want to argue
otherwise. An intricately multiple reciprocity between theory and practice appears to
be unavoidable.
Some years ago I was part of a panel at the Postmodern Movement Theater Festival
in Philadelphia; among the other panelists was a member of a prominent philosophy
department, who argued that there is ultimately a line to be drawn between philoso­
phy and art. For "after all," he said, "one cannot dance Kant's Critique of Pure Reason. "
At this point a n actress i n the audience, Julia Varley, who had performed earlier in
the Festival in a piece staged by Eugenio Barba, a student and collaborator of Jerzy
Grotowski (both are invoked, as inspirations, in Xin Wei's book), said: "I can dance
the Critique of Pure Reason. " I am not sure whether Varley, who is also a theater direc­
tor in the tradition of Grotowski and Barba, ever acted (in either sense) on this
promise, although I had then and still have little doubt that it is possible. In any
event, reading Xin Wei's book reminded me that Kant's great work is also a dance with
a very complex philosophical choreography, and that, conversely, a good dance is
often also a philosophical critique, and is certainly a form of topology. It may not
quite be mathematical topology, a field, it might be added, with much choreography
of its own, some of which we witness in Xin Wei's book, but it might not be as far
from mathematical topology as it might appear. Conversely, while always a material
practice, mathematical topology is never purely mathematical, is never free of physics,
technology, philosophy, art, or ethics and politics. This, however, does not stop it from
being mathematics, but instead enables it to be mathematics.
Notes

Preface

1 . Ludwig Wittgenstein, Tractatus Logico-Philosophicus (London: Routledge, 1 994), 3 .

Chapter S u m m a ry

1 . Derrida writes in Of Grammatology: "Within cultures practicing so-called phonetic writing,

mathematics is not just an enclave . . . . [f]he practice of scientific language [in mathematical
writing] challenges intrinsically and with increasing profundity the ideal of phonetic writing and
all its implicit metaphysics (metaphysics itself) . " Jacques Derrida, OfGrammatology, trans. Gayatri
Chakravorty Spivak (Baltimore: Johns Hopkins University Press, 1 9 76), 10.

2. Here are some starting points for exemplary work: http://topologicalmedialab.net, http//
fO.am, and http://topologicalmedialab.net/xinwei!sponge.org.

1 Why This Book?

1. This entire book could be said to be a response to the question: "What is ethico-aesthetic?"
As a starting place, by ethico-aesthetic I mean the fusion that Felix Guattari famously proposed
in Chaosmosis, in which formal, substantive, and truth effects are all taken to be in the same
ontological stratum. Consequently expression's style and prosody simultaneously constitute its
content, its truth; and any action's manner is also an expression of its meaning and its existential
substance. So, the value of some thing made, done, or uttered is coconstitutive with the manner
in which that thing is made. No action, no utterance would be considered merely an imitation
of something else that has a greater degree of reality or truth. Instead, every action or utterance
is to be construed not as a description-imitation thrice removed-of the world, but a making
of the world.

2. David Bohm, Fragmentation and Wholeness Oerusalem: Van Leer Jerusalem Foundation, 1 976), 2.

3 . Isabelle Stengers, "Beyond Conversation: The Risks of Peace, " in Process and Difference: Between
Cosmological and PoststructuraIist Postmodernisms, ed. Catherine Keller and Anne Daniell (Albany:
SUNY Press, 2002), 238.
294 Notes to Chapters 1 and 2

4. Simone Weil, Oppression and Liberty (1958; Amherst: University of Massachusetts Press, 19 73), 38.

5 . Emesto Laclau and Chantal Mouffe, Hegemony and Socialist Strategy: Toward a Radical Demo­
cratic Politics (London: Verso, 200 1 ) .

6 . Akeel Bilgrami, "accidentalism, the Very Idea: A n Essay o n Enlightenment and Enchant­
ment, " Critical Inquiry 32 (2006) : 396.

7 . Max Weber, The Protestant Ethic and the Spirit of Capitalism and Other Writings on the Rise of the
West, 4th ed (New York: Oxford University Press, 2009) .

8. This last is a point made by Sartre, for example, as well as medieval scholastics.

9. See the Research pages of the Topological Media Lab, http://www.topologicalmedialab.net/.

10. Jean Baudrillard, Simulacra and Simulation (Ann Arbor: University of Michigan Press, 1 994) .

1 1 . Despite this, enthusiasts for mechanical virtuality may propose such simulations as a fairy
tale consoling us for the reduction of social interaction in shopping and solipsistic consumption
of media. A noisy bar makes it adequately impossible to relate intersubjectively or create a con­
versation using spoken language; the noise level scales inversely with the estimated communica­
tive sophistication of the clientele.

12. Sha Xin Wei, "Scholarly Spaces, " technical report (Stanford University, 1995).

13. Sha Xin Wei, " Differential Geometric Performance and the Technologies of Writing, " Ph. D .
dissertation (Stanford UniverSity, 200 1 ) .

1 4 . The Topological Media Lab, Concordia University, http://topologicalmedialab.net/.

1 5 . Isabelle Stengers, "A Constructivist Reading of Whitehead's Process and Reality, " Theory
Culture Society 25, no. 4 (2008) : 95-96.

16. Ibid., 107.

1 7. Stengers, "Beyond Conversation, " 238.

18. Karin Knorr-Cetina, Epistemic Cultures: How the Sciences Make Knowledge (Cambridge, Mass. :
Harvard University Press, 1999).

2 From Tech n ologies of Representation to Tech n ologies of Performance

1. Laszl6 Moholy-Nagy, "Neue Gestaltung in der Musik, Moglichkeiten des Gramophons" [New
form in music, possibilities of the phonograph] , Der Sturm, no. 7 (1 923).

2. Andrea Valle, La notazione musicale contemporanea: Aspetti semiotici (Torino: EDT, 2002) .

3 . Ibid., 20. Translations of Valle by the author.

4. Anthony J. Cirone, On Musical Interpretation in Percussion Performance (Galesville, Md. : Mere­

dith Music Publications, 2008), 2 1 .
Notes to C hapter 2 295

5 . Ibid., 34-35 .

6. Ibid., 3 1 .

7 . Gyorgy Ligeti, Streichquartett = String Quartet No. 2 (1 968) (Mainz: B . Schotts Sohne; New York:
Schott Music Corp., 1 9 7 1), 3 .

8 . Gyorgy Ligeti, Volumina, fU r Orgel (Frankfurt, New York: H . Litolff's Verlag, 1967).

9 . Valle, La notazione musicale contemporanea, 2 1 .

1 0 . Krzysztof Penderecki, Threnos den Opfern von Hiroshima, fU r 5 2 Saiteninstrumente (Threnody

to the Victims of Hiroshima, for 52 Strings) (London: E. Eulenburg, 1 9 6 1 ) .

1 1 . Issam EI-Mallah, Arab Music a n d Musical Notation (Tutzing: Hans Schneider Verlag, 1997), 84.

12. Valle, La notazione musicale contemporanea, 5 1 .

1 3 . Ibid., 64. See also Ryoanji's tracks, ibid., 1 1 2.

14. Felix Guattari, Chaosmosis: An Ethico-Aesthetic Paradigm, trans. Paul Bains and Julian Pefanis
(Bloomington: Indiana University Press, 1995), l 3 .

1 5 . Rudolf Laban, Schri{ttanz: Eine Vierteljahresschrift (Leipzig: Georg Olms, reprint 1 9 9 1 ) , vol. 1
( 1 928), 4.

16. Rudolf von Laban and Lisa Ullmann, Choreutics (London: Macdonald and Evans, 1 966), 67.

1 7 . Rudolf von Laban, A Vision of Dynamic Space, compo Lisa Ullmann (London: Taylor and
FranCis, 1 984).

18. Ibid., 1 8, 23 .

1 9 . Ibid., 14, 20, 22.

20. Ibid., 1 2, 1 4, 1 6, 1 7, 27, 5 8-59.

21. Ibid., 3 3 .

2 2 . Laban, Schri{ttanz, vol. 1 ( 1 928), 3 9 .

23. Paul Kaiser a n d Shelley Eshkar went on to form the OpenEndedGrqup http://
openendedgroup.com/index.php/artworks/ghostcatching/. For Wechsler and Palinqrome, see
http://www.palindrome.de.

24. Gilbert Simondon writes : "It is this sensitivity to information on the part of machines, much
more than any increase in automatism that makes possible a technical ensemble . . . . The machine
with superior technicality is an open machine, and the ensemble of open machine, assumes man
as permanent organizer and as a living interpreter of the interrelationships of machines. Far from
being the supervisor of a squad of slaves, man is the permanent organizer of a society of techni­
cal objects which need him as much as musicians in an orchestra need a conductor. " Gilbert
Simondon, On the Mode of Existence of Technical Objects, trans. Ninian Mellamphy ( 1958; London:
University of Western Ontario, 1 9 80), 4.
296 N otes to Chapter 2

25 . Cage is the obvious reference. But see Iannis Xenakis, "Determinacy and Indeterminacy, "
Organised Sound 1 ( 1 996) : 1 43-1 5 5 .

2 6 . Ludwig Wittgenstein, Philosophical Investigations, trans. G. E . M. Anscombe (New York: Mac­

millan, 1958), §80.

27. Ibid., §20 1 .

28. Ibid., §8 1 .

29. Ibid., § 1 9 9 .

3 0 . Ibid., §48.

3 1 . One of Wittgenstein's rare direct injunctions-"Don't regard a hesitant assertion as an asser­

tion of hesitancy" (Part 2, x)-applies also to a distinction between a continuous theory and its
approximation in computational form.

32. Valle, La notazione musicale contemporanea, 1 9 .

3 3 . Hypertext structures popular i n the 1960s are only the most obvious elaboration, but they
are still locally unidimensional.

34. See the Carnegie-Mellon database of "mocap" data, as a storehouse of machine- but not
human-readable representations of human movement: http://mocap.cs.cmu.edu.

3 5 . There is a large amount of documentation on dance technology using motion capture for
real-time performance instead of off-line representation and analysis. See for example work by
Marie-Claude Poulin and Martin Kusch, kondition pluriel: http://www.fondation-langlois .org/
html!e/page.php?NumPage=23 8; and Vangelis Lympouridis Orient system, University of Edin­
burgh, http: //dancetech.ning.com/video/video/listForContributor?screenName=3meilrf2gn92s.

36. Gerard Genette, The Work of Art: Immanence and Transcendence, trans. G. M. Goshgarian
(Ithaca: Cornell University Press, 1997), 30.

3 7. Ibid. , 3 1 , 72.

38. Ibid., 58.

39. Ibid., 5 9-6 1 .

40. Issam El-Mallah, Arab Music and Musical Notation, Publications o f the Oman Centre for Tra­
ditional Music (Tutzing: Hans Schneider Verlag, 1 9 9 7) , 25, 89.

41. Ibid. , 270.

42. Ibid., 49.

43. Ibid., 49, citing R. G. Kieswetter (1 842) .

44. El-Mallah, Arab Music and Musical Notation, 267, 269.

45. Wittgenstein, Philosophical Investigations, §3 1 .

N otes to Chapter 2 297

46. El-Mallah, Arab Music and Musical Notation, 46.

4 7 . Ibid., 1 1 4.

48. Genette, The Work of Art, 59-60.

49. Ibid., 60.

50. Ibid., 79.

5 1 . Ibid., 93.

52. El-Mallah, Arab Music and Musical Notation, 67.

5 3 . Valle, La notazione musicale contemporanea, 145, citing Reginald Smith Brindle, The New Music:
The Avant-Garde since 1 945 (Oxford: Oxford University Press, 1987).

54. Valle, La notazione musicale contemporanea, 1 6 .

5 5 . Gilbert Simondon, "The Position o f the Problem o f Ontogenesis, " trans . Gregory Flanders,
Parrhesia, a Journal of Critical Philosophy 7 (2009) : 4-1 6 .

56. Antonin Artaud, Th e Theater and Its Double: Essays (1938; London: Calder and Boyars, 1970), 89.

5 7 . Genette, The Work ofArt, 5 9-60.

58. Valle, La notazione musicale contemporanea, 1 .

5 9 . Ibid., 3-4.

60. Ibid., 1 4 .

6 1 . Ibid., quoting " International Conference o n New Musical Notation, Report, " e d . Herrmann
Sabbe, Kurt Stone, and Gerald Warmeld, Interface: Journal of New Music Research 4 (1975): xv.

62. Ibid., 1 5 , citing Kurt Stone, Music Notation in the Twentieth Century (New York: Norton, 1980) .

63. Ibid., 4 1 5 .

6 4 . George Crumb, Ancient Voices o f Children, text b y Federico Garda Lorca (New York: C. F.
Peters, 1 9 70) .

65. John Cage, Fontana Mix (New York: C. F. Peters, 1 965).

66. See Mike Cooley's discussion of expert design play that CAD programs afforded in Architect
or Bee: The Human Price of Technology (London: Hogarth Press, 1987).

6 7 . Detailed in many histories of electronic music and video; see for example Achim Heidenreich,
'' 'Shaping Electronic Sounds Like Clay': The Historical Situation and Aesthetic Position of Elec­
troacoustic Music at the ZKM Institute for Music and Acoustics, " Organised Sound 14, no. 3
(December 2009): 248-256.

68. Yves Abrioux, talk, Society for Science, Literature, and Arts (SLSA), Riga, 20 10. It goes without
saying that focusing on the performer is not the same as focusing on the performance, so
298 N otes to Chapters 2 a n d 3

classifications like Genette's characterizing a performance as an " autographic object" are

beside the point.

69. Navid Navab, Jerome Delapierre, Bruno Gagnon, 2008. http://topologicalmedialab.net ->

Interstitial improvisation.

70. Martin Heidegger, The Metaphysical Foundations of Logic (Bloomington: Indiana University
Press, 1 984), 128.

71. Karen Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter
and Meaning (Durham: Duke University Press, 2007), 1 3 3 .

7 2 . Wittgenstein, Philosophical Investigations, §§ 1 9 7-198. And §202: obeying a rule i s part o f a

"practice. "

7 3 . This attribution seems painted with a rather broad and seemingly orientalizing brush,
however. Bruno Latour and Peter Weibel, Making Things Public: Atmospheres of Democracy (Cam­
bridge, Mass. : MIT Press; Karslruhe: ZKM/Center for Art and Media, 2005), 25.

3 Performance i n Responsive Environments, the Performative Event

1. See RoseLee Goldberg's surveys of performance art, Performance: Live Art since the '60s (New
York: Thames and Hudson, 2004) and Performance Art: From Futurism to the Present, rev. ed. (New
York: Thames and Hudson, 200 1 ) .

2. In this book, " computation" alone should b e understood as any transformation effected b y a
digital computer, whether it is based on numerical or algebraic representations or representations
of images, sounds, gestures, etc. "Computational system" or "computational tool" should be
understood as any piece of software or combination of software plus device built using computer
technology. So the term is considerably broader than symbolic computation, which I take to be
essentially synonymous with symbolic algebra or numerical computation. The term " calculation"
will typically imply the partially forced moves (Pickering's description) made by human math­
ematicians in their symbolic or numerical manipulation. This respects the conventional use of
the term in mathematical literature, where the phrase "by a calculation it follows that . . . "
assumes that a human performs the manipulation. This is not merely a terminological fine point,
since human "calculations" may not be computable (if, for example, they implicitly appeal to
the strong axiom of choice).

3 . Geometer's Workbench, Sha Xin Wei and Fran�ois GUimbretiere, Stanford University, 200 1 .
http://vimeo.com/tml/workbench.

4. See, for example, Sha Xin Wei, review of Mathematica 2, American Mathematics Society Notices
(May/June 1 992) : 428.

S . What human mathematicians mean by "calculation" and "quantity" is quite different from
what computers do as calculation. For example, a mathematician can "compute" some algebraic
invariant of an n-dimensional topological manifold, where n is not specified, with no real number
N otes to Chapter 3 299

(e.g., 0. 1 4285 7 1 42857 . . . ) in sight, and perhaps not even any of what programmers call con­
stants. For a digital computer, however, calculation always devolves to the manipulation by
software code of some data in a register of a computer processor that encodes not a real number
but a "floating-point number, " which is an electrical approximation to a number. Moreover, a
mathematician can "compute II some "quantity" using procedures (such as an axiom of choice
from a transfinite set) that are in principle noncomputable. This means that there is no mechani­
cal procedure using any Turing-complete algorithm that can even in principle, with arbitrary
amounts of memory and speed, perform the operation. Most of contemporary mathematics is,
strictly speaking, noncomputable. For a popular version of this point, see Roger Penrose, The
Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics (New York: Penguin,
1991).

6. There was a period in the 1 9 80s when conferences were held specifically t o bring mathemati­
cal scientists together with computer scientists and technologists to discuss mathematics and
computers. In fact, one of the first conferences was held at Stanford. In 1 988, Jon Barwise started
a column in the Notices of the American Mathematical Society, titled " Computers and Mathemat­
ics, " that gave a high-profile review of many areas of computation and mathematical research,
ranging from logic and symbolic algebra to visualization, pedagogy, and even problems of
systems administration and software development.

7. One of the major achievements of topology in the early part of the twentieth century was a
rigorous and supple definition of continuity in terms of mappings of open sets, independent of
metric. This did not have any application in a digital representation. Since Kronecker and Hilbert,
foundationalist approaches to the notion of infinity and the reciprocal notion of infinitesimals
have focused almost exclusively on models of the real line. For textbook expositions see H. L.
Royden, Real Analysis, 3rd ed. (New York: Macmillan, 1 988), or Edward C . Titchmarsh, The Theory
of Functions, 2d ed. (London: Oxford University Press, 1 939). For an important historical essay,
see Hermann Weyl's 1 9 1 8 essay Das Kontinuum, available in English as The Continuum: A Critical
Examination of the Foundation of Analysis (Mineola, N.Y. : Dover, 1994).
One conspiCUOUS exception to this industry was Ludwig Wittgenstein, whose approach to
mathematics in his Cambridge lectures on the foundations of mathematics in 1939 was concerned
not with logical completeness, or axiomatic foundations, but with use, practice, and efficacy.

8. My favorite example is the " Point2D, Point3D" design problem. The Taligent object-oriented
operating system and application environment had a sophisticated factorization Of geometry
'
classes from graphics classes. However, a geometric point had to be declared in C++ as either a
Point2D (an object embedded in R2), or a Point3D (an object embedded in R3) . This was perfectly
reasonable, from the point of view of a computer graphics system composer who was concerned
with mapping programmer constructs "naturally" and effiCiently to the independent 2D or 3D
graphics subsystems. But to a geometer this was a very strange convention, since a pOint has a
very simple and intrinsic characterization as a zero-dimensional object, independent of any
ambient setting. Moreover, this rigidity made it prohibitively cumbersome to code geometric
operations on points in non-Euclidean ambient spaces, or indeed on points in a manifold of
unspecified dimension.
300 Notes to Chapter 3

9. For example, a typical gesture recognition system designed around a small set of canonical
strokes with preassigned meanings, such as Cut, Copy, Paste, is not easily extended by the user
to accommodate his/her freehand strokes.

10. See Sha Xin Wei, " Differential Geometric Performance and the Technologies of Writing,"
Ph.D. dissertation, Stanford University, 2000.

1 1 . I describe Riemannian manifolds in chapter 6, along with a comment on their latent appear­
ance in Deleuze and Guattari's cosmopolitics. A harmonic function f M � R is a critical point
of the Laplace-Beltrami operator on a Riemannian manifold,

I1f = 0.
For a general map f between Riemannian manifolds, f M � N, the " action" (the analog to
"energy" in variational calculus as described earlier) is taken to be the Dirichlet energy:

E [t/>] :: J) !d¢! !2 * 1 .
A harmonic map is defined as a critical point of E [fJ , i.e., a solution to the Euler-Lagrange differ­
ential equations derived by setting the variational derivative of E[fJ to zero in order to find its
critical "points" f

12. Roy Harris, Signs of Writing (London: Routledge, 1995).

1 3 . By examining the exteriorization of thought, I pay attention not just to what happens
between the ears of a mathematician, but to the actions and tools situated in the built environ­
ment and to ambient technologies and disciplines.

14. By naive space-time, I mean the standard four-dimensional Euclidean model: R3 x R, with
the flat metric implicitly assumed by Kant, together with the model of event as a point <x, D.
By "classical" (in a sense extended from quantum mechanics), I mean a model of space-time and
events with no intertwining of the observer's experiential state with the state of the observed.

15. Search " Hubbub" in the Topological Media Lab site, http://topologicalmedialab.net.

1 6 . On social and literary functions of lettering in public architecture since classical times in the
West, see Armando Petrucci, Public Lettering: Script, Pawer, and Culture (Chicago: University of
Chicago Press, 1993). Also satirized en passant in Monty Python's film The Life of Brian ( 1 9 79) .

1 7. Heraclitus, fragment 5 1 , in The Art and Thought of Heraclitus: An Edition of the Fragments with
Translation and Commentary, ed. and trans . Charles H. Kahn (Cambridge: Cambridge University
Press, 1 9 79), 5 3 .

1 8 . Alain Badiou, Being and Event, trans. Oliver Feltham (London: Continuum, 2006), 1 79 .

1 9 . For a critical analysis, see Ricardo L. Nirenberg a n d David Nirenberg, " Badiou's Number: A
Critique of Mathematics as Ontology," Critical Inquiry 3 7 (20 1 1 ) : 583-61 3 .

20. Christopher Alexander, The Timeless Way o f Building (New York: Oxford University Press,
1 9 79), x.

21. Jerzy Grotowski, Tawards a Poor Theatre (New York: Simon and Schuster, 1 9 68), 1 9 .
Notes to Chapter 3 301

22. Peter Brook, The Empty Space (New York: Atheneum, 1 9 68).

23. Antonin Artaud, "Theater of Cruelty, First Manifesto," in The Theatre and Its Double: Essays
(London: Calder and Boyars, 1 9 70), 93-98.

24. "In the autumn of 1973, [Peter Brook and the International Centre of Theatre Research in
Paris] conducted a five week work period at the Brooklyn Academy of Music . . . including Helen
Mirren, working with four musicians and noted composer Elizabeth Swados" (http://www.lib
.berkeley.edu/MRC/pom02.html) . See Peter Brook, Empty Space, film (BBC, 1 9 73).

25 . To be precise, if the observer's state is f, the observed system is x, and H is a Hermitian self­
adjoint operator representing the apparatus, then the result of observation is given by the inner
product <f, H[x]>.

26. Some of the critical concerns of this chapter date from a seminar in interaction and media
that I coordinated at Stanford from 1995 to 1 9 9 7 . Thanks to partiCipants of the Interaction and
Media Group, whose earliest members included Niklas Damiris, Helga Wild, Alice Rayner, Anne
Weinstone, Ben Robinson, Larry Friedlander, and Diane Middlebrook.

27. Sponge's installations were originally impelled by questions about the phenomenology of
performance, such as: What symbolically, affectively charges an event? How and why does a gesture
make meaning? If we provisionally bracket verbal narrative and invite non experts to improvise
movements that nuance time-based media in a common performative space, how can we possibly
sustain experiences that are as compelling as the works of Brecht, Muller, Grotowski, et al.? One can
consider this a finely tuned question sounding in the methodological silence left by Antonin Artaud's
call to liberate theater from the tyranny of the text, from what he disparagingly called dramatic
literature. In what sense and to what degree this is possible may be questioned, because, as Derrida
argued in his essay on Artaud, "Presence, in order to be presence and self-presence, has always and
already begun to represent itself. " Jacques Derrida, "The Theater of Cruelty and the Closure of
Representation," in Writing and Difference (Chicago: University of Chicago Press, 1 9 78), 249.
Nonetheless there have been diverse responses to Artaud's challenge over the past century in
experimental theater. Collectives such as Sponge in North America and FoAM in Europe, and
the university-based Topological Media Lab for art research and speculative engineering, have
used the emerging technologies of computational responsive media in idiomatic ways, with
insider knowledge and practices from technoscientific research and development. Although there
is much to be said for cargo cult approaches to technology, I do not take the te Chnology of
electronic devices, protocols, and software for granted as naturalized, shrink-wrapped black
boxes. I pursue this material, embodied craft as a way to open the ground for critical and artistic
practice. At the same time, I maintain that we need to remain acutely conscious of the epistemic
frames constructed and imposed by techno scientific practice, a task which becomes more chal­
lenging the more deeply we enter the black box.
Over the past ten years, my work with these per formative spaces has been guided by the
demands of performance research, particularly questions concerning the phenomenology of
performance. I distinguish performance research such as TGarden from the making of particular
performance-event-installations or aesthetic obj ects such as the TG200 1 event that the TGarden
consortium exhibited in Ars Electronica and V2.
302 N otes to C hapter 3

FoAM was founded in 2001 by Maja Kuzmanovic, with Nik Gaffney, Lina Kusaite, Cocky Eeck,
and other artists and engineers.
See http://sponge .org and http://fo.am for links to TGarden, txOom, Moob, and trg responsive
space projects that derived from the TGarden architecture.

28. In this chapter, I borrow the type-token distinction: "TGarden" will refer to the concept and
the research project, and "TG200 1 " will refer to the instance that was exhibited in 2001 in Austria
and the Netherlands.

29. See video of this "epiphany" : http://vimeo.com/tml/tg200l -epiphany.

30. I adopt the notion of "bowing" through responsive temporal media from Joel Ryan and Chris
Salter. Ryan was the principal composer and creator of the sound instruments for TG2001 , and
Salter codesigned the sound environment.

3 1 . http://vimeo.com/tml/tg2001 -hopskip.

32. Complexity has often been valorized as yielding phenomena from large collections of dis­
crete entities in networks of relations modeled on graphs, phenomena that one does not observe
in an individual entity. However, I maintain that complexity does not equal richness, just as
panoply of choice does not equal freedom (as anyone encountering the bewildering array of
differently processed coffee beans in equally tasteless combinations of flavors could attest).
Indeed complexity inevitably tends to overwhelm sense and value. Generally, discrete structures
exhibit this sort of combinatorial, exponentially explosive complexity as one adds more ele­
ments, components, or dimensions to the set. The same is true of networks of discrete nodes
and arcs. As these networks grow larger, we can attempt to salvage the situation by aggregating
subgraphs into nodes, but that merely defers the explosion by one step. Eventually, combinatorial
complexity overwhelms us. On the other hand, if we believe that human experience is continu­
ous, dense and rich, but not combinatorially complex, then it should be a healthy challenge to
try to make our performance technologies themselves topological rather than combinatorial. See
chapter 3 .

3 3 . Joel Ryan a n d Chris Salter have worked intensely o n this and have invented some promising
strategies, described in "TGarden: Wearable Instruments and Augmented Physicality, " Proceed­
ings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal,
Canada, NIME03-87.

34. Tuning, in this context, refers to the delicate process of finding the regions in parameter
space corresponding to the most sensitive, salient, and expressive sensing and response of a
responsive media environment. For example, a flexible sensor may report bend ranging from say,
o to 90 degrees, but it may be most sensitive (Le., report values that change most rapidly for a
given increment of physical flex), most accurate, and most repeatable only in a subrange of
physical flex. In order to make a medium respond most palpably to flex using that sensor, the
software systems mapping physical flex should use only the numerical data that is reported from
the particular subrange, using a mapping that is invariant (or at least predictably variant) over
a range of repeated trials.
Notes to Chapter 3 303

35. Since 200 1 , and especially since 2006, the Topological Media Lab has initiated a few waves
of research in the area of gesturally controlled electroacoustic instruments for extramusical as
well as musical sound, with expert performers and inventors affiliated with the Center for New
Music and Technology (CNMAT) in Berkeley; STEIM, the center for electroacoustic musical per­
formance in Amsterdam; and the Institut de Recherche et Coordination Acoustique/Musique
(lRCAM) in Paris.

36. In a different direction, Benoit Maubrey and Die Audio Gruppe / The Audio Ballerinas have
created witty and whimsical public performances with simple electronics that record and process
sound directly on the body of the performing "audio ballerinas. "

3 7 . O f course, not every example is taken from actual performances . But since w e have built
a performance engine, I am careful to describe examples of system responses that we have
built in prototype form and can easily flesh out in performance with the existing apparatus.
In the few places where the described experience is speculative, I clearly state this. To see the
TGarden systems in action, please visit the Topological Media Lab website and look at some
of the video documentation. Much of the performance-installation videos were taken in 2001-
2002. Later video documentation shows the much-enhanced responsive media system that we
have built since then in Atlanta. http://www.sponge.org/projects/m3_tg_intro.html or http://
topologicalmedialab.net/tgarden/index.html.

38. More precisely but also more conceptually, we move from the base manifold of observables
to a space of transformations on that manifold. Briefly and informally, " observables" are the set
of varying parameters reported by sensors tracing the physical activity or state of the people and
the environment during an event. These parameters can vary through a range of values in a
non-Euclidean space, a "base manifold" whose potentially high dimension and complicated
shape reflect the in principle arbitrarily complex set of physical observables. From an idealist
perspective of classical physics of mechanical systems and more radically of quantum mechanics,
the event is identified with what can be observed via some experimental instrument, whether
an organic sense or an extension of the senses. Esse est percipi. But rather than rest with descrip­
tions of the physically observable configurations of matter and media, the TGarden is designed
around the notion collections of transformations that act on, or modify, the environment. These
collections may be construed as sets of transformations on the manifold of observables.

39. I say complexify, not enrich, because I believe that such combinatorial approaches inevitably
make the user's experience more complicated, but that our experience in the world can be rich
without complexity. In other words, richness is not synonymous with having numerous discrete
choices and possibilities. My colloquial example is: Presenting a customer with 1 00 different
variants of coffee at a coffee stand does not sum to a rich experience of drinking coffee.

40. I use the term " alphabetic" thinking of Brian Rotman's Derridean critique of the linear
semiotics that derives from alphabetic representations of language. He terms the literary analysis
bound conceptually by such artifacts, which thinks of all communication and ratiocination and
creation as reducible to that which can be represented or more extensively conducted in alpha­
betic text, the " alphabetic dogma. "
304 Notes to Chapters 3 and 4

4 1 . I say "quasi-physical systems" to emphasize that these use the same sort mathematical and
software methods that are used to simulate "real"-world physical processes. But since we are
seeking expressive power and enough phenomenal richness or tangibility to sustain high
symbolic charge, there is no need to mimic physical reality. It has to be as rich as the ordinary
physics, but it can be different. For example, one of my students, Yoichiro Serita, implemented
a wave equation model that we could apply to live video streams. Then frame-differencing gave
us the effect of ripples that appeared only where we moved. But then he modified the kernel to
be a function of position, which made it possible to make the waves flutter like cilia along con­
tours around a thickening of a path, something that was as rich as some "naturally occurring"
process but which you would never observe in a physical lab. Jos Stam, in his work with fluid
dynamics for game design, articulates the strategy of pursuing visual plausibility rather than
adherence to "accurate" physics.

42. It seems fruitful to think of gesture as movement that elicits responses from the world. There
can be any number of responses, any multiplicity, and any degree of deferral. Sha Xin Wei,
" Resistance Is Fertile: Gesture and Agency in the Field of Responsive Media, " Configurations 10,
no. 3 (2002) : 439-4 72.

43. The exploration of continuous ontology is a joint investigation with Niklas Damiris.

44. Search Meteor Shower, Cosmicomics, Ouija, Frankenstein's Ghosts, and Remedios Terrarium
in http://topologicalmedialab.net.

45. Michael Montanaro, a choreographer with three decades of performance production practice,
is Chair of Contemporary Dance at Concordia University.

46. Soo-yeon Cho led these experiments as improvisatory movement exercises, with 2-4 dancers
and members of the Topological Media Lab in June and July in the SO' x SO' X 24' Hexagram
Blackbox at Concordia University. Search Ouija in Topological Media Lab website: http://
www.topologicalmedialab.net.

4 7 . San Francisco Exploratorium, Technopolis show, curated by Sponge, 2002.

48. Sha Xin Wei, IL Y A proj ect, 2007-20 1 2, partially supported by the Fonds quebecois de la
recherche sur la societe et la culture (FQRSC).

49. Van Nelle Fabriek Membrane by Sponge (Harry Smoak, Yoichiro Serita, Sha Xin Wei, Chris
Salter), Dutch Electronic Art Festival 2004.

SO. Sha, "Resistance Is Fertile . "

4 Substrate

1. Strictly, a field is a mapping F from a manifold M to a vector bundle over that manifold, such
that F[x] is a vector in a given vector space V. Such a mapping is called a section of the vector
bundle . In physics, such mappings provide one way to describe the potential movement of
particles subject to a field of forces. Less dry examples abound: wind with smoke or birds; water
Notes to C hapter 4 305

with ink, leaves, or fish; and market potentials with goods or capital. F will be continuous or
differentiable, at least up to the differentiability of M itself.

2. Private communications with Tony Bell, Interval Research, August 2000.

3. I mean fresh in the sense in which a poet, using the same lexicon and grammar as anyone
else, can make language that appears fresh.

4. A plenum is a material continuum, with boundlessly variable and varying density and multi­
plicity. Therefore, it cannot be a totality nor should the concept be used in a totalizing way. By multi­
plicity, I mean an ontic coefficient on a material manifold which takes the manifold as if it were
multiplied: m: M --> W. If m is identically 1, then we have the ordinary situation of a " single
copy" of the manifold. But m could be greater than 1, or even less than 1. And note that the
multiplicity does not have to be an integer, something that no naive sense of copy accommodates.
Multiplicity is closely related to but different from density. Although formally it is the same, rep­
resented by a real-valued function defined on M, think instead of density as a function defined
on M (of multiplicity 1) and measuring some sort of scalar field as an intensity in M. See Frank Morgan,
Geometric Measure Theory: A Beginners Guide, 4th ed. (San Diego: Academic Press, 2009), 1 79-1 80.

5. Which Whitehead in turn derived from Locke. Alfred North Whitehead, Process and Reality:
An Essay in Cosmology, corrected ed., ed. David Ray Griffin and Donald W. Sherburne (New York:
Free Press, 1 9 78), 30.

6. Akeel Bilgrami, " accidentalism, the Very Idea: An Essay on Enlightenment and Enchant­
ment, " Critical Inquiry 32 (Spring 2006) : 396-398.

7. Whitehead, Process and Reality, 64.

8. Here I am appealing to a precise notion of connectedness from point set topology: a subset
a of a topological space X is connected if it is not the disjoint union of two open sets A, B in
X. See chapter 6 .

9. Gilbert Simondon, O n the Mode o f Existence o f Technical Objects, trans. Ninian Mellamphy
(London: University of Western OntariO, 1 980), 5 1 .

10. This figure of a fluctuating substrate permeates recent cosmology a s well. See Andrei Linde's
chaotic inflationary model extending Alan Guth's inflationary cosmology: A. D. Linde, Inflation
and Quantum Cosmology (Boston: Academic Press, 1 9 90). And John A. Wheeler's quantum foam
models: " Geons, " Reviews of Modem Physics 97 ( 1 9 5 5 ) : 5 1 1-536; Geometrodynamics (New York:
Academic Press, 1 962) .

1 1 . An algebraic invariant of a flow (such as its winding number) evaporates the processual
temporal quality of the flow. It can only be a static abstraction of a process, and cannot be
durational any more than the sequential time that Bergson found inadequate. Any attempt to
model a temporal process in terms of sequential iterates like 7*[x] , k = 1, 2, 3, . . . , cannot escape
the problem that at each integer index k the operator is timeless, nonprocessual.

12. Poincare had access to all the mathematical elements that Einstein had in hand to assemble
the General Theory of Relativity, but did not do so because of what has been viewed as a residual
306 N otes to C h a pter 4

conservative adherence to a universal background ether. In an essential way, however, Poincare

was onto something, because substituting the concept of a general Riemannian manifold in place
of a universal ether to serve as the substrate would have anticipated general relativity.

1 3 . Gilles Deleuze and Felix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia (Min­
neapolis: University of Minnesota Press, 1 9 8 7); 95 .

1 4 . Ibid., 9 7 .

1 5 . Armando Petrucci, Public Lettering: Script, Power, and Culture (Chicago: University o f Chicago
Press, 1 993).

1 6 . " [I]nsulting or 'fighting' words-those which by their very utterance inflict injury . ".
Chaplinsky v. New Hampshire, 3 1 5 U.S. 568 (1 942).

1 7 . For a different and extensive treatment of touch, the concept of the body and the state, see
Erin Manning, Politics of Touch: Sense, Movement, Sovereignty (Minneapolis : University of Minne­
sota Press, 2007).

1 8 . The Art and Thought ofHeraclitus: An Edition of the Fragments with Translation and Commentary,
ed. and trans . Charles H. Kahn (Cambridge: Cambridge University Press, 1 9 79), 1 1 .

1 9 . Ibid., 1 8 .

2 0 . Ibid., 1 3 8 .

2 1 . Ibid., 5 2 .

2 2 . Emmy Noether, "Invariante Variationsprobleme, " Nachr. d. Koenig. Gesellsch. d. Wiss. zu Goet­
tingen, Math-phys. Klasse ( 1 9 1 8) : 235-25 7; trans . M. A. Tavel as "Invariant Variation Problems,"
Transport Theory and Statistical Physics I , no. 3 (1971): 1 83-207 . Nina Byers, "E. Noether's Discov­
ery of the Deep Connection between Symmetries and Conservation Laws , " presented at the
Symposium on the Heritage of Emmy Noether (December 2-4, 1996). For a general exposition,
see I. M. Gelfand, S. V. Fomin, and Richard A. Silverman, Calculus of Variations (Mineola, N.Y. :
Dover, 2000) .

23. The Art and Thought of Heraclitus, 5 3 .

2 4 . I u s e "vectorial" and "vectorial field" to mark that I a m following Whitehead's u s e o f "vecto­

rial" to describe the directedness of prehension in his framework, to borrow as he does from the
mathematical concept of vector field, without necessarily expecting all the phenomena of interest
to reduce neatly to a fonnal definition of vector field from linear algebra, or analysis on manifolds.

25 . For a discussion of dimension and cardinality of topological spaces, density, and measure,
see chapter 6 .

2 6 . After defining govemmentality a s a specific complex o f institutions and procedures, political

economic knowledge, and security apparatuses targeting population, along with a process since
the Middle Ages of the transformation of the " state of justice" into a govemmentalized admin­
istrative state, Foucault remarks: "Maybe after all, the state is no more than a composite reality
Notes to Chapter 4 307

and a mythicized abstraction . . . . Maybe what is really important for our modernity . . . is not
so much the statization of society, as the 'governmentalization' of the state. " Michel Foucault,
"Governmentality, " in The Essential Works of Foucault, 1 954-1 984, vol. 3, Power, ed. James D.
Faubion (New York: New Press, 2000), 220.

27. I adapt this from Roy Harris's characterization of a theory of languaging that assumes discon­
nected minds transmitting pieces of writing that conduct thought from mind to mind. See Roy
Harris, Signs of Writing (London: Routledge, 1995).

28. As a note, the impatient, both this simpler notion of care and Sorge, can be given a nonan­
thropocentric interpretation (viz. Whitehead, for example), but I ladder this development with
provisional understandings.

29. Maurizio Passerin d'Entreves, " Hannah Arendt, " Stanford Encyclopedia of Philosophy, http://
plato.stanford.edu/entries/arendt/. See Hannah Arendt, The Human Condition, Charles R. Wal­
green Foundation Lectures (Chicago: University of Chicago Press, 1 9 5 8), 9 .

3 0 . Miguel Vatter, "Natality and Biopolitics i n Hannah Arendt, " Revista d e Ciencia Politica 26, no.
2 (2006) : 1 5 3 .

3 1 . Vatter remarks that freedom's "Benjaminian" particularity seems paradoxical against Kantian
automaticism. But in the section of chapter 5 titled "On Local and Global," I discuss how the
global appears in the local also in a nonanthropocentric sense.

32. Vatter, "Natality and Biopolitics in Hannah Arendt, " 1 5 1 .

3 3 . Ibid., citing Roberto Esposito.

34. Deleuze and Guattari, A Thousand Plateaus, 9 7 .

3 5 . Adele Goldberg, SmallTalk-80: Blue Book: Th e Language and Its Implementation (Reading, Mass.:
Addison-Wesley, 1983); Adele Goldberg, SmallTalk-80: Red Book: The Interactive Programming Envi­
ronment (Reading, Mass. : Addison-Wesley, 1983).

36. Grady Booch, Object Oriented Design with Applications (Redwood City, Calif. : Benjamin/
Cummings, 1 9 9 1 ) .

3 7 . However, category theory h a s found life i n a particular philosophical circle. Alain Badiou in
his Logic of the World has turned to category theory as a way to enrich the set-thioretic-and
spars�ntology in his prior magnum opus Being and Event. By contrast, I take a m 6�e modest,
"bare-handed" approach to mathematics = ontology. To sketch my difference briefly, particular
mathematical theories-say of point set topology, CW complexes, or algebraic sequences of
homology groups-are particular modes of articulating the material of the world. Category theory
is not a particular mode of material articulation but a meta-mathematical theory about how to
map one mathematical ontology into another via a particular formal association. Badiou's sophis­
ticated Platonist argument claims that his forms are ontology.

38. Sha Xin Wei, " Calligraphic Video: Using the Body's Intuition of Matter, " International Journal
of Creative Interfaces and Computer Graphics (2009) .
308 Notes to Chapters 4 and 5

39. In Heidegger's meditation on the essence of the jug, hinted by Laozi: "gathering . . . does
speak to the nature of the jug . . . . The jug is a thing insofar as it things . . . . The thing is not
'in' nearness, 'in' proximity, as if nearness were a container. Nearness is at work in bringing to
bear, as the thinging of the thing." Martin Heidegger, "The Thing," in Poetry, Language, Thought
(New York: HarperCollins, 2001), 1 65-182.

40. Jean-Pierre Changeux and Alain Connes, Conversations on Mind, Matter, and Mathematics
(Princeton: Princeton University Press, 1995).

5 Ontogenesis

1. Even mathematics has its materiality. One could perhaps argue this from Alain Badiou's
mathematics = ontology thesis, but I would appeal to a phenomenological argument. See Sha
Xin Wei, " Differential Geometric Performance and the Technologies of Writing, " Ph.D. disserta­
tion, Stanford University, 2000, 23-69.

2. Mary Ann Doane, The Emergence of Cinematic Time: Modernity, Contingency, the Archive (Cam­
bridge, Mass. : Harvard University Press, 2002) .

3 . Orit Halpern, "Dreams for Our Perceptual Present: Archives, Interfaces, and Networks in
Cybernetics, " Configurations 13, no. 2 (2007): 283-320.

4 . Of course this has a distinguished lineage ranging from Heraclitus through Heidegger. Dissatis­
fied with his treatment of spatiality and temporality in the ontology of being and becoming in
Sein und Zeit, Heidegger searched in his late essays for a way forth from the provisional equipri­
mordiality of these basic abstractions. In "Building, Dwelling, Thinking, " for example, he referred
to the burial site inside the post of a hut as part of the irreducible experience of inhabiting that
dwelling.

5. Humberto R. Maturana and Francisco ]. Varela, Autopoiesis and Cognition: The Realization of the
Living (Dordrecht: D. Reidel, 1980), 1 3 3 .

6. The living i s material and more than the material, therefore the formal-immaterial i s not living.

7. In the appendix to the book Noncommutative Geometry, Quantum Fields and Motives (Provi­
dence, R.I.: American Mathematical Society, 2008), Alain Connes and Matilde Marcolli construct
an interpretation of the passage of time by generalizing the concept of parameterizing a continu­
ous family in some topological space to the concept of automorphisms satisfying some condition.
There's nothing wrong technically with this clever generalization, but the problem is method­
ological: starting by looking for a well-defined automorphism generalizing one-parameter sub­
groups constrains the search to homologous abstractions and is inadequate to the much richer
phenomena of history or lived experience. Another case of WYSIWYE-What You See Is What
You Expect.

8. Martin Heidegger, The Metaphysical Foundations ofLogic ( 1 928; Bloomington: Indiana Univer­
sity Press, 1 984), 263.

9. Ibid., 204.
N otes to Chapter 5 309

10. Ibid., 203 .

1 1 . In his Metaphysical Foundations of Logic, Heidegger gave an extended treatment of temporality

(naming the fact that time cannot be construed as something outside of time, but whose essence
is temporal) . After contrasting his temporalized monadology against Leibniz's Cartesian monad
as an autosufficient substance, Heidegger wrote:

Time is essentially a self-opening and expanding into a world. I will not go into the comparison any further,
particularly the question of the extent to which one might conceive the interpretation of Dasein as temporality
in a universal-ontological way-just as the monadology is presented as an exposition of the whole universe
of beings. This is a question which I myself am not able to decide, one which is still completely unclear to
me. (Heidegger, The Metaphysical Foundations of Logic, 271)

12. Alfred North Whitehead, Process and Reality: An Essay in Cosmology, corrected ed., ed. David
Ray Griffin and Donald W. Sherburne (New York: Free Press, 1 9 78), 64.

13. Ibid., 22.

1 4 . Christopher Alexander, The Process of Creating Life, vol. 2 of The Nature of Order: An Essay on
the Art of Building and the Nature of the Universe (Berkeley, Calif. : Center for Environmental Struc­
ture, 2002), 45.

1 5 . A note on tactics: measure theory provides additional nuance to that of point set topology,
treating some aspects of size and measurability without resorting to numerical or metric concepts.

1 6 . One of the fundamental controversies of the "human sciences" is the measurement problem:
how can any plausibly rich part of human experience be measured in a way that is adequate to
that phenomenon? The sheer size and technical scope of the three-volume Foundations of Mea­
surement attest to the centrality and magnitude of the problem. See David H. Krantz, R. Duncan
Luce, Patrick Suppes, and Amos Tversky, Foundations of Measurement, 3 vols. (New York: Academic
Press, 1 9 7 1-1 990) .

1 7. Akeel Bilgrami, "accidentalism, the Very Idea: An Essay on Enlightenment and Enchant­
ment, " Critical Inquiry 32 (2006) : 3 9 7 .

1 8 . Primordial does not mean prior t o o r "below" but i s coincident with. It does connote having
less structure, even though it suffuses that to which it is primordial.

1 9 . Whitehead, Process and Reality, 6 1 .

20. Ibid., 3 1 5 .

2 1 . See Whitehead's discussion o f symbolic reference between presentational immediacy (e.g.,

Process and Reality, 1 24-128) and causal efficacy. These two distinctions are informed by general
relativi ty.

22. Ibid., 3 1 5-3 1 6 .

23. Ibid., 3 1 5 .

24. Ibid., 1 9 .
310 N otes to Chapter 5

25. Ibid., 2 1 .

26. S. W. Hawking and G. F. R . Ellis, The Large Scale Structure of Space-Time (Cambridge: Cambridge
University Press, 1 9 73).

27. Alfred North Whitehead, The Principle of Relativity with Applications to Physical Science (Cam­
bridge: Cambridge University Press, 1 922), 34.

28. This [[s] is not identical to the electromagnetic density J[ t] , which is written as a function of
time rather than of the position.

29. Whitehead, The Principle of Relativity, 76-80.

30. As Donald Crosby wrote in his commentary on Whitehead:

The term "actual entity" is used by Whitehead to designate the irreducible units of reality in his metaphysical
pluralism. This term, although in a sense artificial, is one possible translation into English of Descartes' phrase
res vera. Whitehead wants to make central in his own system the idea of that fullness of existence with respect
to which everything else is a derivative or an abstraction (d. the "ontological principle"). And he finds this
idea implicit in Descartes' use of the two terms res vera and " substance" in the Meditations. The term " substance"
carries with it too much of the substance-attribute metaphysics to be useful for the organic philosophy. But
res vera can serve as a substitute and, translated as "actual entity," can grasp the germ of truth in Descartes'
doctrine of substance.

Donald A. Crosby, "Whitehead on the Metaphysical Employment of Language, " Process Studies
1, no. 1 (Spring 1 9 7 1 ) : 38-54.

3 1 . Alfred North Whitehead, The Concept ofNature (The Tanner Lectures Delivered in Trinity College,
November 1 9 1 9) (Mineola, N.Y. : Dover, 2004).

32. The "reals " refer to the real numbers, but this is a technical rather than a metaphysical
statement.

3 3 . Whitehead, Process and Reality, 24.

34. For a flavor of this sort of geometrical (as opposed to analytic or algebraic) mode of differ­
ential geometric reasoning, see, for example, the following canonical accounts: Heinz Hopf,
Differential Geometry in the Large: Seminar Lectures, New York University, 1 946 and Stanford University,
1 956 (Berlin: Springer-Verlag, 1 983); John Willard Milnor, Topology from the Differentiable View­
point (Princeton: Princeton University Press, 1 9 9 7); and the article by William H. Meeks III,
"Geometric Results in Classical Minimal Surface Theory, " in Surveys in Differential Geometry, vol.
8 (Somerville, Mass. : International Press, 2003), 269-306.

35. Remember that "real" in this context simply refers to the elements of the real line R. An
example of a real-valued function is y = sin [x]/x.

36. Consider the open interval of points {x I such that {'[x] = OJ along a section of the curve that
is horizontal.

3 7 . Gottfried Wilhelm Leibniz, Monadology, trans . Nicholas Rescher ( 1 7 14; Pittsburgh: University
of Pittsburgh Press, 1 9 9 1 ) .
N otes to Chapter 5 31 1

38. But this may displace the transcendental appeal to the transfinite axiom of choice. Does the
proof of the theorem that a continuous function on a compact set achieves its maximum and
minimum on that set rely on the transfinite axiom of choice?

39. Leibniz, Monadology, paragraph 5 6 . In "Truth and Substance" ( 1 6 7 7) Leibniz defines a sub­
stance as a unity, indivisible; a " substantial" subject is uniquely complete in its predicates, and
not predicate of any other subject.

40. Broadly speaking, central insights and existential questions introduced by quantum mechan­
ics have not been resolved against the canon of the digital: computable functions on finite,
discrete domains. Partly this is because public intellectual imaginaries have shifted from physics
to informatics and biology, and partly because accounts of contemporary physics generally have
been too technical or too atmospheric to afford traction for nonspecialists to do extensive con­
ceptual work. Just as Badiou distinguishes between a "petit" philosophy of mathematics and a
philosophy that draws insights from mathematics, we can point to philosophical works that
draw insights from quantum mechanics regarding how we know or articulate the world. Pro­
found examples include Isabelle Stengers's Cosmopolitics, which has been recently translated into
English (trans. Robert Bononno; Minneapolis: University of Minnesota Press, 2010-20 1 1), and
Karen Barad's Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and
Meaning (Durham: Duke University Press, 2007). Regarding the far larger literature of the philoso­
phy of quantum theory, see for example: Henry Stapp, Mind, Matter, and Quantum Mechanics, 3rd
ed. (Berlin: Springer, 2009); Michael Epperson, Quantum Mechanics and the Philosophy of Alfred
North Whitehead (New York: Fordham University Press, 2004); and Timothy E. Eastman and Hank
Keeton, eds., Physics and Whitehead: Quantum, Process, and Experience (Albany: State University of
New York Press, 2004) . In 2009, Michael Bitbol, Pierre Kerszberg, and Jean Petitot published the
results of an ambitious project to reconstitute objectivity by historicizing Kant's concept of it, at
the same time using physics : Constituting Objectivity: Transcendental Perspectives on Modem Physics
(Berlin: Springer, 2009). More work can be done on resolving such informed ontological and
epistemological thought with the more reductive digital models of computational media.

4 1 . Weakening the hypotheses strengthens the account by broadening its scope.

42. More precisely: The eigenvalues of the Laplacian 1l. (also called the spectrum) are discrete
because 1l. is a bounded, compact, linear operator on a Banach space. The heat trace Z, defined by

is a spectral invariant-a function only of the spectrum. By a certain amount of work beyond
the scope of this book, one can prove that this function can be expanded as a function of t:

Z [t] ( 4 nt )dim(M)/2
= ;
L
;"!
a; t ,

where the coefficients a; are integrals of polynomials in the curvature and its covariant deriva­
tives. The first three coefficients already encode a lot of information from the curvature tensor
Riemann and its contractions, the Ricci tensor and scalar curvature 5:
312 N otes to Chapter 5

ao = volume[M]

al = '!' J S
6 M

and

a2 =
1
360 M
J
__ ( 5S2 - 2 1Riccil2 - 1 0 IRiemannI 2 ) :

very lovely results.

43. Ernst Mach, The Science of Mechanics, Supplement to the Third English Edition, trans . Philip E.
B . Jourdain (Chicago: Open Court, 1 9 1 5), 32-34.

44. Riemannian differentiable geometry greatly extends this notion of a tangent to a curve. An
n-dimensional manifold M that is smooth in the differential geometric sense has a unique
n-dimensional tangent space, conventionally denoted TpM, at every point p in M. This tangent
space is a linear vector space corresponding to Deleuze and Guattari's striated space. Each mani­
fold (or multiplicity in Deleuze and Guattari's vocabulary) has a tangent space for each p, so
M is associated densely with a striation, as densely as its pOints. We will elaborate on this in
chapter 6.

4 5 . In that case, the action is the integral of the norm of the Ricci curvature on space-time (the
universe), and its Euler-Lagrange equations yield the Einstein field equations. The space-time
metric appears exactly as a critical point. This particularly elegant derivation appears in Theodore
Frankel, Gravitational Curvature: An Introduction to Einstein 's Theory (San Francisco: W. H. Freeman,
1 9 79).

46. Alexander, The Nature of Order, especially chapter 2.

4 7 . Ibid., 77.

48. Whitehead, The Principle of Relativity, 34.

49. A mathematical comment: A basis of an n-dimensional vector space is a set of n vectors VI

I
V2, • • • , Vn such that any vector in the space is a linear combination of the Vi with some unique
set of coefficients. This formalizes the notion of "building blocks" for a linear theory; but it
requires the notion of a linear space, that every element can be written as a linear combination
of basic elements.
A mathematical and tactical comment: The notion of a linear, first-order theory approximates
the general situation. But this is not valid in general! Not even in the case of functions; for
example, take a measurable function, or a piecewise linear function. Or take algebras; not all
algebras have differentiable structure. (Lie groups do.)

50. (1) This is a typical move in mathematical proof. Use a hypothesis that allows you to use
strong intuitions about familiar things, say continuous functions. Then review the proof in detail
to see whether you can relax the hypothesis to slightly weaker conditions, say Lipschitz continu­
ous functions. Weaker hypothesis <=} stronger theorem. (2) However, we need to be extremely
N otes to Chapter 5 313

cautious about how we rely on intuitions form highly developed parts of mathematics because
the paths we should make that are adequate to the phenomena or the aspiration may not be the
paths built by the existing mathematics. As an exercise, try to see how this can trellis the forma­
tion of an episteme.

5 1 . Alexander, The Nature of Order, 22-23.

52. This point of agreement I share also with Jim Bono in his essay, November 2008 on White­
head's atomicity.

5 3 . See Andrei Linde's chaotic inflationary version of Alan Guth's inflationary cosmology: A. D.
Linde, Inflation and Quantum Cosmology (Boston: Academic Press, 1 990) .
Technical note: on the other hand, often we can weaken a smooth (infinitely differentiable
theory to nonsmooth category, in PDE, to CO to piecewise linear, then Lipschitz continuous
functions, where you compare behavior and bound a function but do not pretend to know
anything about how it behaves pointwise beneath the cover.

54. Alexander, The Nature of Order, 40.

55. Ibid., 4 1 .

56. Rene Thorn, Semio Physics: A Sketch (Redwood City, Calif. : Addison-Wesley, 1 990), 3 .

5 7 . Alexander, The Nature of Order, 45.

58. Michel Serres, Genesis, trans. Genevieve James and James Nielson (Ann Arbor: University of
Michigan Press, 1995), 3 .

5 9 . Ibid., 4.

60. A tactical comment: A formal equivalence does not imply substantial equivalence, or expe­
riential eqUivalence, or material-phenomenological eqUivalence. For some approaches to such a
material phenomenology, see Michel Henry, Material Phenomenology, trans . Scott Davidson (New
York: Fordham University Press, 2008), and for a less anthropocentric account, David Woodruff Smith.

6 1 . Serres, Genesis, 4. More precisely, Paul Cohen was awarded the Fields Medal for demonstrat­
ing, using his "forcing method, " that the continuum hypothesis is logically independent from
standard (Zermelo-Frankel) set theory and the axiom of choice, a theorem from which Badiou
draws an exorbitant amount of material.

62. Ricardo and David Nirenberg summarized their critique of Badiou's project in three steps:
(1) "Badiou's set-theoretical models for ontology are at best a priori commitments rather than
necessary truths of the set theory within which they are made. Here we do not mean his general
a priori commitment to realism but his commitments to specific models and the reduction of
number to set theory. " (2) " [I] n deducing philosophical and political consequences from his set­
theoretical arguments, Badiou confuses contingent attributes of informal models with necessary
consequences of the axioms (we will call this type of confusion a Pythagoric snare) . The politico­
philosophical claims that result have no grounding in the set theory that is deployed to justify
314 N otes t o Chapter 5

them. " (3) " [Ilt could not be otherwise, for the axioms of set theory themselves dictate strict
limitations on the kinds of objects they can and cannot be applied to. Any rigorous attempt to
base an ontology upon them will entail such a drastic loss of life and experience that the result
can never amount to an ontology in any humanly meaningful sense. " Ricardo L. Nirenberg and
David Nirenberg, " Badiou's Number: A Critique of Mathematics as Ontology," Critical Inquiry 3 7
(20 1 1 ) : 585-586. For me, the most salient criticism o f such an ontology i s its inadequacy to life.
At the same time, it is important to acknowledge the Nirenbergs' Pythagorean snare: confusing
" contingent attributes of contingent attributes of informal models with necessary consequences
of the axioms . " The key point to remember is that my project is not to prove or demonstrate
the truth of one metaphysics over another, but to suggest a particular set of orientations and
their affiliate effects. So whether Serres's multitude is "more possible" or "more arguable" than
Badiou's ontology constructed from integer and set is beside the pOint.

63. Serres, Genesis, 1 2-1 3 .

6 4 . Claude E . Shannon, " A Mathematical Theory o f Communication, " Bell System Technical
Journal 27 ( 1 948): 3 7 9-423 and 623-656.

65. For a very extensive construction, see Alain Connes, Noncommutative Geometry (San Diego:
Academic Press, 1 994); and Connes and Marcolli, Noncommutative Geometry, Quantum Fields and
Motives.

66. Kim L. Evans and Kristin Steslow, "A Rest from Reason: Wittgenstein, Drury, and the Differ­
ence between Madness and Religion, " Philosophy 85, no. 2 (201 0) : 245-258 .

6 7 . Such schematic interpretation is encoded i n the present practice o f differential diagnosis. In

Guattari's anti-psychiatric approach, the analyst and the patient come together with their own
sets of interpretations of action and the world, working in two parallel cones of experience.

68. The standard reference of psychiatric disorders: American Psychiatric Association, Diagnostic
and Statistical Manual of Mental Disorders (DSM-IV-TR), 4th ed. (Washington, D . C . : American
Psychiatric Association, 2000) .

69. That is why choreographer William Forsythe's "language" of isometries (William Forsythe,
Improvisation Technologies, DVD [Ostfildern: Hatje Cantz Verlag, 2000]) is not a language so much
as a notational conceit, a way to visually and geometrically (which is not the same mode) trace
a certain aspect of his rich and idiosyncratic movement imaginary. The same is true of Laban
notation-although it makes the space surrounding a dancing body more of a cosmology. And
whereas Laban notation is defined with respect to a moving frame centered on the dancer's
moving body, that body's surround is schematized as a set of octants invariantly fixed to that
moving frame. Forsythe's notation, for all its incompleteness, insists on movement, never just
static form.

70. Madeline Gins and Shiisaku Arakawa, Architectural Body (Tuscaloosa: University of Alabama
Press, 2002) .

7 1 . Michel Serres and Bruno Latour, Conversations on Science, Culture, and Time (1 990; Ann Arbor:
University of Michigan Press, 1995).
Notes to C h a pters 5 and 6 315

Regarding time a s a n effect: Let M b e a complete Riemannian manifold, K a compact set K c

M, and r: [O, ) � M a geodesic such that y[Oj E K. Then y escapes K in finite time, Le., there is
00

a t, 0 < t < such that yftj 11' K.

00

One can define the escape time 7;,scape as the infimum

Tescape '" inf {t : 1 0 < t < oo, y [t] 11' K} .

In this case, time is a quantity that is a result of a process of a "geodesic flow" that depends in
turn on the local as well as global topology and geometry of the Riemannian manifold M.

6 Topology, Manifolds, Dy namical Sy stems, Measure, a n d B u n d les

1. "Whitehead and the Transformations of Metaphysics, " an event hosted in Brussels with Isa­
belle Stengers, James Williams, Mick Halewood, Steven Meyer, and about twenty other partici­
pants. See the Proceedings of the Royal Flemish Academy, 2005.

2. A neo-Bourbakian map of mathematics places to scale some of the subdisciplines: logic, set
theory, graph theory, combinatorics, classical number theory, point set topology, homotopy and
topological manifolds, algebra of rings and fields, algebraic topology, real analysis and theory of
(real) functions, theory of complex functions, differential equations, partial differential equa­
tions, differential topology, functional analysis, differential geometry, complex manifold theory,
low-dimensional topology, index theory, analytic number theory, schemes and categories, sheaf
theory, noncommutative algebra and geometry, etc. Two small stars in the wheeling galaxy
(graph theory and combinatorics for computer science, and point set topology) show how little
of this has been mined by computer science and computational technology.

3 . It is curious how in humanities and social sciences, "theory" indiscriminately lumps together
all philosophical, historiographical, analytical, critical, psychoanalytical, indeed all conceptual
studies. But such a motley set of reflections, representing divergent and even incommensurable
approaches to the diverse objects of literature, art, history, and human experience, seems to create
a set of all sets, in fact an impossible object, a reification error. To a mathematician, the word
"theory" by itself has no meaning; theory is always a theory of something: of Lie groups, of
Riemannian manifolds, of currents and varifolds. There is no impermeable ontological or epis­
temic distinction between the objects of mathematics and their modes of articulation. This
porosity implies a material continuity consonant with Badiou's lemma " mathematics is ontol­
ogy. " See Sha Xin Wei, "Differential Geometric Performance and the Technologies of Writing, "
Ph.D. dissertation, Stanford University, 2000.

4. Any field of mathematics is much more than merely a descriptive scheme. One can say surely
and supraindividually what will follow from the given conditions. This additional expressive
power of a mathematical mode of articulation derives from its structure as proof. But what
mathematicians regard as proof is not what logicians or foundationalists call proof, because
mathematicians rely on the accumulated body of intuition acquired in continuous streams of
face-to-face apprenticeship together with not-necessarily-computational calculation to fill in the
potentially infinite gaps between the steps of a mathematical proof. Mathematical proofs combine
effectively in supraindividual deductive structures.
316 N otes t o Chapter 6

Gbdel's incompleteness theorem does not invalidate this point, because it does not contradict
the correctness of a correct proof, or the collective truth of interdependent theorems relative to
an axiomatic system. Gbdel proved something far more radical than a simple-and naively
untenable-refusal to acknowledge the validity of mathematical proofs. He demonstrated that
in any mathematical theory that contains the logic of arithmetic one can construct a statement
that is both provably true and provably false in that theory!

5. Klaus Janich, Topology (Berlin: Springer-Verlag, 1 9 84) .

6. Brian Cantwell Smith, "The Foundations of Computing," Draft 0,94 (b), 6.

7 . Donald MacKenzie writes about how facts such as the accuracy of the kill radius of nuclear
missiles that appear to be solely numerical functions of the constants of physics and of physical
contingencies are in fact also highly inflected functions of political agendas. Donald MacKenzie,
"Nuclear Missile Testing and the Social Construction of Accuracy, " in Mario Biagioli, ed., The
Science Studies Reader (New York: Routledge, 1 999), 342-3 5 7 .

8 . A much more powerful way t o understand such trajectories i s t o regard them a s orbits of
pOints under the action of a Lie group acting on the given space M. Or even more fleXibly, under
the action of a homeomorphism mapping M to M.

9. Bruno Latour, Science in Action: How to Follow Scientists and Engineers through Society (Cambridge,
Mass. : Harvard University Press, 1987).

10. Hayden prefaces her review of the technics and politiCS of Similarity with an anti-Similares
slogan: " LTe curaste 0 te sientes similar? ("Are you better, or do you feel similar?") Cori Hayden,
"A Generic Solution? Pharmaceuticals and the Politics of the Similar in Mexico," Current Anthro­
pology 48, no. 4 (200 7): 481-482.

1 1 . Especially in this chapter I qualify certain concepts or arguments as "rigorous, " meaning that
they admit definitions that are sufficiently precise and arguments sufficiently verifiable to be
accepted by mathematicians . Such concepts and arguments enjoy a particular mode of portabil­
ity, shareability, and reusability similar to that shared by perspectivally approached aperspectival
entities (objects and processes) of mathematics. But I use this not as a carapace but as a trellis
for articulation, ultimately for poietic articulation.

12. Thanks to Helga Wild, private communication, May 4, 20 1 1 .

1 3 . Christopher Alexander, The Phenomenon of Life: A n Essay on the Art of Building and the Nature
of the Universe (Berkeley, Calif. : Center for Environmental Structure, 2002), 143-242.

14. For vector spaces X and Y, over the scalar field R, a map f. X -7 Y is linear if f(u + v) = f(u) +
f(v), and f(k * u) = k * f(u), for any u, v in X, and any scalar k in R.

15. George Lakoff, public lecture, Berkeley, 2000.

1 6 . Gottfried Wilhelm Leibniz, The Shorter Leibniz Texts: A Collection of New Translations, trans.
Lloyd H . Strickland (London: Continuum, 2006), 1 3 7 .
N otes to Chapter 6 31 7

1 7 . Humberto R. Maturana and Francisco J. Varela, The Tree of Knowledge: The Biological Roots of
Human Understanding (Boston: New Science Library, 1987), 2 7 .

1 8 . David H. Krantz, R. Duncan Luce, Patrick Suppes, and Amos Tversky, Foundations ofMeasure­
ment, 3 vols. (New York: Academic Press, 1 9 7 1-1 990) .

1 9 . In 2010, a European Union Framework-supported proj ect called A Topological Approach to

Cultural Dynamics (ATACD) closed its three-year course with a conference in Barcelona covering
a very large range of responses to the challenge of understanding cultural dynamics, with tech­
niques ranging from quantitative modeling, computational physics, and design to literary and
historical methods. The diverse and energetic response demonstrated a wide recognition of the
need for fresh approaches to the measurement problem, between absolute mutual rejection or
absorption of one side by the other, which in the present age largely means absorption of histori­
cal and cultural by quantitative and computational models.

20. Lest you think that this is merely a mathematician's fancy, physicists have found reason to
fashion odder symmetries. For example, if you rotate 360 degrees a particle with half-spin, what
you see will be the "negative" of the original particle; one needs to spin it around twice to get
back to the original particle.

2 1 . Throughout this chapter, to ease the first turn of the spiral, I describe what mathematicians
might call a classical approach to geometry and analysis. The second turn would take us through
measure theory en route to a mode of articulation informed by contingency and situation (e.g.,
"observer effects"), which would be in the scope of a future work.

22. See William Rowan Hamilton and Charles Jasper Joly, Elements of Quaternions, 3rd ed. (New
York: Chelsea, 1969). More explicitly, the set of quaternions is defined as a four-dimensional
vector space with real coefficients, with the usual vector addition but in which the basis elements
I, i, j, k can be multiplied according to the following relations: i*j = k = -j*i, j*k = i = -k*j, k*i =
j = -i*k, and 1 is the multiplicative identity. Associativity intertwines the addition and multiplica­
tion operations.

23. Another kind of invariant is a spectrum of a linear operator. Consider in particular the
Laplace-Beltrami differential operator on functions on a manifold. You can think of its spectrum

O O
I '
j * k= i k * i =j j * i = _k k * j = -i

k j k j

i*j=k i * k = -j

Figure 6.7
Quaternions, William Rowan Hamilton's noncommutative four-dimensional algebra. Diagram
by the author.
318 N otes t o C h a pter 6

(the set of its eigenvalues) as giving the timbre of the manifold, regarded as a kind of reverberat­
ing object. In the more familiar situation of a two- or three-dimensional compact manifold, this
suggests the notion of how an object's sound relates to its shape, a question famously captured
in the essay by Marc Kac, "Can One Hear the Shape of a Drum?," American Mathematical Monthly
73, no. 4, part 2 ( 1 9 66): 1-23. This was called, more neutrally, the isospectral problem, relating to
earlier work by Hermann Weyl.

24. For example, the theorem that an arbitrary infinite product of compact spaces is compact
seems contrary to intuition because we intuitively perceive sets in a visualist way: based on lhe
surveyability of a set of indices. In fact, by definition, an element x of an infinite product n Xi
j""l
is described exactly by the criterion that only a finite number of its coordinates are nonzero. We
imagine, for example, that an element x has to have nonzero coordinates in a consecutive series
of dimensions, not say: Xk = { . . . , 0, 0, 0, . . . , 1 (in kth dimension), 0, 0, 0, . . . j .

25 . Humberto R . Maturana and Francisco J. Varela, Autopoiesis and Cognition: The Realization of
the Living (Dordrecht: D. Reidel, 1980), 1 3 3 .

2 6 . Alfred North Whitehead, Process and Reality: An Essay in Cosmology (New York: Free Press,
1 9 78), 52.

27. Ibid., 43-44.

28. Ibid., 23 .

29. Ibid., 164.

30. Ibid., 3 1 8 .

3 1 . The same can b e conjectured for events a s well.

32. This last arguably could better be translated as perforated space, for which geometric measure
theory would be a better fit. But this is beyond the scope of this book.

3 3 . Gilles Deleuze and Felix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia (Min­
neapolis: University of Minnesota Press, 1987), 484.

34. Ibid., 488.

35. Ibid., 483.

36. Ibid., 478.

3 7. Ibid., 479.

38. Thanks to Patrick Harrop, architectural researcher at the Topological Media Lab, for this
observation.

39. Rene Thorn, Semio Physics: A Sketch (Redwood City, Calif.: Addison-Wesley, 1 990), 3 .

40. Ibid., 6.

4 1 . Deleuze and Guattari, A Thousand Plateaus, 483.

N otes to Chapter 6 319

42. Ibid., 484.

43. Albert Lautman, Essai sur les notions de structure et d'existence en mathematiques (Paris: Univer­
site de Paris, 1 9 3 7), 23, 34-3 5 .

4 4 . Gilles Deleuze and Felix Guattari, Capitalisme e t schizophrenie, Mille plateaux (Paris: Editions
de Minuit, 1 9 80), 606.

4 5 . I say anexactly rigorous because the delicate but key work here is a four-step philosophical­
philological interpretation: ( 1) the mathematical concept as named in French by Deleuze and
Guattari's mathematical informants, (2) Deleuze and Guattari's reuse which is sometimes an
artful philosophical extension but sometimes a confusion, (3) Massumi's translation into English
without the mathematical sense or context, (4) the corresponding terms and concepts used by
mathematicians in English. Unlike some other philological exercises, we have the twin advan­
tages of high coherence between terms and concepts scaffolded by contemporary mathematical
logical notation and rhetoric, and high coherence between French and English mathematical
theories due to the cosmopolitan nature of twentieth-century mathematics.

46. Deleuze and Guattari, A Thousand Plateaus, 485 .

4 7 . Ibid., 486.

48. Ibid.

49. Jean Petitot, "Morphological Eidetics for Phenomenology of Perception, " in Naturalizing
Phenomenology: Issues in Contemporary Phenomenology and Cognitive Science, ed. Jean Petitot, Fran­
cisco J. Varela, Bernard Pachoud, and Jean-Michel Roy (Stanford: Stanford University Press, 1 999),
330-3 7 1 .

5 0 . Ibid., 344.

5 1 . Petitot refers in fact to a microlocal analytic notion of the germ of a function defined on
the fibers, but I believe this notion does not require continuity.

52. Petitot, "Morphological Eidetics, " 330-33 1 .

5 3 . I conceive the multitude a s the complement i n a dense open set to a particular discrete set
of points, accommodating Serres's multitude in his book Genesis ( 1 9 9 5 ) . Serres's multitude is a
richer notion than Negri and Hardt's multitude as a union of individuals (molecules), as well as
Badiou's bare set theory. It is no wonder that Badiou requires a Herculean adaptation of Paul J.
Cohen's method of forcing from modern set theory to name the "indiscernable" multiple,
because he starts from bare set theory, from the void and the set of the void. This is the same
minimalist starting point as that of Bertrand Russell and the early Whitehead in their Principia
Mathematica.
By characterizing multitude as the clamor of the ocean, as the archaic French word noise taken
in an immeasurably deeper sense than simple randomness or chaos or noise as it is commonly
understood in English, Serres b egins his meditation with a maximal clamor of being close to
Deleuze and Guattari and far from Badiou. I share that proximity because bare set theory does
320 N otes to Chapters 6 and 7

not admit life. But, starting with ingredients from nonatomistic topology and measure theory,
we obtain a lightest articulation of life, with no claim to adequacy. We address this in chapter 5 .
This suggests a reinterpretation o f some fundamental projects of Deleuze i n t h e Logic of
Sense, Difference and Repetition, and Le plio But these considerations extend beyond the scope of
this book.

7 Practices: Apparatus and Atelier

1 . For a survey of related classes of responsive environments and interactive spaces, see Lucy
Bullivant, Responsive Environments: Architecture, A rt and Design (London: Victoria and Albert
Museum, 2006) .

2. http://cycling7 4.com.

3. Preceding instruments were written in SuperCollider (http://www.audiosynth.com) and NATO

(http://www.m9ndfukc.org/korporat/nato.0+55+3d.html) .

4. See Sha Xin Wei, Timothy Sutton, Jean-Sebastien Rousseau, Harry C. Smoak, Michael Fortin,
and Morgan Sutherland, " Ozone: A Continuous State-Based Approach to Media Choreography,"
Topological Media Lab, Concordia University, 2009; Sha Xin Wei, Michael Fortin, and Jean­
Sebastien Rousseau, " Calligraphic Video: A Phenomenological Approach to Dense Visual Interac­
tion, " ACM Multimedia, Beijing, 2009.

5 . CrossBow Technologies (http://www.xbow.com/) .

6. We chose OSC because it can inherit base data transport bandwidth. OSC represents data more
generically than MIDI, the older protocol for intercommunicating sensor-based performance
devices and electronic musical instruments. We map sensor data to floating point instead of
MIDI's 8-bit range. For data transport we use Ethernet rather than other protocols such as Blue­
tooth because this permits us to treat the computational hosts uniformly, whether on body or
fixed, and makes it much easier for us to upload, diagnose, or invoke applications from fixed
computers, with lesser problems of range and robustness. See http: //cnmat.berkeley. edu/OSC/,
Adrian Freed.

7. See Mark Goldstein, " Playing Electronic Instruments: Technique Meets Technology, " People,
Computers and Design Seminar, Stanford University, 1 9 9 7; Joel Ryan and Chris Salter, "TGarden :
Wearable Instruments and Augmented Physicality, " paper presented a t the New Interfaces for
Musical Expression (NIME) conference, 2003.

8. Francine Gemperle, Chris Kasabach, John Stivoric, Malcolm Bauer, and Richard Martin,
"Design for Wearability, " Digest of Papers, Second International Symposium on Wearable Computers
(Los Alamitos, Calif. : IEEE Computer Society, 1 998), 1 1 6-122.

9 . ADXL202e accelerometers, surface-mounted, by Analog Devices (http://www.analog.com/).

10. In 2006-2008, Tim Sutton designed and built the audio analysis and synthesis parts of the
Ozone sound-processing system, replacing the original SuperCollider software built in 2001 by
N otes to Chapter 7 321

Joel Ryan. The description of the audio framework derives from Sutton's part of the Ozone docu­
mentation: Sha et al., "Ozone: A Continuous State-Based Approach to Media Choreography. "

1 1 . Search Remedios Terrarium on the Topological Media Lab website, http://www

. topologicalmedialab.net, 2008.

12. For IRCAM, see "CataRT: Real-Time Corpus-Based Concatenative Synthesis," http://imtr.ircam
.fr/imtr/CataRT (retrieved July 7, 2012) . For CataRT sound corpus-based analysis and synthesis,
see Diemo Schwarz, Gregory Beller, Bruno Verbrugghe, and Sam Britton, " Real-Time Corpus-Based
Concatenative Synthesis with Catart," DAFx (2006); Diemo Schwarz, Roland Cahen, and Sam
Britton, "Principles and Applications of Interactive Corpus-Based Concatenative Synthesis,"
Journees d'Informatique Musicale (2008) . For OMAX machine learning, see http://omax.ircamJr/
(retrieved July 7, 201 2); Shlomo Dubnov, Gerard Assayag, Olivier Lartillot, and Gill Bejerano,
"Using Machine-Learning Methods for Musical Style Modeling, " IEEE Computer Society Press 36,
no. 1 0 (2003) : 73-80; Benj amin Levy, "Visualising Omax," IRCAM, 2009 . For gesture following,
see http://imtr.ircam.fr/imtr/Gesture_Follower (retrieved July 7, 201 2); Frederic Bevilacqua, Bruno
Zamborlin, Anthony Sypniewski, Norbert Schnell, Fabrice Guedy, and Nicolas Rasamimanana,
" Continuous Realtime Gesture Following and Recognition, " in Gesture in Embodied Communica­
tion and Human-Computer Interaction, ed. Stefan Kopp and Ipke Wachsmuth (Berlin: Springer,
2010), 73-84.

1 3 . Search Meteor Shower and the Cosmicomics on the Topological Media Lab website, http://
www.topologicalmedialab. net (retrieved February 5, 2010).

14. Sha Xin Wei, Michael Fortin, Navid Navab, and Timothy Sutton, " Ozone: A Continuous
State-Based Approach to Media Choreography," ACM Multimedia, Florence 2010.

15. Eugene T. Gendlin, Experiencing and the Creation of Meaning: A Philosophical and Psychological
Approach to the Subjective (Evanston: Northwestern University Press, 1 9 9 7), v.

1 6 . Rene Thorn, Mathematical Models ofMorphogenesis, Chichester, UK: Ellis Horwood, 1983; Rene
Thorn, Structural Stability and Morphogenesis: An Outline of a General Theory of Models (Reading,
Mass. : Addison-Wesley, 1 9 89); Jean Petitot, "Morphological Eidetics for Phenomenology of Per­
ception," in Jean Petitot, Francisco J. Varela, Bernard Pachoud, and Jean-Michel Roy, eds., Natu­
ralizing Phenomenology: Issues in Contemporary Phenomenology and Cognitive Science (Stanford:
Stanford University Press, 1 999), 3 3 0-3 7 1 .
\
1 7. �ha, Fortin, and Rousseau, " Calligraphic Video: A Phenomenological Approach. to Dense
Visual Interaction. "

1 8 . I n doing so, they determine a convention for what ought to occur i f a player state reaches
one of the boundaries of the simplex which that player is occupying. The player state may transi­
tion to a neighboring simplex which shares the boundary that it has reached, if one or more
such neighbor simplices exist; otherwise it will bounce off of the boundary, back toward the
interior of the simplex. If a transition from an n-simplex to one of smaller dimension is to be
allowed, a subset of the boundary of the n-simplex which contains the (n - I )-simplex must be
identified through which that transition should occur.
322 N otes to Chapter 7

Transition to the smaller-dimension simplex will occur when the player state crosses any point
on this boundary subset. For example, in our player state topology in Figure Person State Topol­
ogy, the transitions from the 3-simplex to the two 2-simplices occur at the Feed and Reveal
vertices.

19. Yon Visell implemented the state engine in C. The detailed equations of the energy-based
dynamics derive from Visell's section of: Sha Xin Wei, Yon Visell, and Blair MacIntyre. "Media:
Choreography Using Dynamics on Simplicial Complexes, " GVU Technical Report, Georgia Insti­
tute of Technology, 2003 .

20. http://www.topologicalmedialab.net/xinwei/sponge.org.

2 1 . Regarding the specious present, see William James, The Principles o( Psychology (Cambridge,
Mass. : Harvard University Press, 1 983), 5 73; see discussion in Steven Meyer, Irresistible Dictation:
Gertrude Stein and the Correlations o(Writing and Science (Stanford: Stanford University Press, 200 1 ) .

2 2 . S e e work b y Aaron Bobick e t a l . a t the MIT Media Lab, c a . 2000; Jehan Moghazy, "Evaluating
the Playfulness of Responsive Play Spaces, " GVU Technical Report, Georgia Institute of Technol­
ogy, 2004; and Barbara E. Hendricks, Designing (or Play, Design and the Built Environment Series
(Aldershot, UK: Ashgate Publishing, 200 1 ) .

23. Peter Brook, The Empty Space (New York: Atheneum, 1968).

24. Stanley Milgram, Obedience to Authority: An Experimental View (New York: Harper and Row,
1 9 74) .

25 . See Gendlin on felt meaning, in Experiencing and the Creation o( Meaning.

26. Jerzy Grotowski and Eugenio Barba, Towards a Poor Theatre (New York: Routledge, 2002) .

2 7 . See, for example, http://pointclouds .org (last accessed July 19, 201 1 ) .

2 8 . Sha Xin Wei, Yoichiro Serita, Jill Fantauzza, Steven Dow, Giovanni Iachello, Vincent Fiano,
Joey Berzowska, Yvonne Caravia, Delphine Nain, Wolfgang Reitberger, and Julien Fistre, " Dem_
onstrations of Expressive Softwear and Ambient Media, " Adjunct Proceedings of Ubicomp 2003,
Seattle, 1 3 1- 1 3 6; Sha Xin Wei, Giovanni Iachello, Steven Dow, Yoichiro Serita, Tazama St. Julien,
and Julien Fistre, " Continuous Sensing of Gesture for Control of Audio-Visual Media, " Proceed­
ings of the Seventh International Symposium on Wearable Computers, 2003.

29. Doug Van Nort, " Instrumental Listening: Sonic Gesture as Design Principle," Organised Sound
1 4, no. 2 (2009) : 1 7 7- 187.

30. Isabelle Stengers h as retold the stories o f seven scientific disciplines in a way that presents
the partial and provisional messiness of science as it is actually practiced. Telling science in this
way has both cosmological and political implications, hence the title of her book Cosmopolitiques
(trans . Robert Bononno as Cosmopolitics, 2 vols . [Minneapolis : University of Minnesota Press,
20 1 0-201 1]).

3 1 . The Topological Media Lab (TML) is an academic research center for gestural, performative,
and embodied expression in responsive media environments. It was founded in 2001 by the
N otes to Chapter 7 323

author at the Graphics, Visualization, and Usability Center at Georgia Institute of Technology in
Atlanta, and moved to Concordia University and the Hexagram research-creation network in
Montreal in 2005.
The TML's theoretical project concerns ontogenesis in performative events-how subjects and
objects take shape in a continuous dynamical ontology. The critical inquiry starts from the limits
of discrete representation, and seeks alternatives to linguistic-semiotic analysis in the form of
nonmetric topological, dynamical, potential-theoretic, and other material patterning. An impor­
tant aspiration is to discover nonanthropocentric ways to articulate improvisatory ethico­
aesthetic gesture. Improvisation does not mean something willful or totally unpredictable. It is
conditioned by past aspiration and heuristics, but not by a deterministic plan.

32. The TML's two major application domains are movement-based events and architectural
installations . Its installation-events have been exhibited in technology art venues such as SIG­
GRAPH, Ars Electronica, DEAF, Media Terra, and Elektra. Its productions include the continuous
state-based media choreography apparatus (reinstantiated since 2000 as Oz, Oxygen, and Ozone);
the TCostume Cambodian-style dress fitted with custom accelerometers linked with embroidered
conductive thread (2002-2003); a series of video membranes (2004-201 0); the Ouija experiments
on improvisatory movement and collective intentionality (2007); the WYSIWYG wall-scale
weaving with custom capacitive sensing mapping proximity and movement to a series of sound
instruments; and the Memory, Place, Identity phenomenology experiments (with philosopher
David Morris).

33. For a thorough consideration of the limits of the cognitivist thesis, see Vincent Descombes,
The Mind's Provisions: A Critique of Cognitivism, trans . Stephen Adam Schwartz (Princeton: Prince­
ton University Press, 200 1 ) .

34. See A. N. Whitehead, Process a n d Reality: An Essay i n Cosmology, e d . David Ray Griffin and
Donald W. Sherburne (New York: Free Press, 1 9 75) .

3 5 . Some projects include the series of TGarden responsive environments that was the ur-event
for the atelier-lab (SIGGRAPH New Orleans, Medi@Terra Athens, Ars Electronica, and DEAF,
1999-2001 ) , Hubbub speech-sensitive projected typography installation (Brussels, Atlanta, San
Francisco, 2001-2003), Cosmicomics responsive video installation (Elektra Festival Montreal,
2007), IL Y A 1 2-channel audio + video membrane installation (San Francisco, 2010), the Frank­
enstein's Ghosts feature-length performance with analog plus computational media instrumen­
talists (funded as a research-creation project by the Social Sciences and HumanitIes Research
Council [SSHRC]), and the Einstein's Dream time-conditioning installation.

36. For example, Tom Lamarre and colleagues have documented the substantial popular "fan­
based" creation and circulation of images and narratives surrounding Japanese animation.

3 7 . www.fqrsc.gouv.qc.ca. 2000 (accessed 2009) .

3 S . Social Sciences and Humanities Research Council (SSHRC), " Fine Arts Formative Evalua­
tion Formative Evaluation of SSHRC's Research/Creation in Fine Arts Program, Final Report, "
200S, p. 6 .
324 Notes to Chapter 7

39. Cf., respectively, SenseLab, Concordia University; Radical Empiricism group, Universite de
Montreal; Centre for Intelligent Machines and Centre for Interdisciplinary Research in Music
Media and Technology, McGill University; Interstices, Universite du Quebec it Montreal.

40. For example, an executive from an Internet communications company, a professor of English
literature leaving a tenure track job, a postdoc from the Harvard Graduate School of Design, a
professor of architecture with over twenty years of experience as a licensed architect, a director
of research at a music and digital instruments center.

4 1 . To date, it seems easier for the lab to cultivate some of these anthropologists' theoretical
interests in seminars than for them to absorb the skills to participate in the collective experi­
mental practi ces of the atelier. To a large extent this is not for lack of good will-quite the
contrary. It is more because their programs of study are funded and timed to enforce the timely
writing of a dissertation, and the host institution has no program and no long-term fund for
apprenticing such advanced students wishing to cross laterally from a humanities discipline to
art practice and research at the TML.

42. Stephen Wolfram, A New Kind of Science (Champaign, Ill . : Wolfram Media, 2002) .

43. Of course, what is adequate means quite different things in science and literary theory.

44. Streams of inquiry include movement and gesture (epitomized by the Ouija experiment, Sha
and Montanaro, 2007); nonanthropocentric phenomenology and the built environment, with
two subareas: psychology and architecture (H. Wild, L. Tillett), and temporal textures (H. Smoak,
P. Harrop); technologies of performance with subareas: real-time video, real-time sound, sensing/
sensor feature extraction, pattern tracking, and softwear: wearable, active textiles and j ewelry,
and continuous state-based media choreography; memory, place, identity (D. Morris, Sha) and
the technologies of memory; nonanthropocentric ecology and economics, which comprises a
seminar on Spinoza, Bateson, and Guattari; and the Plant Life Support System project for semi­
automating the watering of plants and sensing of plant health. Seminars include Topological
Media 2005-2007, SenseLab 2005-2006 (Deleuze), Soft Architecture 2007-2009 (Alexander, Gins,
and Arakawa), Simondon 2008, Memory + Place + Identity (Merleau-Ponty, Casey) 2009-20 1 0,
Eco-economics and Vegetal Experience 201 0-20 1 1 (Spinoza, Bateson, Guattari), and Maths Group
2010-20 1 1 (point set topology, differential geometry, measure theory, geometric measure theory) .
The TML has brought visitors to calibrate and stimulate discussion, such as Benoit Maubrey (Die
Audio Gruppe / The Audio Ballerinas), Toni Dove (Spectropia and interactive cinema), and Niklas
Damiris (Art Creation; Money and Quantum Mechanics, The Limits of Sustainability) .

45. Workshop themes have included calligraphic video (2007), media choreography (2007),
computationally activated lighting, pneumatics and kinetic sculpture, guided installation-events,
and autopoietic systems.

46. Professional mathematicians customarily credit peers for even one expression, one turn of
logic, one significant line in another article.

4 7 . An affiliate may use any media or code from another affiliate on a TML-identified work. She
or he is asked to keep the series of names of prior TML contributors, and append her own name
N otes to Chapters 7 and 8 325

to the chain of credits. Work may not be exhibited or cited outside TML until it has been pub­
lished crediting the authors and TML in a peer-reviewed journal, or used in a public, juried event
of sufficiently international stature. TML technology-media, techniques, gear, space, resources­
may not be used for any non-TML project, except by permission from the TML creators or
authors. The object is to build reputation capital to help everyone who affiliates with the Topo­
logical Media Lab over the years. This process acts against the "thermodynamic equilibrium
dilemma" in which too rapid diffusion of knowledge and skilled people kills the incentive for
building up rich relationships and knowledge bases that can yield significant distinctions, and
instead diffuses energy and innovation to a uniform level of mediocrity.

48. After the experience of producing the TGarden responsive environment in 200 1 , I
planted the TML in the institution of the research university because the work needed to
leverage institutional ecology-administrative support, space, access to innovation research
funding, the opportunity to mitigate disciplinary education with talented students. However,
accessing the research and programmatic infrastructure of the academy incurs institutional
constraints.

49. Georgina Born and Andrew Barry, "Art Science, " Journal of Cultural Economy 3, no. 1 (2010):
103- 1 1 9 .

s o . Hexagram (http://hexagram.concordia.ca) is a center based principally i n Concordia Univer­

sity and Universite du Quebec a Montreal for research based on art and technology. It was
established in 2002 with grants from the Canadian Foundation for Innovation and the Quebec
provincial government to provide state-of-the-art equipment, labs, and technology infrastructure
for faculty and graduate research.

5 1 . Hexagram, "Hexagram Funding Competition Evaluation Criteria, " Hexagram-Concordia

Centre for Research-Creation in Media Arts and Technologies, Montreal, 2007, p. 1 .

52. Ibid., 1-2.

53. Hexagram, "Hexagram Fund Guidelines, " Hexagram Fund for University Research and Cre­
ation in Media Arts and Technologies, Montreal, 2005, 2-3 .

54. See David C. Mowery, Richard Nelson, Bhaven Sampat, and Arvids Ziedonis, Ivory Tower and
Industrial Innovation: University-Industry Technology Transfer before and after the Bayh-Dole Act (Stan­
ford: Stanford Business Books, 2004) .

5 5 . Madeline Gins and Shiisaku Arakawa, Architectural Body (Tuscaloosa: University cif Alabama
Press, 2002) .

8 Refrai n

1. See Tom Standage, The Mechanical Turk (London: Allen Lane, 2002); Gaby Wood, Edison 's Eve
(New York: Knopf, 2002) .

2. Epitomized, for example, by "cyborg" chic in the 1 9905 era "Borg Lab" of Steve Mann et al.
at the MIT Media Lab.
326 N otes to Chapter 8 and Afterword

3. In fact, it is in this sense that I interpret Deleuze and Guattari's body without organs. See note 2.

4. Gilles Deleuze and Felix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia (Minne­
apolis: University of Minnesota Press, 1 9 8 7), 500.

5. In some ways, substrate is a suggestive concept for what Deleuze and Guattari described by
asignifying BWO-bodies without organs . (I am indebted to Arkady Plotnitsky for clarifying this
relation.) An emergence can be seen either as a change in intensity-to use Deleuze's concept of
change, differentiation versus differenciation-or as a concrescence, to use Whitehead's way of
describing process.

6. See Sha Xin Wei, "The TGarden Performance Research Project, " Modem Drama 48 (2005):
585-608.

7. See the discussion of gesture as an open relation in Sha Xin Wei, "Resistance Is Fertile: Gesture
and Agency in the Field of Responsive Media, " Configurations 10, no. 3 (2002) : 439-4 72.

8 . I thank Helga Wild for the formulation of presentation versus representation.

9 . Naturalizing matter as dumb substance parallels what Bruno Latour identified as sociologists'
tendency to naturalize scientific objects, in his 1 993 book We Have Never Been Modern. In the
subsequent decade, science studies largely responded to Latour's call for the symmetrical disposi­
tion toward social objects and natural objects, but this symmetrization is still slowly percolating
into neighboring domains in cultural and literary studies and philosophy.
In a sense, the discussion of gesture recalls the discussion of the nature of light and vision
prior to relativity theory. Prior arguments about the existence or nonexistence of ether as a
medium which conducts light were subsumed by one of Einstein's deepest insights, the equiva­
lence of geometry (in the sense of geometrodynamics) with the distribution of matter-energy. In
geometrodynamics, the material medium is also the geometry of space, so that a Signal, being
the rarefaction and compression of physical matter, is simultaneously a time-varying informatic
fluctuation as well as a material fluctuation.

10. Dylan Thomas, Selected Poems, 1 934-1 952 (New York: New Directions Books, 2003) .

Afterword b y Arkady Plotnitsky

1 . Bruno Latour, We Have Never Been Modem, trans. Catherine Porter (Cambridge, Mass. : Harvard
University Press, 1 993).

2. Alfred North Whitehead, Process and Reality, corrected ed., ed. David Ray Griffin and Donald
W. Sherburne (New York: Free Press, 1 9 78).

3 . E.g., Immanuel Kant, The Critique ofPure Reason, trans. Paul Guyer and Allen W. Wood (Cam­
bridge: Cambridge University Press, 1 999), 1 75 .

4 . Georg Bernhard Riemann, "On the Hypotheses That Lie a t the Foundations o f Geometry"
( 1 854), in Beyond Geometry: Classic Papers from Riemann to Einstein, ed. P. Pesic (Mineola, N.Y. :
Dover, 2007), 23-40.
N otes to Afterword 327

5 . Henri Bergson, Matter and Memory, trans. Nancy M. Paul and W. Scott Palmer (Seattle: Amazon/
CreateSpace, 20 10).

6. Jacques Derrida, Speech and Phenomena: And Other Essays on Husserl's Theory of Science, trans.
David B. Allison and Newton Carver (Evanston: Northwestern University Press, 1 9 73).

7 . Jacques Derrida, Margins of Philosophy, trans. Alan Bass (Chicago: University of Chicago Press,
1985), 29-68.

8 . Riemann, "On the Hypotheses That Lie at the Foundations of Geometry. "

9. Gilles Deleuze and Felix Guattari, A Thousand Plateaus, trans. Brian Massumi (Minneapolis:
University of Minnesota Press, 1 9 8 7), 485 .

10. Kant, The Critique of Pure Reason, 1 75 .

1 1 . Hermann Weyl, Space-Time-Matter (Mineola, N.Y. : Dover, 1952).

12. Deleuze and Guattari, A Thousand Plateaus, 482-483.

13. Gilles Deleuze and Felix Guattari, What Is Philosophy?, trans. Hugh Tomlinson and Graham
Burchell (New York: Columbia University Press, 1996).

14. Gilles Deleuze, " Immanence: A Life," in Deleuze, Pure Immanence: Essays on Life, trans. Anne
Boyman (Cambridge, Mass.: Zone Books, 2001 ) .

1 5 . Deleuze and Guattari, Wha t Is Philosophy?, 1 1 8 .

1 6 . Ibid., 202.

1 7. Deleuze and Guattari, A Thousand Plateaus, 365 .

1 8 . Gilles Deleuze and Felix Guattari, Kafka: Towards a Minor Literature (Minneapolis: University
of Minnesota Press, 1986).

19. Deleuze and Guattari, What Is Philosophy?, 208 .

20. Ibid., 2 1 1 .

2 1 . Ibid., 2 1 6 .

2 2 . Jacques Derrida, O f Grammatology, corrected e d . , trans. Gayatri Chakravorty S Pivak (Balti­

more: Johns Hopkins University Press, 1 998) .
Bibliography

Abramowicz, Marek A. " Black Holes and the Centrifugal Force Paradox . " Scientific American,
March 1993, 74-8 1 .

Academic Software Development. MMDD o n Next and Macintosh. Video. Spring 1 994.

Addington, Michelle, and Daniel L. Schodek. Smart Materials and Technologies for the Architecture
and Design Professions. Amsterdam: Elsevier, 2004.

Alexander, Christopher. The Nature of Order: An Essay on the Art of Building and the Nature of the
Universe. 4 vols. : The Phenomenon of Life; The Process of Creating Life; A Vision of a Living World;
The Luminous Ground. Berkeley, Calif. : Center for Environmental Structure, 2002.

Alexander, Christopher. The Timeless Way of Building. New York: Oxford University Press, 1 9 79 .

Alexander, Christopher, Sara Ishikawa, and Murray Silverstein. A Pattern Language: Towns, Build­
ings, Construction. New York: Oxford University Press, 1 9 7 7 .

Allen, ]. "Time and Time Again: The Many Ways t o Represent Time. " International Journal of
Intelligent Systems 6 ( 1 9 9 1 ) : 341-3 5 5 .

American Psychiatric Association. Diagnostic and Statistical Manual o f Mental Disorders (DSM-IV­
TR). 4th ed. Washington, D .C.: American Psychiatric Association, 2000.

Arendt, Hannah. The Human Condition. Chicago: University of Chicago Press, 1958.

Arijon, Daniel. Grammar of the Film Language. London: Focal Press, 1 9 76 .

Arnold, V. 1. Geometrical Methods i n the Theory o f Ordinary Differential Equations. Trans. Joseph
Szucs, ed. Mark Levi. New York: Springer, 1988.

Arnold, V. 1. Mathematical Methods of Classical Mechanics. Trans. K. Vogtmann and A. Weinstein.

2nd ed. New York: Springer, 1989.

Aronowitz, Stanley, Barbara Martinson, and Michael Menser, eds. Technoscience and Cyberculture.
New York: Routledge, 1996.

Arrow, Kenneth. "A Difficulty in the Concept of Social Welfare." Journal of Political Economy 58,
no. 4 (1 950) : 328-346.
330 Bibliography

Artaud, Antonin. The Theatre and Its Double: Essays. London: Calder and Boyars, 1 9 70.

Badiou, Alain. Being and Event. Trans. Oliver Feltham. London: Continuum, 2006.

Badiou, Alain. Infinite Thought: Truth and the Return to Philosophy. Trans. Oliver Felham and Justin
Clement. London: Continuum, 1 998.

Badiou, Alain. Theoretical Writings. Ed. and trans. Ray Brassier and Alberto Toscano. London:
Continuum, 2004.

Barad, Karen. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and
Meaning. Durham: Duke University Press, 2007.

Barr, AI, James Arvo, et al. "The Smart Board Project: Human-Computer Interaction for Direct
Manipulation of Formal Systems. " 2000.

Barrett, Edward, ed. Sociomedia: Multimedia, Hypermedia and the Social Construction of Knowledge.
Cambridge, Mas s . : MIT Press, 1992.

Bataille, Georges . The Accursed Share: An Essay on General Economy. New York: Zone Books, 1988.

Batra, D., and J . G . Davis. "Conceptual Data Modelling in Database Design: Similarities and Dif­
ferences between Expert and Novice Designers. " International Journal of Man-Machine Studies 3 7
(1 992): 83-10 1 .

Baudrillard, Jean. Simulacra and Simulation. Trans. Sheila Faria Glaser. Ann Arbor: University of
Michigan Press, 1 994.

Baudrillard, Jean. Symbolic Exchange and Death. Trans. lain Hamilton Grant. London: Sage, 1993.

Baudrillard, Jean. The System of Objects. Trans . James Benedict. London: Verso, 1 996.

Bell, John S. Speakable and Unspeakable in QJJa ntum Mechanics. 2nd ed. Cambridge: Cambridge
University Press, 2004.

Bell, Tony. " Levels and Loops: The Future of Artificial Intelligence and Neuroscience. " Philosophi­
cal Transactions of the Royal Society of London B 3 5 4 ( 1 999): 201 3-2020.

Bellamy, Rachel, E. B . Cooper, and R. D. Borovoy. "Supporting Collaborative Learning through

the Use of Electronic Conversational Props." East-West Conference on Human-Computer Interac­
tion, 1 994.

Bergson, Henri. Duration and Simultaneity: Bergson and the Einsteinian Universe. Ed. Robin Durie.
2nd ed. Manchester: Clinamen Press, 1999.

Bergson, Henri. Matter and Memory. Trans. Nancy M . Paul and W. Scott Palmer. Seattle: Amazon/
CreateSpace, 20 1 0 .

Berners-Lee, T. J . "The World Wide Web. " Talk a t Stanford P C D seminar, C S 5 4 7, 1 9 9 3 .

Berners-Lee, T. J . , et al. "World-Wide Web: The Information Universe . " Electronic Networking:
Research, Applications and Policy 2 ( 1 992) : 52-58 .
Bibliography 331

Bertin, Jacques. Semiology o f Graphics: Diagrams, Networks, Maps. Trans. William J. Berg. Madison:
University of Wisconsin Press, 1983.

Bevilacqua, Frederic, Bruno Zamborlin, Anthony Sypniewski, Norbert Schnell, Fabrice Guedy, and
Nicolas Rasamimanana. " Continuous Realtime Gesture Following and Recognition. " In Gesture
in Embodied Communication and Human-Computer Interaction, ed. Stefan Kopp and Ipke Wachs­
muth, 73-84. Berlin: Springer, 2010.

Bickhard, Mark H. "Emergence. " In Downward Causation, ed. P. B . Andersen et al., 322-348.
Aarhus: University of Aarhus Press, 2000.

Bickhard, Mark H. " Process and Emergence: Normative Function and Representation. " In Process
Theories: Crossdisciplinary Studies in Dynamic Categories, ed. J. Seibt. Dordrecht: Kluwer Academic,
2003 .

Bickhard, Mark H., and 1. Terveen. Foundational Issues in Artificial Intelligence and Cognitive Science:
Impasse and Solution. Amsterdam: Elsevier Scientific, 1995.

Bilgrami, Akeel. "Occidentalism, the Very Idea: An Essay on Enlightenment and Enchantment. "
Critical Inquiry 3 2 (2006) : 381-4 1 1 .

Birringer, Johannes. Media and Performance. Baltimore: Johns Hopkins University Press, 1998.

Bitbol, Michael, Pierre Kerszberg, and Jean Petitot. Constituting Objectivity: Transcendental Perspec­
tives on Modern Physics. Berlin: Springer, 2009.

Bloor, David. Knowledge and Social Imagery. London: Routledge and Kegan Paul, 1 9 76.

Bobick, Aaron, et al. "The KidsRoom: A Perceptually-Based Interactive and Immersive Story
Environment." Presence: Teleoperators and Virtual Environments 8, no. 4 (August 1999): 367-39 1 .

Boguraev, Bran, and Ted Briscoe, eds. Computational Lexicography for Natural Language Processing.
London: Longman, 1989.

Boguraev, Bran, and J. Pustejovsky. " Lexical Knowledge: Acquisition and Representation. " In
preparation.

Bohm, David. Fragmentation and Wholeness. Jerusalem: Van Leer Jerusalem Foundation, 1 9 7 6 .

B6k, Christian. 'Pataphysics: The Poetics of an Imaginary Science. Evanston: Northwes�rn Univer­
sity Press, 2002.

Bono, James J. "Why Metaphor? Toward a Metaphorics of Scientific Practice. " In Science Studies:
Probing the Dynamics of Scientific Knowledge, ed. Sabine Maasen and Matthias Winterhager, 2 1 5-
234. Bielefield: Transcript, 200 1 .

Booch, Grady. Object Oriented Design with Applications. Redwood City, Calif. : Benjamin/Cum­
mings, 1 9 9 1 .

Boretz, Benjamin, and Edward T. Cone. Perspectives on Notation a n d Performance. New York:
Norton, 1976.
332 Bibliography

Born, Georgina, and Andrew Barry. "Art Science . " Journal of Cultural Economy 3, no. 1 (20 10) :
103-1 1 9 .

Brecht, Bertolt. Brecht on Theater. New York: Hill and Wang, 1964.

Brook, Peter. The Empty Space. New York: Atheneum, 1 968.

Brouwer, L. E. ] . Col/ected Works. Ed. A. Heyting and H. Freudenthal. 2 vols. Amsterdam: North­
Holland, 1 9 7 5 .

Buford, John E, Lloyd Rutledge, and John E Rutledge. " Integrating Object-Oriented Scripting
Languages with HyTime . " Proceedings of the International Conference on Multimedia Comput­
ing and Systems, 1994.

Bullivant, Lucy. Responsive Environments: Architecture, Art and Design. London: Victoria and Albert
Museum, 2006.

Byers, Nina. "E. Noether's Discovery of the Deep Connection between Symmetries and Conserva­
tion Laws. " Paper presented at the Symposium on the Heritage of Emmy Noether, December 2-4,
1996.

Cabrera, Bias; James Lewis Terman, and Sha Xin Wei. Physics Simulations. Computer software,
Stanford University, 1 986.

Cage, John. Fontana Mix. New York: C. E Peters, 1 9 6 5 .

Cage, John. Silence: Lectures and Writings. Middletown, Conn. : Wesleyan University Press, 1 9 6 1 .

Caillois, Roger, and Meyer Barash. Man, Play, and Games. Urbana: University o f Illinois Press, 200 1 .

Carlson, Marvin. Performance, a Critical Introduction. London: Routledge, 1 996.

Cassirer, Ernst. An Essay on Man: An Introduction to a Philosophy of Human Culture. New Haven:
Yale University Press, 1944.

Casti, John L., and Anders Karlqvist, eds. Real Brains, Artificial Minds. New York: North-Holland,
1987.

Chadabe, Joel. Electric Sound: The Past and Promise of Electronic Music. Upper Saddle River, N.J . :
Prentice Hall, 1 9 9 7 .

Changeux, Jean-Pierre, and Alain Connes. Conversations o n Mind, Matter, and Mathematics. Prince­
ton: Princeton University Press, 1995.

Chase-Dunn, Christopher K. Global Formation: Structures of the World Economy. Rev. ed. Lanham,
Md. : Rowman and Littlefield, 1 998.

Cheng, Ho-Lun, Tarnal K. Dey, Herbert Edelsbrunner, and John Sullivan. " Dynamic Skin Trian­
gulation. " Discrete Computational Geometry 25 (2001 ) : 5 25-5 68.

Cirone, Anthony J. On Musical Interpretation in Percussion Performance. Galesville, Md. : Meredith

Music Publications, 2008.
Bibl iography 333

Clarke, E., and X. Zhao. "Analytica: A Theorem-Prover in Mathematica. " Proceedings of the 1 1th
International Conference on Automated Deduction, 1 992.

Connes, Alain. Noncommutative Geometry. San Diego: Academic Press, 1994.

Connes, Alain, and Matilde Marcolli. Noncommutative Geometry, Quantum Fields and Motives.
Providence, R.I . : American Mathematical Society, 2008.

Cook, Nicholas. A Guide to Musical Analysis. London: J. M. Dent, 1 9 8 7 .

Cooley, Mike. Architect or Bee: The Human Price of Technology. London: Hogarth Press, 1 98 7.

Cope, David. Virtual Music: Computer Synthesis ofMusical Style. Cambridge, Mass. : MIT Press, 200 1 .

Cotter, Sean, and Michael J. Potel. Inside Taligent Technology. Reading, Mass. : Addison-Wesley,
1995.

Coyne, Richard. Designing Information Technology in the Postmodern Age: From Method to Metaphor.
Cambridge, Mass. : MIT Press, 1 9 9 5 .

Crosby, Donald A. "Whitehead on the Metaphysical Employment o f Language. " Process Studies
I, no. 1 (Spring 1 9 7 1) : 38-54.

Crumb, George. Ancient Voices of Children. Text by Federico Garda Lorca. New York: C. F. Peters,
1 9 70.

Cutting, Douglass R., David Karger, and Jan Pedersen. " Constant Interaction-Time Scatter/Gather
Browsing of Very Large Document Collections . " In Proceedings of the 1 6th Annual International
ACM SIGIR Conference on Research and Development in Information Retrieval, 1 26-1 3 5 . New York:
ACM Press, 1993.

Cutting, Doug, Jan Pedersen, and Per-Kristian Halvorsen. "An Object-Oriented Architecture for
Text Retrieval. " In RIAO 91: Conference Proceedings with Presentation of Prototypes and Operational
Systems: Intelligent Text and Image Handling. Barcelona: Universitat Aut6noma de Barcelona, 1 99 1 .

Davidson, Arnold. "The Horror o f Monsters. " I n The Boundaries of Humanity: Humans, Animals,
Machines, ed. James J. Sheehan and Morton Sosna, 3 6-67 . Berkeley: University of California Press,
1 99 1 .

Davis, Marc Eliot. "Media Streams : Representing Video for Retrieval and Repurpo'si ng. " Ph.D.
thesis, MIT, 1995.

Dean, Roger. Hyperimprovisation: Computer-Interactive Sound Improvisation. Middleton, Wisc.: A-R

Editions, 2003 .

Debord, Guy. The Society of the Spectacle. New York: Zone Books, 1994.

Debray, Regis. Media Manifestos. 1 994, 1996.

Deleuze, Gilles. Cinema. 2 vols. Minneapolis: University of Minnesota Press, 1986.

Deleuze, Gilles. Desert Islands and Other Texts, 1 953-1 974. Cambridge, Mass. : Semiotext(e), 2004.
334 Bibliography

Deleuze, Gilles . Difference and Repetition. New ed. London: Continuum, 2004.

Deleuze, Gilles . L'image-mouvement. Paris: Editions de Minuit, 1983.

Deleuze, Gilles. L'image-temps. Paris: Editions de Minuit, 1985.

Deleuze, Gilles. " Immanence: A Life . " In Deleuze, Pure Immanence: Essays on Life, trans. Anne
Boyman. Cambridge, Mass. : Zone Books, 200 1 .

Deleuze, Gilles, and Felix Guattari. Capitalisme e t schizophrenie, Mille plateaux. Paris: Editions de
Minuit, 1980.

Deleuze, Gilles, and Felix Guattari. Kafka: Towards a Minor Literature. Minneapolis: University of
Minnesota Press, 1 986.

Deleuze, Gilles, and Felix Guattari. A Thousand Plateaus. Trans. Brian Massumi. Minneapolis:
University of Minnesota Press, 1987.

Deleuze, Gilles, and Felix Guattari. What Is Philosophy? Trans. Hugh Tomlinson and Graham
Burchell. New York: Columbia University Press, 1996.

Derrida, Jacques. Margins ofPhilosophy. Trans. Alan Bass. Chicago: University of Chicago Press, 1985.

Derrida, Jacques. Of Grammatology. Trans. Gayatri Chakravorty Spivak. Baltimore: Johns Hopkins
University Press, 1 9 76; corrected ed. 1998.

Derrida, Jacques. Speech and Phenomena: And Other Essays on Husserl's Theory of Science. Trans.
David B. Allison and Newton Carver. Evanston: Northwestern University Press, 1 9 7 3 .

Derrida, Jacques. Writing and Difference. Chicago: University o f Chicago Press, 1 9 78.

Descombes, Vincent. The Mind's Provisions: A Critique of Cognitivism. Trans. Stephen Adam
Schwartz. Princeton: Princeton University Press, 200 1 .

Doane, Mary Ann. The Emergence o f Cinematic Time: Modernity, Contingency, the Archive. Cam­
bridge, Mass. : Harvard University Press, 2002.

Dourish, Paul. Where the Action Is: The Foundations ofEmbodied Interaction. Cambridge, Mass. : MIT
Press, 200 1 .

Drucker, Johanna. The Visible Word: Experimental Typography and Modem Art, 1 909-1923. Chicago:
University of Chicago Press, 1 994.

Dubnov, Shlomo, Gerard Assayag, Olivier Lartillot, and Gill Bejerano. "Using Machine-Learning
Methods for Musical Style Modeling." IEEE Computer Society Press 36, no. 10 (2003): 73-80.

Duda, Richard 0., Peter E. Hart, and David G. Stork. Pattern Classification. 2nd ed. New York:
Wiley, 200 1 .

Dupuy, Jean Pierre. The Mechanization o f the Mind: O n the Origins o f Cognitive Science. Princeton:
Princeton University Press, 2000.
Bibliography 335

Eagleton, Terry. Ideology: An Introduction. London: Verso, 1 99 1 .

Eastman, Timothy E., and Hank Keeton, eds. Physics and Whitehead: Quantum, Process, and Experi­
ence. Albany: State University of New York Press, 2004.

Ehn, Pelle. "Towards a Philosophical Foundation for Skill-Based Participatory Design. " In Usabil­
ity: Turning Technologies into Tools, ed. Paul S. Adler and Terry A. Winograd, 1 1 6-132. New York:
Oxford University Press, 1992.

El-Mallah, Issam. Arab Music and Musical Notation. Tutzing: Hans Schneider Verlag, 1 9 9 7 .

Epperson, Michael. Quantum Mechanics and the Philosophy of Alfred North Whitehead. New York:
Fordham University Press, 2004.

Ernest, Paul. Social Constructivism as a Philosophy of Mathematics. Albany: State University of New
York Press, 1998.

Evans, Kim 1., and Kristin Steslow. "A Rest from Reason: Wittgenstein, Drury, and the Difference
between Madness and Religion. " Philosophy 85, no. 2 (2010) : 245-258.

Evans, Robin. The Projective Cast: Architecture and Its Three Geometries. Cambridge, Mass. : MIT
Press, 1995.

Ezrahi, Yaron, Everett Mendelsohn, and Howard P. Segal, eds. Technology, Pessimism, and Postrnod­
ernism. Amherst: University of Massachusetts Press, 1995.

Faloutsos, c., W. Equitz, M. Flickner, W. Niblack, D . Petkovic, and R. Barber. "Efficient and Effec­
tive Querying by Image Content. " Journal of Intelligent Information Systems 3 ( 1 994) : 231-262.

Favret-Saada, Jeanne. Deadly Words: Witchcraft in the Bocage. Trans. Catherine Cullen. Cambridge:
Cambridge University Press, 1 980.

Federer, Herbert. Geometric Measure Theory. Berlin: Springer, 1969.

Flusser, Vilem. Writings. Ed. Andreas Strohl, trans . Erik Eisel. Minneapolis: University of Minne­
sota, Press, 2002.

FoAM. " Grow Your Own Worlds. " http://fo.am/.

FoAM. "txoom Play Spaces. " 2002. http://fo.am/txoom/.

FoAM and Time's Up. Trg: On Transient Realities and Their Generators. Ed. Maj a Kuzmanovic and
Tim Boykett. Brussels: Bezjak Tisk Maribor, 2006.

Fonds quebecois de la recherche sur la societe et la culture. "Report: Comite d'etude sur Ie finance­
ment du secteur des arts et lettres . " 2003 .

Forsythe, William. Improvisation Technologies. DVD. Ostfildern: Hatje Cantz Verlag, 2000.

Foucault, Michel. The Archaeology ofKnowledge. Trans. A. M. Sheridan Smith. New York: Pantheon,
1 9 72.
336 B i b liography

Foucault, Michel. The Birth of the Clinic: An Archaeology ofMedical Perception. New York: Vintage, 1994.

Foucault, Michel. Discipline and Punish: The Birth of the Prison. 2nd Vintage Books ed. New York:
Vintage, 1995.

Foucault, Michel. " Governmentality. " In The Essential Works ofFoucault, 1 954-1 984, vol. 3, Power,
ed. James D. Faubion, 201-222. New York: New Press, 2000.

Foucault, Michel. The History of Sexuality. Trans. Robert Hurley. New York: Vintage, 1988.

Foucault, Michel. Madness and Civilization: A History ofInsanity in the Age ofReason. Trans. Richard
Howard. New York: Vintage, 1 9 7 3 .

Frankel, Theodore. Gravitational Curvature: A n Introduction to Einstein's Theory. San Francisco: W.

H. Freeman, 1 9 79 .

Frankl, Paul. Principles o f Architectural History: The Four Phases o f Architectural Style, 1 420-1 900.
Trans. James F. O'Gorman. Cambridge, Mass. : MIT Press, 1 968.

Fraser, J. T. , N. Lawrence, and F. C. Haber, eds. Time, Science, and Society in China and the West.
Amherst: University of Massachusetts Press, 1 986.

Gaines, Brian R., and Mildred L. G. Shaw. "Open Architecture Multimedia Documents. " Proceed­
ings of the ACM International Conference on Multimedia, 1993, 1 3 7-146.

Geertz, Clifford. The Interpretation of Cultures: Selected Essays. London: Hutchinson, 1 9 7 5 .

Gelfand, 1 . M., S . V. Fomin, and Richard A. Silverman. Calculus of Variations. Mineola, N.Y. : Dover,
2000.

Gemperle, Francine, Chris Kasabach, John Stivoric, Malcolm Bauer, and Richard Martin. " Design
for Wearability. " In Digest of Papers, Second International Symposium on Wearable Computers, 1 1 6-
122. Los Alamitos, Calif. : IEEE Computer Society, 1 998.

Gendlin, Eugene T. Experiencing and the Creation of Meaning: A Philosophical and Psychological
Approach to the Subjective. Evanston: Northwestern University Press, 1997.

Genette, Gerard. The Work ofArt: Immanence and Transcendence. Trans. G. M. Goshgarian. Ithaca:
Cornell University Press, 1 9 9 7 .

Gershenfeld, Neil A. When Things Start to Think. New York: Henry Holt, 1999.

Geuss, Raymond. The Idea of a Critical Theory: Habermas and the Frankfurt School. Cambridge:
Cambridge University Press, 1 9 8 1 .

Gilarranz, Julio, Julio Gonzalo, and Felisa Verdejo. " Language-Independent Text Retrieval with
the Eurowordnet Multilingual Semantic Database. " In Second Workshop on Multilinguality in the
Software Industry: The AI Contribution, August 1 997, ed. C. D. Spyropoulos, 9-1 6. San Francisco:
Morgan Kaufmann, 1 9 9 7 .

Gill, Satinder. "Body Moves a n d Tacit Knowing . " In Cognitive Technology: In Search of a Humane
Interface, ed. Barbara Gorayska and Jacob L. Mey. Amsterdam: North-Holland, 2004.
Bibliography 337

Gill, Satinder, M. Kawamori, Y. Katagiri, and A. Shimojima. "The Role o f Body Moves in Dia­
logue. " International Journal for Language and Communication 12 (April 2000) : 89-1 14.

Gins, Madeline, and Shiisaku Arakawa. Architectural Body. Tuscaloosa: University of Alabama Press,
2002.

Godel, Kurt. "An Example of a New Type of Cosmological Solutions of Einstein's Field Equations
of Gravitation. " Reviews of Modern Physics 21 (1 949) : 447-450.

Goldberg, Adele. SmallTalk-80: Blue Book: The Language and Its Implementation. Reading, Mass.:
Addison-Wesley, 1983.

Goldberg, Adele. SmallTalk-80: Red Book: The Interactive Programming Environment. Reading, Mass. :
Addison-Wesley, 1983.

Goldberg, RoseLee. Performance Art: From Futurism to the Present. Rev. ed. New York: Thames and
Hudson, 200 1 .

Goldberg, RoseLee. Performance: Live Art since the '60s. New York: Thames and Hudson, 2004.

Goldfarb, Charles F. The SGML Handbook. Ed. Yuri Rubinsky. Oxford: Oxford University Press, 1 990.

Goldsmith, David S. " On the Priority of the Musical Impulse and the Acoustic Limits to Sonic
Gesture. " Journal of Aesthetics and Art Criticism 32 ( 1 9 74): 409-4 1 3 .

Goldstein, Mark. "Playing Electronic Instruments: Technique Meets Technology. " People, Com­
puters and Design Seminar, Stanford University, 1 9 9 7 .

Goodman, Nelson. Languages of Art: An Approach to a Theory of Symbols. Indianapolis: Bobbs­

Merrill, 1968.

Goodman, Nelson. Ways of Worldmaking. Hassocks, U.K.: Harvester Press, 1 9 78.

Goodman, Nelson, and Catherine Z. Elgin. Reconceptions in Philosophy and Other Arts and Sciences.
Indianapolis: Hackett, 1988.

Griffiths, H. B. Surfaces. Cambridge: Cambridge University Press, 1 9 7 6 .

Grosse-Brauckmann, Karsten, Robert B . Kusner, and John M. Sullivan. "Triunduloids: Embedded

Constant Mean Curvature Surfaces with Three Ends and Genus Zero . " Journal {iir die reine und
angewandte Mathematik (Crelle's Journal) 5 64 (2003): 3 5-6 1 .

Grotowski, Jerzy. Akropolis. Videotape o f performance; Arthur Cantor Films 1 . 800.23 7.3801, ZVC
1 3072, 1968.

Grotowski, Jerzy. Towards a Poor Theatre. New York: Simon and Schuster, 1968.

Grotowski, Jerzy, and Eugenio Barba. Towards a Poor Theatre. New York: Routledge, 2002.

Grundlehner, Philip. "Manual Gesture in Kafka's Prozess. " German Quarterly 55 (1982): 1 86-199.

Guattari, Felix. Chaosmosis: An Ethico-Aesthetic Paradigm. Trans. Paul Bains and Julian Pefanis.
Bloomington: Indiana University Press, 1995.
338 Bibliography

Guest, Ann Hutchinson. Choreo-Graphics: A Comparison ofDance Notation Systems from the Fifteenth
Century to the Present. New York: Gordon and Breach, 1989.

Guest, Ann Hutchinson. Labanotation. New York: New Directions, 1954.

Gunn, Charles, and Mark Phillips. "Geomview. " Technical report, The Geometry Center, 1993.

Haas, Cathy, and Sha Xin Wei. "Stanford American Sign Language Videodisc Project." In Proceed­
ings of the Johns Hopkins National Search for Computing Applications to Assist Persons with Disabilities,
4 1-44. Los Alamitos, Calif. : IEEE Computer Society Press, 1 992.

Haas, Christina. Writing Technology: Studies on the Materiality of Literacy. Mahwah, N.] . : Lawrence
Erlbaum Associates, 1996.

Hacking, Ian. The Social Construction of What? Cambridge, Mass. : Harvard University Press, 1 999.

Hadamard, Jacques . The Mathematician's Mind: The Psychology of Invention in the Mathematical
Field. Princeton: Princeton University Press, 1945.

Halpern, Orit. " Dreams for Our Perceptual Present: Archives, Interfaces, and Networks in Cyber­
netics. " Configurations 1 3 , no. 2 (2007): 283-320.

Hamilton, William Rowan, and Charles Jasper Joly. Elements of Quatemions. 3d ed. New York:
Chelsea, 1969.

Harada, Mikako, Andrew Witkin, and David Baraff. "Interactive Physically-Based Manipulation
of Discrete/Continuous Models." SIGGRAPH 95 Conference Proceedings, ACM SIGGRAPH, 1 9 9 5 .

Haraway, Donna Jeanne. Crystals, Fabrics, and Fields: Metaphors That Shape Embryos. Berkeley,
Calif. : North Atlantic Books, 2004.

Haraway, Donna Jeanne. The Haraway Reader. New York: Routledge, 2003.

Haraway, Donna Jeanne. Primate Visions: Gender, Race, and Nature in the World of Modem Science.
New York: Routledge, 1989.

Haraway, Donna Jeanne. Simians, Cyborgs, and Women: The Reinvention of Nature. New York:
Routledge, 1 99 1 .

Hardcastle, Valerie Gray, and C . Matthew Stewart. " What D o Brain Data Really Show?" Philosophy
of Science 69 (September 2002): S 72-S82.

Hardt, Michael, and Antonio Negri. Empire. Harvard University Press, 2000.

Hardt, Michael, and Antonio Negri. Multitude: War and Democracy in the Age of Empire. New York:
Penguin, 2004.

Harris, Roy. The Origin of Writing. London: Duckworth, 1986.

Harris, Roy. Signs of Writing. London: Routledge, 1995.

Hauser, Arnold. The Social History of Art. 4 vols. New York: Vintage, 195 1 .
Bibliography 339

Hawking, S. W., and G. F. R. Ellis . The Large Scale Structure of Space-Time. Cambridge: Cambridge
University Press, 1 9 7 3 .

Hayden, Corio "A Generic Solution? Pharmaceuticals and the Politics o f the Similar i n Mexico. "
Current Anthropology 48, no. 4 (2007): 4 7 5-495 .

Hege, Hans-Christian, and Konrad Polthier. Mathematical Visualization. Berlin: Springer, 1998.

Heidegger, Martin. The Metaphysical Foundations of Logic. 1 928i Bloomington: Indiana University
Press, 1984.

Heidegger, Martin. Poetry, Language, Thought. New York: HarperCollins, 200 1 .

Heidegger, Martin, and Eugen Fink. Heraclitus Seminar. Trans. Charles H. Seibert. Evanston, Ill . :
Northwestern University Press, 1 9 9 3 .

Heidenreich, Achim. " 'Shaping Electronic Sounds Like Clay': The Historical Situation and Aes­
thetic Position of Electroacoustic Music at the Zkm Institute for Music and Acoustics. " Organised
Sound 1 4, no. 3 (December 2009): 248-256.

Hemel, Annemoon van, Hans Mommaas, and Cas B . Smithuijsen. Trading Culture: GATT; European
Cultural Policies and the Translatlantic Market. Amsterdam: Boekman Foundation, 1996.

Hendricks, Barbara E. Designing for Play. Aldershot, U.K.: Ashgate Publishing, 200 1 .

Henry, Michel. Material Phenomenology. Trans. Scott Davidson. New York: Fordham University
Press, 2008.

Heraclitus . The Art and Thought of Heraclitus: An Edition of the Fragments with Translation and Com­
mentary. Ed. and trans. Charles H. Kahn. Cambridge: Cambridge University Press, 1 9 79.

Herman, 1 . , et al. "Premo: An ISO Standard for Presentation Environment for Multimedia Objects. "
Proceedings of the ACM International Conference o n Multimedia, 1 994, 1 1 1-1 1 8 .

Hexagram. "Hexagram Fund Guidelines . " Hexagram Fund for University Research a n d Creation
in Media Arts and Technologies, Montreal, 2005

Hexagram. "Hexagram Funding Competition Evaluation Criteria. " Hexagram-Concordia Centre

for Research-Creation in Media Arts and Technologies, Montreal, 2007 .

Hill, Christopher. The World Turned Upside Down: Radical Ideas During the English Revolution. New
York: Viking Press, 1 9 72.

Hillier, Bill, and Julienne Hanson. The Social Logic of Space. Cambridge: Cambridge University
Press, 1984.

Hirata, Kyoji, Yoshinori Hara, Naoki Shibata, and Fusako Hirabayashi. "Media-Based Navigation
for Hypermedia Systems. " In Hypertext 93: Proceedings of the Fifth A CM Conference on Hypertext,
1 5 9-1 7 3 . New York: ACM, 1 993.

Hobbes, Thomas. Leviathan. 1 65 1 i New Haven: Yale University Press, 20 10.

340 Bibliography

Holt, Anatol W. Organized Activity and Its Support by Computer. Dordrecht: Kluwer Academic, 1 9 9 7 .

Hopf, Heinz. Differential Geometry in the Large: Seminar Lectur�s, New York University, 1 946 and
Stanford University, 1 956. Berlin: Springer, 1983.

Horn, Bob. Guide to Simulations/Games for Education and Training. 4th ed. London: Sage, 1 980.

Horwich, Paul. Asymmetries in Time: Problems in the Philosophy of Science. Cambridge, Mass. : MIT
Press, 1987.

Hudson, S. E., and G. 1. Newell. " Probabilistic State Machines: Dialog Management for Inputs
with Uncertainty. " Proceedings of the ACM Symposium on User Interface Software and Technol­
ogy, 1 992, 199-208 .

Huizinga, Johan. Homo Ludens: A Study of the Play-Element in Culture. London: Routledge and
Kegan Paul, 1 949.

Husserl, Edmund. Ideas Pertaining to a Pure Phenomenology and to a Phenomenological Philosophy.

The Hague: M. Nijhoff, 1 980.

Husser!, Edmund. " Origins of Geometry. " Appendix IV in Husser!, Crisis of European Sciences,
trans . David Carr. Evanston: Northwestern University Press, 1 9 70.

Husser!, Edmund. The Phenomenology ofInternal Time-Consciousness. 1 928; Bloomington: Indiana

University Press, 1 964.

Hutchins, Edwin. Cognition in the Wild. Cambridge, Mass.: MIT Press, 1995.

Hyde, Lewis. The Gift: Imagination and the Erotic Life of Property. New York: Vintage, 1983.

Iannello, Kathleen. Decisions without Hierarchy: Feminist Interventions in Organization Theory and
Practice. New York: Routledge, 1992.

Ihde, Don. Technology and the Lifeworld: From Garden to Earth. Bloomington: Indiana University
Press, 1990.

International Standard Organization. "Hypermedia/Time-Based Document Structuring Language

(Hytime) . " ISO/lEC IS 1 0 744, April 1 992.

IRCAM. "CataRT: Real-Time Corpus-Based Concatenative Synthesis . " http://imtr.ircamJr/imtr/

CataRT (retrieved July 7, 201 2) .

IRCAM. Gesture Follower, http://imtr.ircam.fr/imtr/Gesture_Follower (retrieved July 7, 20 12).

IRCAM. OMAX, http://omax.ircam.fr/ (retrieved July 7, 2012).

Itzykson, Claude, and Jean-Michel Drouffe. Statistical Field Theory. 2 vols. Cambridge: Cambridge
University Press, 1989.

Jackiw, R. Nicholas, and William F. Finzer. "The Geometer's Sketchpad: Programming by Geom­
etry. " In Watch What I Do: Programming by Demonstration, ed. Allen Cypher, 293-308. Cambridge,
Mass. : MIT Press, 1 9 9 3 .
Bibl iog raphy 341

James, William. The Principles of Psychology. 1890; Cambridge, Mass . : Harvard University Press,
1983.

Janich, Klaus. Topology. Berlin: Springer, 1984.

Jay, Martin. Downcast Eyes: The Denigration of Vision in Twentieth Century French Thought. Berkeley:
University of California Press, 1 9 9 3 .

Jeremijenko, Natalie. " Dialogue with a Monologue: Voice Chips and the Products o f Abstract
Speech. "

Johnson, Mark. Th e Body i n the Mind: Th e Bodily Basis o f Meaning, Imagination, and Reason.
Chicago: University of Chicago Press, 19 87.

Jordan, Michael I., e d . Learning in Graphical Models. Cambridge, Mass. : MIT Press, 1 999.

Kac, Marc. "Can O n e Hear t h e Shape o f a Drum?" American Mathematical Monthly 73, n o. 4, part
2 ( 1 966): 1-23.

Kant, Immanuel. The Critique of Pure Reason. Trans. Paul Guyer and Allen W. Wood. Cambridge:
Cambridge University Press, 1999.

Kappe, Frank, Keith Andrews, Jorg Faschingbauer, Mansuet Gaisbauer, Michael Pichler, and
Jiirgen Schipflinger. " Hyper-G: A New Tool for Distributed Hypermedia. " International Confer­
ence on Distributed Multimedia Systems and Applications, 1994.

Katznelson, Yitzhak. An Introduction to Harmonic Analysis. 3rd ed. Cambridge: Cambridge Univer­
sity Press, 2004.

Keller, Catherine, and Anne Daniell. Process and Difference: Between Cosmological and Poststructural­
ist Postmodernisms. Albany: State University of New York Press, 2002.

Kircher, Athanasius. A thanasii Kircheri . . . Ars Magna Sciendi, in xii libros digesta, qua nova & uni­
versali methodo per artificiosum combinationum contextum de omni re proposita plurimis & prope
infinitis rationibus disputari, omniumque summaria quaedam cognito comparari potest. Amsterdam:
J. Jansson, 1 669.

Kittler, Friedrich A. Discourse Networks 1 800/1 900. Stanford: Stanford University Press, 1990.

Klas, W., E. Neuhold, and M. Schrefl. "Visual Databases Need Data Models for Multirl!!, e dia Data . "
In Visual Database Systems, e d . Toshiyasu Kunii, 433-462. New York: North-Holland, " 1 989.

Knorr-Cetina, Karin. Epistemic Cultures: How the Sciences Make Knowledge. Cambridge, Mass . :
Harvard University Press, 1999.

Kobayashi, Shoshichi, and Katsumi Nomizu. Foundations of Differential Geometry. 2 vols. New
York: Wiley, 1996.

Koppen, Mario. "The Curse of DimenSionality. " Proceedings of the Fifth Online World Confer­
ence on Soft Computing in Industrial Applications, September 4, 2000, 4-8.

Kovar, 1., and M. Gleicher. "Simplicial Families of Drawings. " Proceedings of the ACM Sympo­
sium on User Interface Software and Technology, 2001 , 1 63-1 72.
342 Bibliography

Krantz, David H., R. Duncan Luce, Patrick Suppes, and Amos Tversky. Foundations of Measurement.
3 vols. New York: Academic Press, 1 9 7 1-1990.

Krueger, Myron. Artificial Reality II. 2nd ed. Reading, Mass. : Addison-Wesley, 1 9 9 1 .

Krueger, Myron W., T. Gionffrido, and K. Hinrichsen. "Videoplace, a n Artificial Reality. " Proceed­
ings of the ACM CHI 85 Human Factors on Computing Systems Conference, 1985, 35-40.

Kwinter, Sanford. Architectures of Time: Toward a Theory of the Event in Modernist Culture. Cam­
bridge, Mass. : MIT Press, 200 1 .

Laban, Rudolf. Schri(ttanz: Eine Vierteljahresschrift. 4 vols. 1 928-193 1 ; Leipzig: Georg Olms, 1 99 1 .

Laban, Rudolf von . A Vision of Dynamic Space. Compo Lisa Ullmann. London: Taylor and Francis,
1984.

Laban, Rudolf von, and Lisa Ullmann. Choreutics. London: Macdonald and Evans, 1966.

Lachterman, David. The Ethics of Geometry. London: Routledge, 1 989.

Laclau, Ernesto, and Chantal Mouffe. Hegemony and Socialist Strategy: Towards a Radical Democratic
Politics. 2nd ed. London: Verso, 200 1 .

Lakatos, Imre. Proofs and Refutations: Th e Logic o fMathematical Discovery. Cambridge: Cambridge
University Press, 1 98 1 .

Lakoff, George. Women, Fire, and Dangerous Things: What Categories Reveal About the Mind. Chicago:
University of Chicago Press, 1 9 8 7 .

Lakoff, George, a n d Mark Johnson. Philosophy in the Flesh: The Embodied Mind and Its Challenge
to Western Thought. New York: Basic Books, 1999.

Lakoff, George, and Rafael E. Nunez. Where Mathematics Comes From: How the Embodied Mind
Brings Mathematics into Being. New York: Basic Books, 2000.

Landau, Rubin H., and Manuel Jose Paez Mejia. Computational Physics: Problem Solving with Com­
puters. New York: Wiley, 1 9 9 7 .

Landau, Rubin H . , and M. J. paez. A First Course in Scientific Computing: Symbolic, Graphical, and
Numeric Problem Solving Using Maple, Java, Mathematica, and Fortran. Princeton: Princeton Univer­
sity Press, 2005 .

Landow, George, and Paul Delany, eds. Hypermedia and Literary Studies. Cambridge, Mass. : MIT
Press, 1 99 1 .

Langer, Susanne K . An Introduction to Symbolic Logic. 3rd ed. New York: Dover, 1 9 6 7 .

Lao-Tzu. Tao Te Ching. Trans. with a n intro. b y D . C. Lau. Baltimore: Penguin Books, 1 9 6 3 .

Latour, Bruno. Science in Action: How to Follow Scientists and Engineers through Society. Cambridge,
Mass. : Harvard University Press, 1 9 8 7 .

Latour, Bruno. We Have Never Been Modern. Trans. Catherine Porter. Cambridge, Mass. : Harvard
University Press, 1993.
Bibl iography 343

Latour, Bruno. "Why Has Critique Run Out of Steam? From Matters of Fact to Matters of
Concern. " Critical Inquiry 30, no. 2 (2004) : 225-248.

Latour, Bruno, and Peter Weibel. Making Things Public: Atmospheres of Democracy. Cambridge,
Mass.: MIT Press; Karlsruhe: ZKMjCenter for Art and Media, 2005 .

Laurel, Brenda. The Art of Human-Computer Interface Design. Reading, Mass. : Addison-Wesley,
1990.

Laurel, Brenda. Computers as Theatre. Reading, Mass.: Addison-Wesley, 1 99 1 .

Lautman, Albert. Essai sur les notions de structure et d'existence en mathematiques. Paris: Universite
de Paris, 1 9 3 7 .

Lefebvre, Henri. The Production o f Space. 1 9 74; Oxford: Blackwell, 1 99 1 .

Leibniz, Gottfried Wilhelm. De summa rerum: Metaphysical Papers, 1 675-1 676. Trans. and intro.
G. H. R. Parkinson. New Haven: Yale University Press, 1 992.

Leibniz, Gottfried Wilhelm. Discourse on Metaphysics. Manchester: Manchester University Press,

1953.

Leibniz, Gottfried Wilhelm. The Early Mathematical Manuscripts of Leibniz. Trans. and intro. J. M.
Child. Mineola, N.Y. : Dover, 2005.

Leibniz, Gottfried Wilhelm. General Investigations Concerning the Analysis of Concepts and Truths.
Trans. and evaluation by Walter H. O'Briant. Athens: University of Georgia Press, 1968.

Leibniz, Gottfried Wilhelm. Monadology: An Edition for Students. Trans. Nicholas Rescher. Pitts­
burgh: University of Pittsburgh Press, 1 99 1 .

Leibniz, Gottfried Wilhelm. The Shorter Leibniz Texts: A Collection of New Translations. Trans. Lloyd
H. Strickland. London: Continuum, 2006.

Lenoir, Tim, and Sha Xin Wei. "Authorship and Surgery: The Shifting Ontology of the Virtual
Surgeon . " In From Energy to Information: Representation in Science and Technology, Art, and Literature,
ed. Bruce Clarke and Linda Dalrymple Henderson, 283-308. Stanford: Stanford University Press,
2002.

Lenoir, Tim, and Sha Xin Wei. "Networked Scholarly Workspaces for History of High Technol­
ogy. " Talk, MIT, March 1 9 9 5 .

Leroi-Gourhan, Andre. Gesture and Speech. Cambridge, Mass. : MIT Press, 1 9 9 3 .

Levi-Strauss, Claude. Structural Anthropology. Trans. Monique Layton. Chicago: University o f

Chicago Press, 1 9 8 3 .

Levy, Benj amin. "Visualising Omax." IRCAM, 2009 .

Levy, Pierre. Les technologies d'intelligence. Paris: La Decouverte, 1 990.

Liere, Robert van, Jurriaan D. Mulder, and Jarke ]. van Wijk. "Computational Steering. " Proceed­
ings of the High-Performance Computing and Networking conference, 1 996, 35-40.
344 Bibl iography

Ligeti, Gyorgy. Streichquartett = String Quartet No. 2 (1 968). Mainz: B . Schotts Sohnei New York:
Schott Music Corp., 1 9 7 1 .

Ligeti, Gyorgy. Volumina, fU r Orgel. Frankfurt, New York: H. Litolff's Verlag, 1967.

Linde, A. D. Inflation and Quantum Cosmology. Boston: Academic Press, 1990.

Lupton, Ellen, and J. Abbott Miller. Design, Writing, Research. New York: Kiosk, 1 996.

Mach, Ernst. The Science of Mechanics. Trans. Philip E. B. Jourdain. Chicago: Open Court, 1 9 1 5 .

MacKenzie, Donald. "Nuclear Missle Testing and the Social Construction o f Accuracy. " In Mario
Biagioli, ed., The Science Studies Reader, 342-3 5 7 . New York: Routledge, 1 999.

Maletic, Vera. Body, Space, Expression: The Development of Rudolf Laban 's Movement and Dance
Concepts. Berlin: Mouton de Gruyter, 1987.

Manning, Erin. Politics of Touch: Sense, Movement, Sovereignty. Minneapolis : University of Min­
nesota Press, 200 7 .

March, Lionel, and Philip Steadman. The Geometry ofEnvironment: An Introduction to Spatial Orga­
nization in Design. Cambridge, Mass. : MIT Press, 1 9 74.

Massey, William S . A Basic Course in Algebraic Topology. New York: Springer, 1 9 9 7 .

Massumi, Brian. Parables for the Virtual: Movement, Affect, Sensation. Durham: Duke University
Press, 2002.

Mathe, Nathalie. " Facilitating Access to Information in Large Documents with an Intelligent
Hypertext System." Proceedings of the 9th Aiaa Conference on Computing in Aerospace, 1993.

Mathe, Nathalie, and James Chen. "A User-Centered Approach to Adaptive Hypertext Based on
an Information Relevance Model . " Fourth International Conference on User Modeling, 1994.

Matthews, J., P. Gloor, and F. Makedon. "Videoscheme: A Programmable Video Editing System
for Automation and Media Recognition. " Proceedings of the ACM International Conference on
Multimedia, 1 993, 4 1 9-426.

Maturana, Humberto R., and Francisco J. Varela. Autopoiesis and Cognition: The Realization of the
Living. Dordrecht: D. Reidel, 1 980.

Maturana, Humberto R., and Francisco J. Varela. The Tree of Knowledge: The Biological Roots of
Human Understanding. Boston: New Science Library, 1 9 8 7 .

McCartney, James. "Supercollider: A New Real Time Synthesis Language. " Proceedings of the
International Computer Music Conference, 1996.

McConnell, Mick. LAX: The Los Angeles Experiment. Ill. Ron Bartels. New York: SITES/Lumen
Books, 1 994.

McNeil, David. Hand and Mind: What Gestures Reveal about Thought. Chicago: University of
Chicago Press, 1 992.
Bibliogra phy 345

Meeks, William H., III. "Geometric Results in Classical Minimal Surface Theory. " In Surveys in
Differential Geometry, vol. 8, 269-306. Somerville, Mass.: International Press, 2003.

Menninger, Karl. Number Words and Number Symbols: A Cultural History of Numbers. Trans. Paul
Broneer. Cambridge, Mass. : MIT Press, 1 969.

Metz, Christian. Film Language: A Semiotics of the Cinema. Trans. Michael Taylor. New York: Oxford
University Press, 1 9 74.

Meyer, Steven. Irresistible Dictation: Gertrude Stein and the Correlations of Writing and Science. Stan­
ford: Stanford University Press, 200 1 .

Milgram, Stanley. Obedience to Authority: A n Experimental View. New York: Harper and Row, 1 9 74.

Milnor, John. " On Manifolds Homeomorphic to the 7-Sphere." Annals of Mathematics, 2nd ser.,
64, no. 2 ( 1 956): 399-405.

Milnor, John Willard. Topology from the Differentiable Viewpoint. Rev. ed. Princeton: Princeton
University Press, 1 9 9 7 .

Miranda, Eduardo Reck, a n d Marcelo WanderJey. New Digital Musical Instruments: Control and
Interaction beyond the Keyboard. Middleton, Wisc . : A-R Editions, 2006.

Mitchell, William J. The Logic of Architecture: Design, Computation, and Cognition. Cambridge,
Mass.: MIT Press, 1 990.

Moghazy, Jehan. "Evaluating the Playfulness of Responsive Play Spaces." GVU Technical Report,
Georgia Institute of Technology, 2004.

Moholy-Nagy, Laszlo. "Neue Gestaltung in der Musik, M6glichkeiten des Gramophons. " Der
Sturm, no. 7 (1 923).

Morgan, Frank. Geometric Measure Theory: A Beginner's Guide. 4th ed. San Diego: Academic Press,
2009 .

Morrison, Philip and Phylis, and the Office of Charles and Ray Eames. Powers of Ten: A Book about
the Relative Size of Things in the Universe and the Effect of Adding Another Zero. Redding, Conn. :
Scientific American Library, 1 982.

Mouffe, Chantal, ed. Dimensions ofRadical Democracy: Pluralism, Citizenship, communltY. London:
Verso, 1992.

Mowery, David C., Richard Nelson, Bhaven Sampat, and Arvids Ziedonis. Ivory Tower and Indus­
trial Innovation: University-Industry Technology Transfer before and after the Bayh-Dole Act. Stanford:
Stanford Business Books, 2004.

Myler, Harley R., and Arthur R. Weeks. Computer Imaging Recipes in C. Englewood Cliffs, N.J . :
Prentice-Hall, 1993.

Nardi, Bonnie, ed. Context and Consciousness: Activity Theory and Human-Computer Interaction.
Cambridge, Mass. : MIT Press, 1996.
346 B i b liogra p hy

Needham, Tristan. Visual Complex Analysis. New York: Oxford University Press. 199 7 .

Negroponte, Nicholas. The Architecture Machine: Toward a More Human Environment. Cambridge,
Mass. : MIT Press, 1 9 70.

Newton-Smith, W. H. The Structure of Time. London: Routledge and Kegan Paul, 1 980.

Ngo, Tom, et al. "Accessible Animation and Customizable Graphics via Simplicial Configuration
Modeling. /I SIGGRAPH Conference Proceedings, ACM SIGGRAPH, 2000, 403-4 10.

Niblack, W., et al. "The QBIC Project: Querying Images by Content Using Color, Texture, and
Shape. /I Proc. Storage and Retrieval for Image and Video Databases, vol . I, 1 73-1 8 7 . Bellingham,
Wash.: SPIE, 1993.

Nirenberg, Ricardo L., and David Nirenberg. " Badiou's Number: A Critique of Mathematics as
Ontology./I Critical Inquiry 37 (20 1 1 ) : 5 83-6 1 3 .

Noether, Emmy. "Invariant Variation Problems./I Trans. M. A. Tavel. Transport Theory and Statisti­
cal Physics I, no. 3 ( 1 9 7 1 ) : 183-207.

Noether, Emmy. "Invariante Variationsprobleme. /I Nachr. d. Koenig. Gesellsch. d. Wiss. zu Goet­

tingen, Math-phys. Klasse ( 1 9 1 8) : 235-25 7 .

Ong, Walter J. Orality and Literacy: The Technologizing of the Word. London: Methuen, 1 982.

Ong, Walter ]. The Presence of the Word: Some Prolegomena for Cultural and Religious History. Min­
neapolis: University of Minnesota Press, 198 1 .

Ortony, Andrew, e d . Metaphor and Thought. 2nd ed. Cambridge: Cambridge University Press, 1993.

Ousterhout, J. K. "An X I I Toolkit Based on the TCL Language. /I Proceedings of the Winter 1991
USENIX Conference, 105-1 1 5 .

Paepcke, Andreas . "Viewing Heterogeneous Text Databases: Object Technology Works for the
View and Its Implementation. /I Technical Report HPL-92-85, Hewlett-Packard Laboratories, 1992.

Parker, S. G., D. M. Weinstein, and C. R. Johnson. "The SCIRun Computational Steering Software
System . /I In Modern Software Tools in Scientific Computing, ed. Erlend Arge, Are Magnus Bruaset,
and Hans Petter Langtangen, 1-40. Boston: Birkhauser, 1 9 9 7 .

Passerin d'Entreves, Maurizio. "Hannah Arendt. /I Stanford Encyclopedia ofPhilosophy, http://plato.

stanford. edu/entries/arendt/.

Pavis, Patrice. "Problems of a Semiology of Theatrical Gesture. /I Trans. Elena Biller-Lappin. Poetics
Today 2, no. 3 ( 1 9 8 1 ) : 65-93.

Peirce, Charles S. Essays in the Philosophy of Science. New York: Liberal Arts Press, 1 9 5 7 .

Peirce, Charles S., Morris Raphael Cohen, and John Dewey. Chance, Love, and Logic: Philosophical
Essays. Lincoln: University of Nebraska Press, 1 998.

Penderecki, Krzysztof. Threnos den Opfern von Hiroshima, fUr 52 Saiteninstrumente (Threnody to the
Victims of Hiroshima, for 52 Strings). London: E. Eulenburg, 1 9 6 1 .
Bibliography 347

Penrose, Roger. The Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics.
New York: Penguin, 1 99 1 .

Petitot, Jean. Morphogenesis o f Meaning. Bern: P. Lang, 2004.

Petitot, Jean. "Morphological Eidetics for Phenomenology of Perception. " In Naturalizing Phe­
nomenology: Issues in Contemporary Phenomenology and Cognitive Science, ed. Jean Petitot, Francisco
J. Varela, Bernard Pachoud, and Jean-Michel Roy, 330-3 7 1 . Stanford: Stanford University Press, 1999.

Petitot, Jean. Morphologie et esthi'tique: La forme et Ie sens chez Goethe, Lessing, Levi-Strauss, Kant,
Valery, Husserl, Eco, Proust, Stendhal. Paris: Maisonneuve et Larose, 2004.

Petitot, Jean. Physique du sens: De la theorie des singularites aux structures semio-narratives. Paris:
Editions du Centre national de la recherche scientifique, 1 992.

Petitot, Jean, et al., ed. Naturalizing Phenomenology: Issues in Contemporary Phenomenology and
Cognitive Science. Stanford: Stanford University Press, 1999.

Petrucci, Armando. Public Lettering: Script, Power, and Culture. Chicago: University of Chicago
Press, 1993.

Pickering, Andrew. The Mangle ofPractice: Time, Agency and Science. Chicago: University of Chicago
Press, 1 9 9 5 .

Pikovsky, Arkady, Michael Rosenblum, and J. Kurths. Synchronization: A Universal Concept in Non­
linear Sciences. Cambridge: Cambridge University Press, 200 1 .

Pinhanez, Claudio, and Aaron Bobick. " 'It/I' : A Theater Play Featuring a n Autonomous Computer
Graphics Character." Proceedings of the ACM Multimedia'98 Workshop on Technologies for
Interactive Movies, 1 998, 22-29.

Prigogine, lIya, and Isabelle Stengers. Order out of Chaos: Man's New Dialogue with Nature. New
York: Bantam Books, 1 984.

Pustejovsky, ]., and Bran Boguraev. "Lexical Knowledge Representation and Natural Language
Processing. " Artificial Intelligence 63 (1993): 193-223 .

Rao, R., et al. "The Information Grid: A Framework for Building Information Retrieval and
Retrieval-Centered Application s . " Proceedings of the ACM Symposium on User Interface Software
and Technology, 1992.

Rayner, Alice. To Act, to Do, to Perform: Drama and the Phenomenology of Action. Ann Arbor: Uni­
versity of Michigan Press, 1994.

Reddy, Michael J. "The Conduit Metaphor: A Case of Frame Conflict in Our Language about
Language. " In Metaphor and Thought, ed. Andrew Ortony. Cambridge: Cambridge University Press,
1 9 79 .

Rheingold, Howard. Virtual Reality. New York: Summit Books, 199 1 .

Richards, Thomas, and Jerzy Grotowski. At Work with Grotowski o n Physical Actions. London :
Routledge, 1995.
348 Bibl iography

Riemann, Bernhard. "Habilitationschrift: Ueber die Hypothesen, welche der Geometrie zu Grunde
liegen ( 1 854) . " In The Collected Works of Bernhard Riemann, ed. H. Weber. New York: Dover, 1 9 5 3 .

Riemann, Bernhard. " On the Hypotheses That Lie a t the Foundations o f Geometry" ( 1 8 5 4 ) . I n
Beyond Geometry: Classic Papers from Riemann to Einstein, e d . P. Pesic, 23-40. Mineola, N.Y. : Dover,
200 7.

Riemann, Bernhard. Gesammelte mathematische Werke und wissenschaftlicher Nachlass. Ed. Heinrich
Martin Weber. 2nd ed. Leipzig: B. G. Teubner, 1 892.

Rodowick, David N. Reading the Figural, or, Philosophy after the New Media. Durham: Duke Uni­
versity Press, 200 1 .

Rector, Alan, and Jeremy Rogers. " Ontological Issues i n Using a Description Logic t o Represent
Medical Concepts: Experience from GALEN . " IMIA Working Group 6 Workshop, 1999.

Rota, Gian-Carlo. " Kant and Husserl . " In Rota, Indiscrete Thoughts, ed. Fabrizio Palombi, 1 62-1 7 1 .
Boston: Birkhauser, 1 9 9 7 .

Rotman, Brian. " Corporeal o r Gesturo-Haptic Writing. " Configurations 1 0 , n o . 3 (2002) : 423-
438.

Rotman, Brian. Mathematics as Sign: Writing, Imagining, Counting. Stanford: Stanford University
Press, 2000.

Rotman, Brian. Taking God out of Mathematics and Putting the Body Back In: An Essay in Corporeal
Semiotics. Stanford: Stanford University Press, 1993.

Rotman, Brian. "Thinking Dia-Grams: Mathematics and Writing. " In The Sdence Studies Reader,
ed. Mario Biagioli, 430-44 1 . New York: Routledge, 1 999.

Rowe, Colin. "The Mathematics of the Ideal Villa. " In Rowe, The Mathematics of the Ideal Villa,
and Other Essays. Cambridge, Mass. : MIT Press, 1 9 76.

Rowe, Robert. Machine Musidanship. Cambridge, Mass. : MIT Press, 200 1 .

Royden, H. L . Real Analysis. 3rd ed. New York: Macmillan, 1 988.

Ryall, K., J. Marks, and S. Shieber. "An Interactive Constraint-Based System for Drawing Graphs. "
Proceedings o f the ACM Symposium o n User Interface Software and Technology, 1 9 9 7,
9 7-104.

Ryan, Joel. "Some Remarks on Musical Instrument Design at STEIM. " Contemporary Music Review
6, no. 1 ( 1 9 9 1 ) : 3-1 7.

Ryan, Joel, and Chris Salter. "TGarden: Wearable Instruments and Augmented Physicality. " Pro­
ceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal,
Canada.

Rydeheard, David E., and Rod M. Burstall. Computational Category Theory. New York: Prentice-Hall,
1988.
Bibliog raphy 349

Sacks, Sheldon. On Metaphor. Chicago: University of Chicago Press, 1 9 79 .

Said, Edward W. Th e Edward Said Reader. Ed. Moustafa Bayoumi and Andrew Rubin. New York:
Vintage, 2000.

Said, Edward W. Humanism and Democratic Criticism. New York: Columbia University Press, 2004.

Said, Edward W. Interviews with Edward W Said. Ed. Amritjit Singh and Bruce G. Johnson. Jackson:
University Press of Mississippi, 2004.

Said, Edward W. Orientalism. New York: Vintage, 1979.

Salter, Chris. "Performing in (Public) Space: Performative Architecture, Ubiquitious Computing

and Interaction. " Conference " Performing Spaces: Spatiality in Visual Culture, " Freie Universitat,
Berlin, May 2003 .

Saussure, Ferdinand de. Course in General Linguistics. Trans. and annotated Roy Harris. LaSalle,
Ill . : Open Court, 1983.

Schechner, Richard. Between Theater and Anthropology. Philadelphia: University of Pennsylvania

Press, 1985.

Schechner, Richard. The Future of Ritual: Writings on Culture and Performance. London: Routledge,
1993.

Schechner, Richard. Performance Studies: An Introduction. 2nd ed. New York: Routledge, 2006.

Schechner, Richard. Performance Theory. Rev. ed. New York: Routledge, 1 988.

Schwarz, Diemo, Gregory Beller, Bruno Verbrugghe, and Sam Britton. " Real-Time Corpus-Based
Concatenative Synthesis with Catart." DAFx (2006) .

Schwarz, Diemo, Roland Cahen, and Sam Britton. " Principles and Applications of Interactive
Corpus-Based Concatenative Synthesis." Joumees d'Informatique Musicale (2008) .

Schweizer, Jon-HeIgL Theoretische Grundlagen der Psychologie. 5 vols. Diessen: Nota Bene Verlag,
2004.

Scott, David W. Multivariate Density Estimation: Theory, Practice, and Visualization. New York: Wiley,
1992.

Selz, Peter, and Kristin Stiles, eds. Theories and Documents of Contemporary Art: A Sourcebook of
Artists ' Writings. Berkeley: University of California Press, 1996.

Sen, Amartya. Inequality Reexamined. New York: Russell Sage Foundation; Cambridge, Mass. :
Harvard University Press, 1992.

Serres, Michel. Genesis. Trans. Genevieve James and James Nielson. Ann Arbor: University of
Michigan Press, 1 9 9 5 .

Serres, Michel, and Bruno Latour. Conversations on Science, Culture, and Time. 1 990; Ann Arbor:
University of Michigan Press, 1 9 9 5 .
350 B i b liography

Sha Xin Wei. " Calligraphic Video: Using the Body's Intuition of Matter. " International Journal of
Creative Interfaces and Computer Graphics (2009) .

Sha Xin Wei. "Differential Geometrical Performance and Poiesis." Configurations 1 2, no. 1 (2004) :
1 3 3-1 60.

Sha Xin Wei. "Differential Geometric Performance and the Technologies of Writing." Ph.D. dis­
sertation, Stanford University, 2000.

Sha Xin Wei. " Ethico-Aesthetics in T* Performative Spaces." In On Transient Realities and Their
Generators, ed. Maja Kuzmanovic and Tim Boykett. Maribor, Slovenia: Kibla, 2005 .

Sha Xin Wei. " Ethico-Aesthetics in T* Performative Spaces. " In FoAM and Time's Up, Trg: On
Transient Realities and Their Generators, ed. Maja Kuzmanovic and Tim Boykett, 22-39 . Brussels:
Bezjak Tisk Maribor, 2006.

Sha Xin Wei. "Mediaweaver: A Media Authoring System for Networked Scholarly Workspaces."
Journal of Multimedia Tools and Applications 6, no. 2 ( 1 998) : 9 7-1 1 1 .

Sha Xin Wei. "Network-Distributed Media-Rich Detachable Simulations . " Technical report, Stan­
ford University, 1994.

Sha Xin Wei. "An Object-Oriented Multimedia Distributed Meta-Database . " ACM International
Conference on Multimedia, 1994.

Sha Xin Wei. "An O pen Testbed for Distributed Multimedia Databases . " Proceedings the AAAI
Workshop on Indexing and Reuse in Multimedia Systems, 1 994.

Sha Xin Wei. "Poetics of Performative Space. " Poetics Today (2005) .

Sha Xin Wei. "Proposal for an Implidt Surface Evolver. " Research proposal, Stanford University, 1994.

Sha Xin Wei. " Resistance Is Fertile : Gesture and Agency in the Field of Responsive Media . " Con­
figurations 1 0, no. 3 (2002): 439-4 72.

Sha Xin Wei. Review of Mathematica 2. American Mathematics Sodety Notices (May/June 1 992): 428.

Sha Xin Wei. "Scholarly Spaces." Technical report, Stanford University, 1 9 9 5 .

Sha Xin Wei. "Situating MMDD with Respect t o WWW, Portfolio, PREMO, Hyper-G, Script X . "
Technical report, Stanford University, 1 994.

Sha Xin Wei. "The TGarden Performance Research Project." Modern Drama 48 (2005): 585-608.

Sha Xin Wei. "Whitehead's Poetical Mathematics. " Configurations 13, no. 1 (2005): 7 7-94.

Sha Xin Wei, Michael Fortin, Navid Navab, and Timothy Sutton. "Ozone: A Continuous State­
Based Approach to Media Choreography. " ACM International Conference on Multimedia, Flor­
ence, 2010.

Sha Xin Wei, Giovanni Iachello, Steven Dow, Yoichiro Serita, Tazama St. Julien, and Julien Fistre.
"Continuous Sensing of Gesture for Control of Audio-Visual Media." Proceedings of the Seventh
International Symposium on Wearable Computers, 2003 .
B i b l i og raphy 351

Sha Xin Wei, Chris Salter, and Laura Farabough. "Sponge: Unstable Surfaces." ec/artS, issue 2 (2000).

Sha Xin Wei, Yoichiro Serita, Jill Fantauzza, Steven Dow, Giovanni Iachello, Vincent Fiano, Joey
Berzowska, Yvonne Caravia, Delphine Nain, Wolfgang Reitberger, and Julien Fistre. "Demonstra­
tions of Expressive Softwear and Ambient Media. " Adjunct Proceedings of Ubicomp 2003, Seattle,
1 3 1-136.

Sha Xin Wei, Robert Street, and Michael Keller. "NSF Proposal: A Multimedia Distributed Library. "
Technical report, Stanford University, 1 994.

Sha Xin Wei, Timothy Sutton, Jean-Sebastien Rousseau, Harry C. Smoak, Michael Fortin, and
Morgan Sutherland. "Ozone: A Continuous State-Based Approach to Media Choreography. "
Topological Media Lab, Concordia University, 2009 .

Sha Xin Wei, Yon Visell, and Blair Macintyre. " Choreographing Responsive Media Environments
Using Continuous State Dynamics within a Simplicial Complex." GVU Technical Report, Georgia
Institute of Technology, 2002.

Sha Xin Wei, Yon Visell, and Blair Macintyre. "Media: Choreography Using Dynamics on Sim­
plicial Complexes." GVU Technical Report, Georgia Institute of Technology, 2003 .

Shannon, Claude E. "A Mathematical Theory of Communication. " Bell System Technical Journal
27 ( 1 948) : 3 79-423, 623-656 .

Shapin, Steven, and Simon Schaffer. Leviathan a n d the A ir-Pump: Hobbes, Boyle, and the Experimen­
tal Life. Princeton: Princeton University Press, 1 9 8 5 .

Sharples, Mike, and Thea van d e r Geest, eds. The New Writing Environment: Writers at Work in a
World of Technology. Berlin: Springer, 1996.

Simondon, Gilbert. Du mode d'existence des objets techniques. Ed. augm. 1 958; Paris: Aubier, 1989.

Simondon, Gilbert. On the Mode of Existence of Technical Objects. Trans. Ninian Mellamphy.
London: University of Western Ontario, 1 980.

Simondon, Gilbert. "The Position of the Problem of Ontogenesis . " Trans. Gregory Flanders. Par­
rhesia, a Journal of Critical Philosophy 7 (2009): 4-1 6 .

Smith, Barry, and David Woodruff Smith, eds. The Cambridge Companion to Husser/. f ambridge:
Cambridge University Press, 1 9 9 5 .

Smith, David Woodruff. " Intentionality Naturalized?" In Naturalizing Phenomenology: Issues in

Contemporary Phenomenology and Cognitive Science, ed. Jean Petitot et aI., 83-1 10. Stanford: Stan­
ford University Press, 1 999.

Smith, David Woodruff. "Mind and Body. " In The Cambridge Companion to Husserl, ed. Barry
Smith and David Woodruff Smith, 323-393 . Cambridge: Cambridge University Press, 1995.

Smithies, Steve, Kevin Novins, and James Arvo. "A Handwriting-Based Equation Editor. "
Proceedings of the 1 999 Conference on Graphics Interface, 84-9 1 . San Francisco: Morgan Kaufmann,
1 999.
352 Bibliography

Sorkin, Michael, ed. Variations on a Theme Park: The New American City and the End of Public Space.
New York: Hill and Wang, 1 992.

Spanier, Edwin Henry. Algebraic Topology. New York: Springer, 1 9 8 1 .

Sparacino, Flavia, Glorianna Davenport, and Alexander Pentland. "Media i n Performance: Inter­
active Spaces for Dance, Theater, Circus, and Museum Exhibits. " IBM Systems Journal 39, nos. 3-4
(2000) .

Stacey, Ralph D. Complex Responsive Processes in Organizations: Learning and Knowledge Creation.
London: Routledge, 2001

Starn, Jos. "Stable Fluids. " SIGGRAPH 99 Conference Proceedings, ACM SIGGRAPH, 1999,
101-1 10.

Standage, Tom. The Mechanical Turk: The True Story of the Chess-Playing Machine That Fooled the
World. London: Allen Lane, 2002.

Stapp, Henry. Mind, Matter, and Quantum Mechanics. 3rd ed. Berlin: Springer, 2009 .

Star, Susan Leigh, and James Griesemer. "Institutional Ecology, 'Translations,' and Boundary
Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-1 9 3 9 . "
I n The Science Studies Reader, e d . Mario Biagioli, 505-524. London: Routledge, 1 9 9 9 .

Star, Susan Leigh, and Karen Ruhleder. " Steps toward a n Ecology o f Infrastructure: Borderlands
of Design and Access for Large Information Spaces." Information Systems Research 7 ( 1 99 6) :
1 1 1-134.

Steadman, J. P. Architectural Morphology: An Introduction to the Geometry ofBuilding Plans. London:

Pion, 1 983.

Stengers, Isabelle. "Beyond Conversation: The Risks of Peace." In Process and Difference: Between
Cosmological and Poststructuralist Postmodernisms, ed. Catherine Keller and Anne Daniell, 235-255 .
Albany: SUNY Press, 2002.

Stengers, Isabelle. "A Constructivist Reading of Whitehead's Process and Reality. " Theory Culture
Society 25, no. 4 (2008) : 9 1 - 1 1 0 .

Stengers, Isabelle. Cosmopolitics. Trans Robert Bononno. 2 vols. Minneapolis: University o f Min­
nesota Press, 201 0-20 1 1 .

Stengers, Isabelle. Cosmopolitiques. 2 vols. 1997; Paris: La Decouverte, 2003.

Stengers, Isabelle. Penser avec Whitehead: Une libre et sauvage creation de concepts. Paris: Seuil, 2002.

Sutherland, Ivan E. "Sketchpad: A Man-Machine Graphical Communication System. " Ph.D. dis­
sertation, Massachusetts Institute of Technology, 1 9 6 3 .

Sutton-Smith, Brian. The Ambiguity of Play. Cambridge: Harvard University Press, 200 1 .

Thaller, Manfred. "What I s 'Source Oriented Data Processing'; What Is a 'Historical Computer
Science' ?" In Historical Informatics: An Essential Tool for Historians?, ed. Jan Oldervoll. Association
for History and Computing, 1994.
Bibliography 353

Thorn, Rene. Mathematical Models of Morphogenesis. Chichester, U.K.: Ellis Horwood, 1983.

Thorn, Rene. Semio Physics: A Sketch. Redwood City, Calif. : Addison-Wesley, 1990.

Thorn, Rene. Structural Stability and Morphogenesis: An Outline of a General Theory of Models.
Reading, Mass. : Addison-Wesley, 1989.

Thomas, Dylan. Selected Poems, 1 934-1 952. New York: New Directions Books, 2003.

Thurston, William P. "On Proof and Progress in Mathematics . " Bulletin of the American Mathemati­
cal Society, n.s. 16 (1 994): 1 6 1-1 7 7 .

Titchmarsh, Edward C. The Theory of Functions. 2 d ed. London : Oxford University Press,
1939.

Tobagi, Fouad A., and J. Pang. " Starworks-a Video Applications Server. " In Digest of Papers,
Compean Spring '93. Los Alamitos, Calif. : IEEE Computer Society Press, 1 9 9 3 .

Tomlinson, Gary. "Music in Renaissance Magic. " Topological Media Lab, Research Atelier at
Concordia University, http://topologicalmedialab.net/.

Trehan, Raj iv, Nobuyuki Sawashima, Koji Yamaguchi, and Koichi Hasebe. "Toolkit for Shared
Hypermedia on a Distributed Object Oriented Architecture. " Proceedings of the ACM Interna­
tional Conference on Multimedia, 1993, 1 75-182.

Tuan, Yi-Fu. Passing Strange and Wonderful: Aesthetics, Nature and Culture. Washington, D.C.: Island
Press, 1993.

Valle, Andrea. La notazione musicale contemporanea: Aspetti semiotici. Torino: EDT, 2002.

Van Nort, Doug. "Instrumental Listening: Sonic Gesture as Design Principle . " Organised Sound
1 4, no. 2 (2009): 1 7 7-187.

Varela, Francisco J., Evan Thompson, and Eleanor Rosch. The Embodied Mind: Cognitive Science
and Human Experience. Cambridge, Mass. : MIT Press, 1 99 1 .

Vatter, Miguel. "Natality and Biopolitics i n Hannah Arendt . " Revista de Ciencia Po/{tica 26, no. 2
(2006) : 1 3 7-159.

Viola, Bill. Selected Works. VHS videocassettte. Los Angeles: Voyager Press, 1987.

Wachsmuth, Ipke, and Timo Sowa, eds. Gesture and Sign Language in Human-Computer.Interaction.
Intemational Gesture Workshop, GW 2001, London, UK, April 2001, Revised Papers. Berlin: Springer,
2002.

Wanderley, Marcelo M . , and Philippe Depalle. "Gestural Control of Sound Synthesis. " Proceedings
of the IEEE 92, no. 4 (2004) : 632-644.

Wang, James Ze, Gio Wiederhold, Oscar Firschein, and Sha Xin Wei. "Wavelet-Based Image Index­
ing Techniques with Partial Sketch Retrieval Capability. " Proceedings of the IEEE Forum on
Research and Technology Advances in Digital Libraries, 1997, 1 3-24.
354 B i b liography

Weber, Max. The Protestant Ethic and the Spirit of Capitalism and Other Writings on the Rise of the
West. 4th ed. New York: Oxford University Press, 2009.

Wegenstein, Bernadette. Getting under the Skin: Body and Media Theory. Cambridge, Mass.: MIT
Press, 2005 .

Weil, Simone. First and Last Notebooks. London: Oxford University Press, 1 9 70.

Weil, Simone. Formative Writings, 1 929- 1 94 1 . Ed. and trans. Dorothy Tuck McFarland and Wil­
helmina Van Ness . Amherst: University of Massachusetts Press, 1987.

Weil, Simone. Gravity and Grace. London: Routledge, 2002.

Weil, Simone. The Need for Roots: Prelude to a Declaration of Duties Towards Mankind. London:
Routledge, 2002.

Weil, Simone. The Notebooks of Simone Weil. London: Routledge, 2004.

Weil, Simone. Oppression and Liberty. 1958; Amherst: University of Massachusetts Press, 1 9 73 .

Weitzman, Louis, and Kent Wittenburg. II Automatic Presentation o f Multimedia Documents

Using Relational Grammars. " Proceedings of the ACM International Conference on Multimedia,
1 994, 443-45 1 .

Wente, Henry C . "Counterexample t o a Conjecture o f H . Hopf. " Pacific Journal of Mathematics

1 2 1 ( 1 ) : 193-243.

Weyl, Hermann. The Continuumv: A Critical Examination of the Foundation of Analysis. Mineola,
N.Y. : Dover, 1 994.

Weyl, Hermann. Space-Time-Matter. Mineola, N.Y. : Dover, 1 952.

Wheeler, John Archibald. Geometrodynamics. New York: Academic Press, 1962.

Wheeler, John Archibald. "Geons. " Reviews of Modern Physics 9 7 (1955): 5 1 1-5 3 6 .

Whitehead, Alfred North. The Concept of Nature (The Tamer Lectures Delivered in Trinity College,
November 1 9 1 9). Mineola, N.Y. : Dover, 2004.

Whitehead, Alfred North. The Principle ofRelativity with Applications to Physical Science. Cambridge:
Cambridge University Press, 1 922. Rpt., Mineola, N.Y. : Dover, 2004.

Whitehead, Alfred North. Process and Reality: An Essay in Cosmology. Corrected ed. Ed. David Ray
Griffin and Donald W. Sherburne. New York: Free Press, 1 9 78.

Whyte, William H. The Social Life of Small Urban Spaces. Washington, D.C.: Conservation Founda­
tion, 1 9 79.

Wilden, Anthony. Man and Woman, War and Peace: The Strategist's Companion. London: Routledge
and Kegan Paul, 1 987.

Wilden, Anthony. The Rules Are No Game: The Strategy of Communication. London: Routledge and
Kegan Paul, 1987.
B i b l iography 355

Wilden, Anthony. System and Structure: Essays in Communication and Exchange. London: Tavistock,
1 9 72.

Williams, James. Lyotard and the Political. London: Routledge, 2000.

Williams, James. The Transversal Thought of Gilles Deleuze: Encounters and Influences. Manchester:
Clinamen Press, 2005 .

Winograd, Terry. "Towards a Human-Centered Interaction Architecture. " Working paper, Stanford
University.

Wise, M. Norton. "Architectures for Steam . " In The Architecture of Science, ed. Peter Galison and
Emily Thompson, 107-140. Cambridge, Mass.: MIT Press, 1 999.

Wittgenstein, LudWig. Lectures on the Foundations of Mathematics, Cambridge, 1 939. Ed. Cora
Diamond. Ithaca: Cornell University Press, 1 9 76.

Wittgenstein, Ludwig. Philosophical Investigations. Trans. G. E. M. Anscombe. New York: Macmil­

lan, 1 9 5 8 .

Wittgenstein, Ludwig. Tractatus Logico-Philosophicus. London: Routledge, 1 994.

Wolfflin, Heinrich. Principles of Art History: The Problem of the Development of Style in Later Art.
Trans. M. D. Hottinger. 1 9 1 5 ; New York: Dover, 1950.

Wolford, Lisa. Grotowski's Objective Drama Research. Jackson: University Press of Mississippi, 1996.

Wolfram, Stephen. A New Kind of Science. Champaign, Ill . : Wolfram Media, 2002.

Wood, Gaby. Edison 's Eve. New York: Knopf, 2002.

Woodland, P. C., and S. J. Young. "The HTK Tied-State Continuous Speech Recogniser. " In Pro­
ceedings of Eurospeech '93, Berlin, 1993, 2207-22 1 0 .

Xenakis, lannis. " Determinacy and Indeterminacy. " Organised Sound 1 ( 1 996): 143-1 5 5 .

Yang, Yiming, and Jan O. Pedersen. "A Comparative Study o n Feature Selection i n Text Catego­
rization. " Proceedings of the Fourteenth International Conference on Machine Learning, ed. Douglas
H. Fisher, 4 1 2-420. San Francisco: Morgan Kaufmann, 1997.

Zeleznik, Robert C., Kenneth P. Herndon, and John F. Hughes. "Sketch: An Interface for Gestural
Modeling. " SIGGRAPH Conference Proceedings, ACM SIGGRAPH, 1 996, 1 63-1 70.

Zevi, Bruno. The Modem Language of Architecture. Seattle: University of Washington Press, 1 9 78 .

Zhuangzi. Zhuangzi: Basic Writings. Trans. Burton Watson. New York: Columbia University Press,
2003.

Zinn, Howard. You Can't Be Neutral on a Moving Train: A Personal History of Our Times. Boston:
Beacon Press, 1994.
I ndex

Abrioux, Yves, 52 Audio. See Sound

Accountability, 229, 244-245 Authorship, 246-247
Alexander, Christopher, 32, 69, 93, 1 25-126, Autopoiesis, 77, 89, 1 24, 2 1 5 , 23 1
1 48-1 52, 1 75
Alexander, J. w., 1 5 2- 1 5 3 Badiou, Alain, 69, 1 6 1
Algebra, 64-65, 97, 1 02, 183-184 Banff New Media Institute, 20
Allegory, 1 6 1 , 267, 270 Barad, Karen, 59-60
Allographic art object, 40, 43 Barba, Eugenio, 29 1
Andrei, Linde, 1 0 Barry, Andrew, 240
Andrew File System, 1 1 Bergson, Henri, 124
Anthropocentrism, 9 , 103, 108, 1 9 7-199 Bilgrami, Akeel, 7-8, 91, 128
Anthropology, 72, 1 8 1 Bohm, David, 4, 1 42
Anti-psychiatry, 1 5 7- 1 5 8 Born, Georgina, 240
Apprenticing, 240 Boulez, Pierre, 192
Arakawa, Shusaku, 234 Brindle, Reginald Smith, 44
Archimedes, 261, 267 Brook, Peter, 70, 7 1 , 228
Arendt, Hannah, 107-108 Brouwer, L. E. J., 140
Ars Electronica Festival, 73
Art, 3-4, 63, 1 10, 1 5 7, 205-207, 236-238, Cabrera, BIas, 10
254-255, 288 Cage, John, 29-30, 47, 80, 1 5 6
dance as, 29-34 Calculus, 1 34-1 35, 1 43, 1 45-1 48, 152
evaluation of, 228 Calligraphic video, 1 1 6-1 1 7, 1 59, 214-2 1 6,
mathematics as, 1 7 23 1-232
objects, 29, 40-44, 58 Care, 7, 60-62, 87, 94, 103, 1 06-107, 1 5 0
performative, 63, 1 1 9 medical, 100, 229
a s profession, 238-239 Category theory, 1 1 1-1 1 2, 307
research, 73, 236-24 7 Changeux, Jean-Pierre, 121
Artaud, Antonin, 4 5 , 7 1 , 229 Chaos, 286-288
Atelier-lab, 235-24 7 . See also Topological Chatelet, Gilles, 1 6 3
Media Lab Chromaticization, 1 0 9
Atomism, 94-9 7 Cirone, Anthony J., 2 4
358 I ndex

Civil rights, 106 Discrete sets, 94-9 7, 1 4 1 , 183

Coenen, Olivier j.-M. D., 199 Doane, Mary Ann, 1 24
Cohen, Paul, 1 69 Drugs, 1 70
Combinatorial complexity, 94-9 7, 302 Dynamical system, 139, 1 62, 1 6 7 , 2 1 6
Computational equivalence principle, 1 6,
1 32-133, 1 62 Einstein, Albert, 1 32-1 34, 2 72-274, 282
Computational media, 8, 2 1 , 34-36, 7 1 , 74, 90 Electromagnetism, 105, 274
and live performance, 48-49, 83 El-Mallah, Issam, 26, 4 1-43, 45, 48
motivation for, 90, 1 1 8-1 1 9, 230 Emergence, 1 49-1 50
music, 35, 48-49 Enchantment, 93, 2 7 1
Computational software, 1 0, 5 1 , 65 Eternal objects, 1 9 1-192
Computational technology, 10, 15, 19, 21-22, Ethico-aesthetics, 60-62, 120, 125, 293
48-5 1 , 298 ethico-aesthetic play, 158, 229, 25 1 , 259,
attitudes toward, 1 2- 1 3 266-267
dominance of, 5 9 Ethics, 9, 93, 94, 96, 99-100, 106, 1 28, 243
Computer graphics, 65, 1 1 6, 1 1 8, 299 ethical subject, 1 03, 105
Computer vision, 1 9 7-202, 2 1 4 institutional review, 229
Concordia University, 255 Euler characteristic, 1 42-143
Concrescence, 89, 1 30-1 33, 1 39, 200 Euler-Lagrange equations, 146
Connes, Alain, 1 2 1 , 202 Event, 20-2 1 , 56, 69-72, 1 28, 131, 1 5 9-1 60
Constellation, 5 7-58 explanation of term, 21
Continuity, 94, 1 1 7, 123, 1 39-140, 1 76, 1 9 7, metaevents, 62, 207
266, 284 musical, 23, 26-29
discontinuity, 1 5 1 , 278 as obj ect, 40
Convexity, 1 3 7, 224 performative, 2 1 , 45, 63-88
Cosmicomics, 83, 2 1 4 and responsive environments, 1 68
Cosmology, 1 0 1 , 1 6 1 , 1 8 7, 204 substrate of, 258
Counterfeitability, 40 text of, 43, 45
Crumb, George, 4 7 Evolution, 1 5 0-1 5 1
Cunningham, Merce, 3 4
Farabough, Laura, 73
Dance, 29-34, 5 6-5 7, 1 20, 130 Fiber bundles, 186
notation, 29-34, 3 6-3 7, 39, 3 1 4 Fire, 101-104
Dasein, 1 0 7, 1 2 1 , 126 Flow, 35, 1 00, 1 65-1 6 7, 1 8 7
Deleuze, Gilles, 99, 1 09, 1 24, 1 52, 192- 1 9 7, FoAM, 7 3 , 1 6 1 , 227, 302
2 78, 286-287 Foucault, Michel, 2, 6, 148, 2 8 1
d'Entreves, Maurizio, 107 Frankenstein's Ghosts, 83, 204
Derrida, Jacques, 4, 79, 2 7 7-2 78, 290 Freedom, 6, 97, 107-109
Desargues, Gerard, 287 Funding, 10, 238-239, 245-24 7, 256
Descartes, Rene, 1 30, 2 74-2 75
Differential geometry, 1 3-14, 64-6 7, 105-106, Gauss, C. F. , 143
1 34, 1 39, 1 42, 1 84-1 8 7, 200, 279 Gauss curvature, 142-143
Directing, 240 Geertz, Clifford, 72
I ndex 359

Genette, Gerard, 29, 39-40, 43-45, 52, 58 Heraclitus, 1 0 1 - 1 04, 1 40

Geodesics, 1 32, 1 39, 1 42, 1 88, 279 Hexagram, 245-246, 25 5-256
Geometers Workbench, 1 3-15, 64-68 Hirschhorn, Thomas, 6
Geometry, 32, 1 0 1 , 105, 1 3 3-1 34, 1 8 7, 1 92, Homotopy, 1 2 7
273-277, 281-290 Hopf, Heinz, 1 40
differential, 1 3-14, 64-67, 1 05-106, 1 34, Hubbub, 1 4-1 5 , 6 7-68, 99
1 39, 1 42, 1 63-167, 1 84-1 8 7, 200, 279 Husserl, Edmund, 1 25, 200-202, 267
Euclidean, 1 00, 133, 1 3 6-138, 2 7 5 Hutchinson, Ann, 3 3
notation, 32
Riemannian, 1 63, 195, 276, 284 IL Y A , 8 3 , 85-8 7
Gestural instruments, 210-2 1 1 , 2 1 7 Improvisation, 3 9-40, 61, 63-64, 70-7 1 , 89,
Gestural sound, 1 1 7, 230, 233-234 1 39, 262
Gesture, 60, 66-67, 70, 73, 8 1 -82, 1 1 9, 2 1 1 , collective, 1 9 1 , 203, 25 7
2 1 5 , 257, 2 5 8 dance, 1 20
collective, 83, 265 ethico"aesthetic, 105, 235
ethico-aesthetic, 3 gesture, 60, 62, 73-77, 82-83, 1 20, 203,
improvised, 60, 62, 73-7 7, 82-83, 1 20, 203, 2 1 0-2 1 1 , 256-257, 262-263
2 1 0-2 1 1 , 256-25 7, 262-263 musical, 2 1 , 44
and intentionality, 83 Intentionality, 79-83
mapping of, 14, 209 Interface, 12, 53, 208, 246
meta-, 64 clothing as, 210-2 1 1
and responsive media, 5 5-56, 74, 7 7-80, graphical, 1 0, 1 2, 20, 4 7
1 1 7, 2 1 3 , 229, 232, 253 keyboard-mouse, 5 5 , 65
Gins, Madeline, 234 video, 2 1 5
God, 8, 91, 101, 141, 254 WIMP, 5 3
Goodman, Nelson, 40, 43 Interstitial, 5 6-5 7
Grace, 3 Intersubj ectivity, 9, 94-9 7
Gramophone, 1 9 Invariants, 1 43, 1 90
Graphs, 5, 6 6 , 1 64
as musical notation, 24, 29 James, William, 266
Grotowski, Jerzy, 45, 70-7 1 , 1 5 3 , 23 1 , 29 1
Groups (mathematics), 1 83-184 Kagel, MauriciO, 23-24
Lie, 1 20- 1 2 1 , 1 62, 1 89, 200 Kahn, Charles, 1 02, 104
Guattari, Felix, 3, 99, 109, 1 5 7- 15 8, 192-197, Kant, Immanuel, 1 08, 275, 29 1
235, 259, 286-28 7 Kempelen, Wolfgang von, 25 1
Guimbretiere, Fran�ois, 1 4 Kierkegaard, S0ren, 7 1
Kinetography. See Notation: dance
Halpern, Orit, 1 24
Hawking, Stephen, xix Laban, Rudolf, 32-33, 39
Heidegger, Martin, 58, 106, 1 24-125, 1 29, Lakoff, George, 1 76
266, 2 7 7 Language, 3 7-39, 59, 72, 99, 1 39, 158, 1 6 1 ,
Heidenreich, Achim, 5 2 266-26 7
Hendricks, Barbara, 228 programming, 68-69, 94-9 5 , 1 10-1 1 1
360 I ndex

Laozi, 9 7 Measure theory, IS, 65, 1 2 7-128, 1 38-143,

Latour, Bruno, 5 8 , 6 1 , 273 I SO, 1 54, 1 9 7, 232
Lattice theory, 138 Mechanical Turk, 25 1-252, 254
Leibniz, Gottfried Wilhelm, 1 32, 1 4 1-143, Media choreography systems, 80, 85,
1 76 203-207, 208-2 1 7
Levine, Myron, 26 MediaWeaver, 1 1-12
Lie, Sophus, 189 Meeks, William, 1 40
Lie groups, 1 20- 1 2 1 , 1 62, 189, 200 Membranes, 85-8 7
Ligeti, Gyorgy, 25-2 7 Metaevents, 2 9 , 62, 20 7
Limits, 7 1 , 1 3 7- 1 4 1 , 1 72, 1 79, 192 Metaphor, 23, 1 76-1 77, 203, 205, 2 1 7-220,
human, 48 253
Metaphysics, 7, 1 6, 89, 91, 93, 1 3 1, 1 3 5 ,
Mach, Ernst, 1 44 156
Magic Blackboard, 1 3-14 Meteor Shower, 8 3
Manifolds, 104-105, 1 34, 1 8 7-1 89, 280-28 1 Methodology, 4, 228-229, 269
differentiable/Riemannian, 127, 1 39, Microphenomenology, 23 1
1 84-1 87, 2 7 7-283 Milnor, John, 1 40
equations on, 1 8 7-199 Minimax design, 23 1 , 236
and topology, 1 64-1 68, 1 75, 1 84-1 8 7, 278 Miro, Joan, 1 1 4
Mapping, 1 75-1 7 7 Moholy-Nagy, Uiszlo, 1 9
o f gesture, 1 4 , 209 Monge, Gaspar, 287
Material dynamics, 72, 90, 104, 125, 1 28, 2 1 6 Montanaro, Michael, 83
Materiality, 5 9 , 90, 1 94, 263, 2 72-2 73, 283 Morphisms, 1 1 1-1 1 2
Mathematics, I, 13, 1 5- 1 7 , 64-6 7, 128, 1 83, Morris, David, 247
249, 28 7-288, 29 1 . See also Algebra; Music, 22-23, 26, 35, 40-44, 48, 98-99
Calculus; Geometry notation, 22-29, 35-36, 40-42, 44, 46-48
notation, xviii-xix, 43
as poetic material, 1 6 1- 1 64 Nadal, Jean-Pierre, 1 9 9
Matter, 7-8, 59, 7 1 , 90-9 1 , 101, 1 1 8, 128, Natality, 107-109
1 76, 1 90, 258, 2 7 1-275, 282, 286 Navab, Navid, 234, 263
alchemical, 230-23 1 Newton, Isaac, 272-274
animate, 76, 89, 203 Noether, Emmy, 1 03
articulation of, 8 7, 185, 20 1 , 229, 250 Noise, 1 5 4-1 5 7
continuous, S , 55, 1 92, 1 9 7, 215, 250 Notation, 19, 21-22, 40, 43, 46
dumb, 206 dance, 29-34, 36-3 7, 39, 3 1 4
dynamical, 63, 90-9 1 , 1 04, 1 1 8 geometriC, 3 2
and energy, 1 32-1 3 5 kinesics, 23-24
flow, 35, 1 00, 1 65-1 6 7 linguistic, 39
inert, 253 mathematical, xviii-xix, 43
qUickened, 263-266 metaevents, 29
and value, 93, 125, 1 28, 1 48 music, 22-29, 35-36, 40-42, 44, 46-48
Maturana, Humberto, 7 7, 1 24-125, 1 80, 1 9 1 , software, 54
215 Nuance, 46, 82, 94, 1 09, 1 80, 233
I ndex 361

OMax, 52-54 Philipona, David, 199

Ontogenesis, 1 23-1 60, 1 9 7 , 2 1 5 , 235, Philosophy, 1 , 4, 8, 96, 1 29, 1 54, 1 64, 237,
26 1-263 249-250, 273, 278
cultural, 1 80-1 82 as art, 4-6
explanation of, 1 24 experiment, 255
Open sets, 1 00, 1 38, 1 69-1 7 1 , 1 8 1 informed by movement, 229-230
O'Regan, Kevin, 1 9 9 of mathematics, 66, 288-29 1
Ouija, 83, 263 of mind, 80
Oulipo, 80 of process, 93, 1 0 1 , 1 26, 1 28-1 29, 1 39-1 40,
Ozone, 80, 1 36, 203-209, 2 1 6, 23 1 , 256 272, 274
Phonograph, 19
Painting, 97, 1 1 4-1 1 5 Planck's constant, 1 04
Palindrome, 3 4 Play, 1 58, 229, 25 1 , 259, 266-267
Palpation, 99-100, 107 Plenum, 3, 9, 90-94, 97, 1 00, 1 06, 141, 1 62
Penderecki, Krzysztof, 26, 28 dynamical, 1 0 1 , 266, 2 7 1-273, 278
Penrose, Roger, xix explanation of, 305
Performance, 1, 40, 44-45, 68-69, 78, 80, and process, 1 39-1 40
1 1 9, 249 Poetry, 3, 50, 266-26 7
and computational technology, 48-5 1 mathematics a s , 1 6 1
as event, 69-72, 1 5 8 Poiesis, 6 , 67, 9 0 , 9 3-94, 1 92, 269, 273
explanation o f term, 63-64 explanation of, 1 09, 25 1
history of practice, 46-48 and freedom, 1 0 7-109
immanent text, 44 mathematical, 1 6 1-1 64, 1 68, 275, 280,
mathematical, 1 3 , 1 7 28 7-288
musical, 22-23, 26, 40-44, 48 Poincare, Henri, 1 26, 1 39, 285, 305-306
nonclassical theory of, 44-45 Politics, 7, 61, 1 06-107, 1 08
notation as, 46 Principle of least action, 1 26, 1 3 5-136,
phenomenology of, 73 1 40-1 4 1 , 147, 200
representation of, 4 alternatives to, 1 49-1 5 0
space, 73-77, 82 Principle o f unfolding wholeness, 1 5 1-154
substrate of, 8 Proof (mathematics), xix, 19, 1 29, 1 36, 163
technology of, 1 9-63, 68, 82 Proximity, 1 00, 1 64, 1 79, 1 8 1 , 250
variability of, 48 Psychiatry, 1 5 7- 1 5 8
Performance art, 6 3 Psychoanalysis, 87, 1 24, 1 5 7, 259
Performativity, 36, 5 8-60 Pythagoras, 101
Personal computing, 7, 10- 1 3
Petitot, Jean, 1 2 1 , 1 66, 1 8 5 , 1 9 7-202, Quantum mechanics, 58, 71, 1 04, 1 42, 1 54,
289-290 1 5 7, 273, 3 1 1
Phenomenology, 1, 1 5 , 66, 1 2 1 , 1 30, 139, Quebec (province), 238, 256
1 50, 1 82, 1 90, 201-202
experiments, 70, 7 3-74, 76- 7 7 , 83, 199, Rationality, 7-8, 106, 1 20, 181, 229
228-229, 249 Relativity theory, 1 28, 1 32, 1 3 3-1 34,
of performance, 7 3 , 261-263 2 7 2-274, 282
362
Religion, 89
Remedios Terrarium, 83, 2 1 2
Representationalism, 36-3 7, 5 8-59
Research creation, 238, 244
Responsive environment, 63-8 7, 136, 1 68,
256-259, 263. See also individual
installations by name
design and construction of, 1 1 7, 204-207,
22 1 , 253
explanation of, 56, 58
Responsive media, 2, 19, 54-58, 73-84
Rhythm, 42, 233-234
Riemann, Bernhard, 1 39, 2 72, 2 7 7-279
Riverbed, 34
Robots, 1 99-20 1
Rotman, Brian, 42
Ryan, Joel, 78
Salter, Chris, 73, 78
San Francisco Symphony Orchestra, 24
Score. See Notation: music
Sensors, 55, 7 1 , 78, 221-226, 262
Serita, Yoichiro, W I , 232
Serres, Michel, 125-127, 1 5 4-1 5 7, 250
Set theory, 128, I S S , 1 68-1 74, 250
Shannon, Claude, 1 5 6
Simondon, Gilbert, 3 5 , 4 5 , I S S , 1 79, 263
Simulation, 1 0-1 1 , IS, 5 1, 1 1 8, 23 1
inefficiency of, 23 1
numerical, 1 3 , 49-50, 232
physics, 1 0-1 1 , 48, 76, 1 1 8, 207
social, 1 0-1 1
virtual reality, I S , 84, 23 1
Singularity, 148
SmallTalk, 11 0-1 1 1
Smith, B . c., 1 64
Smooth space, 192-1 9 7
Software, 1 2-13, 2 9 , 3 9 , 49-52, 209
graphics, 1 1 2-1 1 6
mathematical, 6 5
programming, 1 1 0-1 1 2
responsive media, 5 5 , 203-207
Sony Computer Science Lab, 199-20 1
I ndex
Sound, 7 7-78, 2 1 2-2 1 4
audio signal analysis, 2 1 2-2 1 3
gestural, 1 1 7, 230, 233-234
responsive, 80
as substrate, 99
triggered, 233
Spectator, 20, 45, 48, 50, 70
in relation to actor, 45, 70, 1 04, 1 5 3, 207,
227
Speech, 1 4-15, 60, 6 7-68, 72, 79, 99, 109. See
also Language
as substrate, 99
Speech recognition, 1 4-15, 6 7-68, 99, 254
Spinoza, Baruch, 1 29, 1 39, 141
Sponge (art group), 6, 60, 73, 80, 84, 227
Stanford University, xviii, 10, 23, 73, 250
Star, Susan Leigh, 1 5 8
Stein, Julian, 263
Stengers, Isabelle, 1 6- 1 7 , 132
Striated space, 192- 1 9 7
Subjectivation, 3, 9, 1 1 9, 1 5 7- 1 5 8, 25 7
Subjectivity, 9, 1 2 1 , 1 26, 1 82, 202, 229
intersubjectivity, 9, 94-9 7
subjectification, 158
Substrate, 8-9, 29, 9 7-99, 1 20, 1 2 1 , 141,
258
Surveillance, I S , 6 7
Sutton, Tim, 263
Tanaka, Yutaka, 134
Temporality, 90, 125-126, 1 90-1 9 1 , 2 1 7,
266-26 7, 273, 276-2 7 7
Texture, 108- 1 1 0
TGarden, 2 , 20, 60-6 1 , 73-84, 1 1 8, 2 5 6
Theater, 70- 7 1 , 1 5 3 , 228-229
Thickness, explanation of, 72
Thorn, Rene, 129, 148, l S I , 1 66, 185, 193,
197
Thomas, Dylan, 266
Time, 124-127, 1 3 1-132, 190- 1 9 1 , 2 1 3 ,
276-2 7 7
Topological dynamics, 1 26-1 30, 1 3 9-1 40,
1 5 9-1 60, 1 80-182, 208, 227, 266, 269
I ndex 363

Topological media, 82, 258, 287 Viola, Bill, 254

explanation of, 5-7, 1 62, 250, 2 73-274 Virtual reality, 15, 84, 23 1
genealogy of, viii, 2, 250
Topological Media Lab, 1 , 6, 7 7 , 1 19, Weber, Max, 8
227-236, 253, 2 5 7 Wechsler, Robert, 34
founding, 14, 2 2 7 , 25 1 Weibel, Peter, 61
funding, 25 1 , 322 Wei!, Simone, 3
institutional setting, 255 Weinstone, Ann, 25 1
motivations of, 59, 1 1 7 Whitehead, A. N., 1 6-1 7, 9 1 , 1 26-1 4 1 , 1 8 1 ,
operational structure, 235, 239-242 1 90-1 92, 274
Topology, 1 00, 1 32, 1 3 7- 1 4 1 , 1 5 5 , 1 62-1 68, Winograd, Terry, 1 3 , 229
1 82, 192, 250, 259, 27 1-275, 280-286 Wittgenstein, Ludwig, 20, 3 7-39, 42, 1 5 7
axioms of, 1 7 1 Wolfram, Stephen, 1 6, 6 5 , 1 62
explanation of, 272 Writing, 1 4-15, 66-67, 240
pOint-free, 139 as performance, 64
point-set, 1 68-1 74, 261 software, 68-69
state, 208, 213, 2 1 6-222, 261
Trajectory, 1 33-134, 166- 1 6 7 Xenophanes, 1 0 1
Transcendentalism, 7-8, 63, 1 4 1
Transcription, 1 5 , 4 1 , 6 7 Zhuangzi, 8 9
Twombly, Cy, 1 1 5

Unlearning, 240

Valle, Andrea, 46-48

Varela, FranciSCO, 7 7 , 1 24-125, 1 80, 1 9 1 ,
215
Varley, Julia, 29 1
Vatter, Miguel, 108
Vector, 1 35, 1 40
fields, 92, 105-106, 1 8 7
spaces, 1 74, 1 9 3
Vectoriality, 1 0 5 , 1 30-1 33
Video, 34-36, 50, 78, 1 1 2-1 1 7, 254
analysis, 2 1 2
calligraphic, 1 1 6-1 1 7, 1 59, 2 1 4-2 1 6,
23 1-232
computational, 1 5 8-159
and live performance, 20
in live performance, 5 7 , 74, 7 6
real-time, 5 1 , 5 6
responsive, 78-79, 85-8 7
synthesis, 5 2-53, 2 1 4-21 6