for this, we propose that computationally expensive visual
objects support a “fast rendering mode” that trades image
quality for run-time overhead. More work is needed on
this problem.
If interaction is designed into the performance, clearly
only that portion of the performance that does not rely on
interaction can be rendered off-line. Our system allows
for this circumstance by rendering the static portion of the
performance, and allowing for simultaneous playback of
the “recorded” performance and arbitrary additional
visual material in real-time.
6. Relationship to other efforts
Instruments for playing graphics with music can be
grouped into two categories. The first consists of what
might be termed image sequencers. These programs
typically play back sequences of images that have been
created elsewhere, sometimes providing transitional and
other effects. Xpose and Director are examples of such
programs.
The second category is that of image generators. These
programs permit the player to create some kind of
graphic, sometimes specifically to accompany music.
Bomb, GEM, MusiKalscope and BlissPaint are examples
of this type of program.
Imager is designed to function in both categories. In
this section we briefly consider how it differs from some
of these other programs.
Bomb and MusiKalscope represent good illustrations
of programs that implement a particular graphic algorithm
or family of algorithms. Bomb’s algorithms, for example,
are based primarily on ideas from artificial life research.
MusiKalscope’s are based on kaleidoscopic imagery. The
knobs that these programs provide to the player directly
reflect the structure of their algorithms. Although these
controls can be used to produce some exciting and useful
effects, the models are not constructive, and thereby
extensible, in the sense that Imager’s graphic model is.
Moreover, because of the close association between the
controls and the algorithmic structures, the controls do not
map readily into the kinds of understandable aesthetic
choices that artists are used to making.
BlissPaint provides a library of animated shapes and
patterns, called scribblers, and permits you to determine
where these shapes are drawn using distributors. A
sequencer allows you to script the scribblers, and a color
synthesizer can be used to control them in real-time.
Though BlissPaint provides a substantial set of primitives,
it does not provide a programming language with which
to mix and arrange them. Where BlissPaint uses fairly
static arrangements of coarse-grained primitives (that is,
each encapsulates relatively large amounts of behavior),
Imager provides more fine-grained primitive elements
and a powerful programming environment that enables
the artist to combine and control them.
GEM [3], like Imager, is embedded in a real-time,
visual programming environment (Miller Puckette’s Pd).
To first order, GEM’s architecture simply exposes the
various primitives present in OpenGL. In contrast,
Imager’s primitives embody models derived from the
work of painters and graphic artists. Therefore, we
believe it will be easier to produce non-trivial works of
high artistic quality using Imager.
7. Conclusions
We have described an architecture that integrates
Imager into Sonnet, resulting in a system that is capable
of generating a wide variety of dynamic visuals. Further,
our architecture is based on strong aesthetic principles,
which makes our system a powerful tool in the hands of
artists and musicians.
The value of this very flexible and rich framework
would clearly be augmented by a user-interface that is
more imitative of painting and similar modes of
interaction than the circuit metaphor. This is an important
area for future research. However, we believe that, unlike
other competing systems, Sonnet+Imager’s modular
architecture provides a platform that is uniquely suited to
achieving this long-standing objective.