Aaron the Artist -- Harold Cohen
Harold Cohen has been the creator and mentor of AARON, a painting robot, for more than 20 years. The result is the first robot in human history to paint original art. AARON mixes its own paints, creates striking artwork and even washes its own brushes. Read Cohen's answers to questions about AARON and his creative process with this highly evolved expert system.
I'd like to know more about how you were inspired to create AARON. It seemed the genesis of your idea may have been your observations about how children draw. Is this so? And how did your idea develop from there?
I was always fascinated by the fact that we can look at a drawing and "see" a face, when we are actually looking at a few dirty marks on a piece of paper. That mysterious ability is obviously central to representation, and I'd been stewing over it, not really understanding how it worked, for at least ten years before AARON began. When I met my first computer in 1968 I had an intuition that I might be able to learn more about this mystery by programming than I had learned in twenty years as a painter. I couldn't persuade the computer to produce images -- marks to which the viewer would assign meaning -- unless I could propose a mechanism capable of generating the images. (One of the nice things about the computer is that it allows you to test out in simulation things that were untestable previously.)
At that point I did, indeed, look very closely at the way young children draw. It seemed to me, from having watched my own children when they were very small, that there's a point at which they will produce a sort of closed, containing shape for their scribbles, and that this point corresponds to their notion that they are making a representation -- that the marks they make "stand for" something. AARON's earliest efforts involved being able to draw an enclosing form around a scribble.
Actually, that has never changed. AARON can now construct a range of "core figures" to replace the scribble, and it's smart enough now to know that it shouldn't always draw the entire outline -- it will never draw all the way around a nose, for example. But the notion of first constructing an invisible core figure and then drawing a line around it remains a constant feature.
When you first began programming your robot and moving away from painting yourself, what was the reaction of other artists? Were they skeptical of your new interest? Generally, how do other artists feel about Aaron today?
I really don't have much idea how other artists feel. Some have told me they thought that what I was doing was wonderful. Curiously enough, they were often the very people I would have assumed would hate it. On the other hand it wouldn't surprise me to know that there are many who think AARON doesn't have anything to do with art, since artists have a deep commitment to what THEY understand art to be. Some of my University colleagues thought I was wacky, giving up a distinguished career as a painter to play with these silly machines -- until I started getting exhibitions at major museums. A little authentication goes a long way!
I liked AARON's works of art! Are they for sale? If so, where, and can you tell us approximately how much the paintings cost?
Yes, AARON's work is for sale. I haven't worked through a dealer in more than thirty years; I sell it myself. I don't think this is the right place to start quoting prices, but anyone who's seriously interested can send me email and I'll be happy to tell them whatever they want to know. I will say, however, that the machine paintings generated during the Computer Museum show three years ago were selling for $2000.
The paintings we saw that Aaron had created were all portraits of people. Does Aaron also paint landscapes, animals or other subjects?
AARON can make paintings of anything it knows about, but it actually knows about very little -- people, potted plants and trees, simple objects like boxes and tables, decoration. From time to time I wonder whether it wouldn't be a good idea to tell it about more, different, things, but I can never persuade myself that it would be any better for knowing how to draw a telephone, for example. So I always end up trying to make it draw better, not more.
Did you have to build all the software to program AARON yourself or did you build it off an existing program?
I wrote all the software myself; originally in 'C' but for most of AARON's development in LISP. I couldn't have used existing software even if I'd wanted to, for one very simple reason; AARON has to know what it's doing, and has to spend most of its time building an internal representation of the developing drawing so that it can decide what to do next. Existing programs are based on the assumption that it's the user that has to know what's going on, not the program, and consequently they don't provide the facilities for building an internal representation.
What kind of paints and other materials does AARON use to make the paintings? Does he always use the same media?
AARON uses a range of fabric dyes, PROCION, on the painting machine. When I color myself I use oil paint, and I would have preferred to have used oil paint on the machine. But that wasn't really feasible; even mixing and thinning the colors would have been a major robotic feat, much less being able to paint with them, clean the brushes and so on.
Do you like everything Aaron creates - or sometimes do you hate it? If it made something you didn't like, would you keep it or throw it away?
No, I like some of its works more than others. I'm not sure how big an issue my own personal taste is, however. When it was doing black and white drawings, for example, I found that I preferred the more elaborate ones to the simple ones. But when I did exhibitions where the audience could buy the drawings, it turned out that I sold just as many of the simple ones as I did of the heavily-worked ones.
I've always discarded work if I concluded it was inadequate in some way. Curiously enough, I have more difficulty discarding AARON's; it's as if the work was done by "someone" else and I don't have the right to destroy it. It was very much the same as when I was teaching. There would always be some students who didn't bother to collect their portfolios after I'd graded them, and I had drawers full of the stuff for years because I couldn't bring myself to destroy someone else's work.
You said Aaron paints what you "tell him about." Can you please explain something about how you "tell" it?
How do I "tell" AARON what it knows? That's a bit hard to understand if you don't know anything about programming. Some of AARON's knowledge is what we call "declarative" knowledge; for example, how long arms and legs are. That sort of thing is easy to represent. You simply make a list of parts -- left-upper-arm, torso, etc. -- each of which is a list of all the points in that part, with the position of the point in relation to the origin of the part. For example, the origin of "left-upper-arm" is "left-shoulder", and "left-elbow" is at some position in relation to it: so much below, so much to the left, so much in front.
AARON knows that the "left-upper-arm" is tacked on to the torso, but it can only tack it on in ways that are plausible for a real body. The program has to know how to go about doing things, and this "procedural knowledge" is usually represented in the form of rules. It might look something like this:-
if (left-arm-posture is "hand-on-hip")
(add-upper-arm left -.3 .5 .65)
if (left-arm-posture is "arms-folded")
and so on. The three numbers here would represent the rotations around the shoulder on the three axes that would put the arm at the correct angle before the arm is attached to the torso at the shoulder. But this is a much-simplified account, because of course there isn't just one "correct angle," and the way it chooses one of the three rotations will then influence how the remaining two can be chosen.
And at a higher level, of course AARON had to decide, using rules similar in form but different in content, what the "left-arm-posture" was going to be.
I was wondering what would happen to Aaron in the future? Do you believe that Aaron could become so advanced that it could actually make a professional picture that seems hand painted? I mean, could it make all the tiny details in pictures without flaws and mistakes -- such as a texture on the back of a reptile. Also if it could do this how could you program it so well that it would do it?
Of course a computer program can do the texture on a reptile's back -- just think of the dinosaur skin in "Jurassic Park." Being able to produce an imitation color photograph isn't really all that advanced today, and I personally have never been interested in advancing in that direction. Just for the record, it would be hard to find more than a small handful of artists in the entire history of art who were interested.
I wonder all the time about what AARON will be doing in the future. To make a good guess, the question to ask is not what a machine might be capable of doing -- after all, a camera can do the tiny details, so can a printing press -- but why AARON was written in the first place. I didn't write it to have it do what I can do perfectly well myself; I wrote it to discover what an independent (machine) intelligence might do, given some knowledge of the world and some rudimentary physical capabilities. And, in the process, to have IT teach ME about possibilities I hadn't imagined. I'd be happier if AARON's work in the future were LESS like human work, not MORE like human work.
Are you the only person that can "direct" AARON's artistic work? Or have you experimented with letting other people tell it what to do and see what happens?
I'm the only person who deals with AARON, and I don't tell it what to do. I tell it what it knows, and IT decides what to do. Sometimes, though, I have to tell it what NOT to do; for example, while I'm working on the program's coloring functions I don't want to wait fifteen minutes while it draws an elaborate tree or a potted plant before it starts coloring. I want it to get to the stuff I'm interested in as quickly as possible. The program isn't set up for interaction, so I have to get inside the program itself and modify the code so that AARON "forgets" -- i.e., it temporarily has no access to -- the things I don't want it to do.
AARON is currently about as long as a fair-sized novel and obviously I'm the only person who knows it well enough to make those modifications without screwing it up completely. But the truth is that I'm not really interested in what other people would do; I'd be much more interested in having them write their own programs based upon their knowledge and their understanding of art.
How does AARON know if he is staying within the lines he draws? Does he have a sensor so he knows when he hits the black, or does AARON actually remember where the line is?
AARON doesn't have any physical or visual feedback system. It remembers everything it does in the painting; in fact, it spends at least half of its time in building a very elaborate "internal representation" of the image as it develops. Knowing where the lines are is necessary, but it isn't nearly enough; it needs to know what each little patch of the painting represents, what is in front of what, and so on. When it comes to coloring, for example, it's quite likely to find that a part has been cut up by something passing in front of it, and it has to be able to find each of the sub-parts so that it can make them all the same color. That's quite a lot to remember, and having sensors wouldn't actually help it very much.
Do you teach classes about how to make a painting machine like AARON or have you published any books or papers that could help someone get started building a simple version of their own robot like this?
No, I don't teach any classes now; when I did they were in painting and in programming. I designed all of AARON's drawing and painting machines, and built most of them. But I had no training in either electronics or engineering, and had to discover what I needed to know as I went along.
There are some books on simple robotics, though I never found them to be very helpful. There's also a magazine, "The Robotics Practitioner," which is useful both for the articles and for the ads it carries. You need a workshop, of course, and a little skill. But, even more important, you need to know what's available. Get hold of some engineering catalogs and visit any electronic surplus stores you can get to; there's an astonishing variety of clever things available, and just seeing them will stimulate your imagination. (You might want to think of some way of stimulating your bank account while you're at it, by the way. Robotics is not the world's cheapest pastime.)
Having said all of that, I should say also that a robot is only interesting if it DOES something interesting. My own painting machine wouldn't have been worth building if AARON hadn't had anything worth using it for. The first thing is programming; the last thing is engineering. And once you can program, controlling your robot becomes a much simpler job. Using a simple interface card, anything that can be controlled by a change of voltage can be controlled by software.
Scientific American Frontiers
Fall 1990 to Spring 2000
Sponsored by GTE Corporation,
now a part of Verizon Communications Inc.