Dancing with Machines: Aesthetic interactions with an autonomous system.
As artists who create interactive sensory spaces for young audiences, the TAS (Trust Autonomous Systems) Artists Residency allowed us to delve deeper into robotics and help narrow the gap between robotics and interactive arts for children. Though artists have used it for decades, robotics in art is still a novel idea. In the 1920s, the Czech playwright Karel Čapek coined the word 'Robot' in his play, R.U.R. (Rossum's Universal Robots). The theme of his play, man vs machines, is still perforated in films about robots and many artworks. By the mid-twentieth century, there was a rise in artists' and performers' interest in robots for critical reflection, particularly around themes of human /robot relationships and 'society's obsession with technologies.' [1] However, very little of this work was created for young audiences. Today most young children interact with robots are through books, films, and toys. Robots are a popular item in the children's toy market; a recent report forecasted that there would be a significant rise in robotic toys over the next few years [2]. We are interested in designing work where children can engage with robots not as small toys but as imaginative embodied kinetic sculptures that can create novel, fun, and empowering experiences.
We choose to work with a robot arm in our first robotic work for young audiences. Industrial robot arms are usually associated with functional design; they do not have a face or humanoid features synonymous with social robots. Our work and research explore an imaginative interpretation of the industrial robotic arm to make it into a playful kinetic sculpture that becomes something unexpected, which performs and encourages its user to follow suit. We experiment with how variables, such as a costume, can change the status of a robotic arm and how fiction can create a willing suspension of disbelief where audiences begin to trust and permit themselves to reimagine a new reality.
Our performative installation, designed in May 2021, is based around a fictional character called NED, the Never Ending Dancer whose only interest is dancing with people. It was developed with researchers at Mixed Reality Lab, the Cobot Maker Space at the University of Nottingham [3], and choreographer Liz Clark. It uses facial tracking to follow the audience's actions, and three musical scores accompany the work to encourage the audience to dance. Its costume is designed to make it look like a tropical, mechanical, hybrid bird-like creature. The long blue feathers attached to the arm extends its reach, and its metal skirt sways as the NED follows an audience member.
NED was part of the Thingamabobas installation, a circus troupe of four mechanical creations which opened in the summer of 2021 for families at Lakeside Arts, Nottingham, and has subsequently toured to a local library and national arts venues.
The Thingamabobas installation at Theatre Hullabaloo July 2022
The installation intrigued audiences; as a result, we had dozens of children and families dancing with NED, and to one parent's surprise, a child who did not dance in public danced with NED! Although the facial tracking only follow one audience member, we found entire families engaged with NED simultaneously.
NED @ Broadway Gallery
Cose up of NED's costume
People, young and old, displayed a sense of curiosity and wonder when interacting. A child commented, " it was amazing; it reacted to you, and it was so amazing!" while another said, "I liked how NED knew where you were."
Our first experience using robotics demonstrated the potential for developing this work further. As we began our TAS artist residency, we wondered what kind of imaginative interfaces we could design to optimise the playability of a robot to engage and enchant audiences. We wanted to find out the following:
- How much do children understand and give meaning to their communication with an autonomous system?
- How curious are they about what is happening in the experience?
- What happens during the intergenerational engagement?
- How might a system otherwise engage the public?
Pilot study
To try and understand our young audience better and how they give meaning to their communication with NED, we set up a pilot study at Broadway Gallery in Nottingham between the 5th – 14th of April 2022 aimed at families with children aged between the ages of 4 to12. Participants pre-booked a session to engage with NED. We decided that the children should have little prior knowledge of NED to understand what they experience when interacting with the robot, so we decided against using an actor to frame or contextual NED. As part of Broadway Galley interactions, we programmed additional events, including 'Meet the Co-Bot Makers Space Robots', 'Scribble bot Workshops', and 'Lego Mindstorm Robotic Arm Workshops' run by the Inspire Foundation [4]. These workshops let us spend time with the families to gather informal feedback.
A Scribble bot made by a workshop participant. The task was to design a robot that makes colorful and chaotic drawings
Outcomes:
Outcomes:
We found many children who had not encountered NED before had difficulties engaging with the experience. Their engagement seemed more inhibited and restricted compared to the children who had seen NED previously at Lakeside Arts. Some asked the question before interacting with NED, and sometimes parents initiated the interaction. There could be multiple reasons for this:
- There was no 'actor' to contextualise NED, making it difficult for young children.
- The Broadway Gallery setting was different from the theatre and not an atmospheric space.
- NED without the troupe of Thingamabobas means the back story of the circus was not applicable here, creating a different experience.
- During some of the interactions, NED seemed to behave a bit erratic and would go to sleep more often; this created confusion and lost some children's attention.
These elements raise questions for further study and highlight the importance of framing the experience to help actively engage the participants. Further research and development of NED's programming could make it clearer to the audience what NED is responding to, as a foundation for trusting a system is its reliability. In addition, check the effects of the room size and how the three pre-set zones of interaction are affected, and if it could have caused Ned's erratic behaviour.
After the experience, we asked the children what they liked or disliked about NED. Most children liked that NED followed and reacted to them. Some commented on his colour and the feathers. A few children disliked that he was slow.
Children's comments:
I like how NED is motion-sensing and reacts to your movement and face.
I like him dancing; I don't like it stopping.
I like him because he follows you and he is colourful. I dislike that he is slow.
He is beautiful.
Like Ned because he follows you and he's very colourful.
I like the feathers.
He copied me.
I like that he was an animal.
He could dance.
He was very silly.
Every time I moved somewhere, he moved over to me.
I like the beep-beep sound.
That he copies my dance moves.
I like Ned dancing.
How he looks around for you, the feathers, and how he dances.
I like him because he copies us.
I like how and when he moves.
Like the dancing, he looks fancy.
Like the hidden camera, dislike, he does not go all around.
When asked what they think a robot is, most children categorise robots into two main areas a helper that does things for them and others a machine that moves, that can look like a human, is made from metal, and can be programmed.
Children's robot drawings. April 2022
Finally, we asked them to design and draw their robot. Interestingly, some drawings were of traditional robots with square bodies compared to children drawing from the '70s; some were remarkably similar.
Drawings from children in the 1970s book Robots: Fact Fiction, and Prediction, by Jasia Reichardt and drawing from children in 2022
It finds itself in space and it has a screen so you see updates, cute face, goes to the moon, grippy hands and feet for walking on the moon.
It's a robot that's shaped like a human but only sometimes looks like a human, but it has a big wheel for it to follow you around.
Children's feedback on their robot's abilities
A Theatre of Robots: Performing Autonomy
AHRC Symposium III: Immersion and Embodied Experiences: AI, VR and Inclusive Futures
Workshop participants choosing materials for their prototype at Cobot Makers Space
The workshop was part of the AHRC Symposium III: Immersion and Embodied Experiences: AI, VR, and Inclusive Futures. The aim was to discuss the relationship between trust in autonomous objects and the notion of willing suspension of disbelief. The hands-on activities were designed to encourage collaborative interdisciplinary exploration and address the challenges of using robotics in art installations.
Nine practitioners and academics from various theatre backgrounds attended. We started with an icebreaker, and participants then had the opportunity to interact with 'NED', followed by a brief overview of robotics in art and MOIW's work.
We outlined the workshop's aims and asked the participants to think about connecting trust and the suspension of disbelief, the effect of costuming a robot and how a character and identity are created, and how we create anthropomorphic qualities in inert materials. For the prototyping workshop, we supplied a wide variety of art materials and household and found objects and asked participants to form small groups to:
- Design a robot that focuses on children and allows for intergenerational engagement.
- How might a system otherwise engage the public?
- What strategies could you build on encouraging audiences to trust your robot?
There were three groups, and they spent 45 minutes discussing and making their prototype robot. The following ideas and prototypes were presented to the entire group.
SAM
SAM
'A small playful robot whose eyes are sensors. During eye contact between SAM and a person, SAM ducks and hides, his long tail sticking up in the air betraying his whereabouts. SAM doesn't speak and is incredibly playful.'
Having no speech/sound capabilities, SAM crosses communication boundaries through playful movements. Children's curiosity is aroused by this fluffy tail ducking and diving to get attention. SAM's oversized features and bright colours encourage children to engage.
The Living Doll
The Living Doll
'A series of parts that can fit into the doll's main body to custom create an interactive doll. Crossing cultures through no limit of language. Each part brings new characteristics to the doll.'
This doll is particularly distinctive in its ability to fit with any language. Each part of the doll introduced has characteristics programmed to suit the needs/wants/desires of the child/parent. Each doll is as unique as its owner.
R2D2
'R2D2'
'These two robots are visually and physically tactile. Communicating through a series of chirps, whirrs, and other sounds, they cross cultural boundaries by not using language but rather sounds to communicate. The use of tactile materials makes them particularly attractive.'
The two robots engage and excite using sounds to communicate with each other. The use of bright, tactile materials makes them particularly wonderful and enticing for children. There is a relationship between touch and the emotional centres in the brain that help us to make decisions and remember details about tactile experiences.
The prototype robots had many features in common they were playful, tactile, fun, and adaptive and language was not a barrier. The participants enjoyed the workshop and practical prototyping session; one participant commented ''I woke up bored this morning...I am so glad I am here today; it is just what I needed.
‘'I woke up bored this morning...I am so glad I am here today, it is just what I needed.
Workshop participant feedback
‘I wish I had a 100 hands’
After the workshops and experiencing various works and different kinds of robots during the residency, we began thinking about our next installation with a robot. We are interested in continuing to use robotic arms for our next piece of work, 'I wish I had 100 hands' we would like to:
- Explore Gordon Pask's conversational theory and poetic system. There is a potential to explore designing interactions for generative conversation to create a shared space between the participants and the robot. We like exploring robot agency using a Paskian approach, 'performance', 'conversation', 'interaction', 'environment', and 'participation' as outlined in Haque's paper ' The Architectural Relevance of Gordon Pask
- Continue pursuing costuming a robot within a performative space; it serves to situate a robot as a character in a performative installation and deliberately 'transform' the traditional visions of a robot. It may help young audiences trust a system.
- Use dance as a medium to explore embodied, relational meaning-making and movement with the robot as a dance partner.
- Better understand the potential of robot audition and computational auditory scene analysis, soundtracking & localisation of sound in an installation.
- Develop soundtracks that encourage movement, communication, fun, and playfulness.
In July 2022 we exhibited NED and the Ideas for our next installation using robots at the All Hands Meeting 2022, BMA House, London.
The TAS residency provided us with the time to meet various academics and practitioners and have time to research, think, view multiple works, and read about cybernetic theories. It helped us understand the limitation and potential of robotics and autonomous systems. Role costumes, narrative, and the creation of a willing suspension of disbelief can enhance the immersive experience and help audiences trust a system. We are confident that this pursuit can be fruitful and bring novel experiences to young audiences.
References
[1] Robots and Art: Exploring an Unlikely Symbiosis, Editors: Damith Herath, Christian Kroos and Stelarc
[2] Robot Toys for Children Market 2022 Incredible Possibilities, Growth Analysis and Forecast To 2028
[3] We worked with the MRL and the Co-bot Maker Lab, researchers at the University of Nottingham - Feng Zhou, Ayse Kucukyilmaz, Mithun Poozhiyil, and choreographer Liz Clark.
[4] The Inspire Foundation is formed of researchers and practitioners in STEM fields
