Dr Kelly Jakubowski is an Associate Professor in Music Psychology. Her research examines a range of topics, including memory for music, music-evoked autobiographical memory, musical imagery and imagination, earworms, absolute pitch, musical timing and movement, and cross-cultural music perception.
Dialogue caught up with Kelly to find out how her research has informed the launch of the Alice Dalí app, an educational app based on Lewis Carroll's story of Alice’s Adventures in Wonderland and artwork by Salvador Dalí.
Dr Kelly Jakubowski with the Alice Dalí app
Tell us about the ‘Alice Dalí’ app and the concept behind it.
The Alice Dalí app was originally developed by my research collaborators, Dr Mads Haahr and Dr Svetlana Rudenko, who are both affiliated to Trinity College Dublin and Haunted Planet Studios. This free app comprises a navigation game in which users search for augmented reality art and music experiences using a radar on their mobile phone screen. It’s a fun and educational experience based around the story of Lewis Carroll’s book Alice’s Adventures in Wonderland. The artwork consists of 12 paintings by Salvador Dalí, who painted these for the 1969 edition of Carroll’s book, and the soundscape of the experience comprises newly composed piano music by Dr Rudenko alongside narrations from the story itself. Haunted Planet Studios has produced other experiences of this type in Ireland and Denmark, but this is the first one installed in the UK.
How has your research informed it?
We have recently conducted a study testing the intergenerational impact of the game on children and their carers. Over 40 primary school-aged children participated in a study in Durham’s Botanic Garden, in pairs alongside a parent, grandparent, or other carer. We found a range of initial evidence that the game encourages intergenerational cultural engagement and cooperation, and increased feelings of social connectedness. We hope in future to further explore the benefits of games like this for learning about new art and music styles and encouraging people to get active and go outdoors whilst engaging with new technologies.
Visitors at the Botanic Garden with Dr Kelly Jakubowski
What benefits do you think augmented reality can bring to arts and humanities research?
Augmented reality can make art experiences accessible to a wider audience than many traditional exhibition formats. The purpose of Alice Dalí and similar apps is to be able to bring cultural experiences, in this case paintings by Salvador Dalí and newly composed classical music, to anyone with a smartphone, to be enjoyed in a game-like, outdoor setting. Initial research findings have shown that this type of “less formal” engagement with the arts can encourage users to subsequently seek out art and music in more traditional formats (e.g., galleries, concerts).
Who is the app aimed at?
The app can be downloaded for free by any visitor to the Botanic Garden and provides a fun “virtual art exhibit” that can enhance a walk around the garden with an educational and cultural experience. The app was developed with the aim of providing an intergenerational play experience, where children (particularly aged 5-12) are encouraged to seek out the music-art experiences together with their carers. We’ve also found plenty of interest from other users across a range of ages and demographic backgrounds.
Why is the Botanic Garden a good location?
Durham’s Botanic Garden is an ideal location for hosting this app, as many of the physical landmarks in the garden align quite congruously with the augmented reality world of the Alice story. For instance, the iconic insect and mushroom sculptures that are already present in the garden serve to augment the feeling of Alice shrinking to a very small size and the fantastical world she encounters throughout the story.
Why do you think music forms an important part of this?
The music forms an integral part of the app experience, as the user begins to hear the music as they get closer to finding the art image (but before they see the art). This begins to effectively set the mood for the art encounter, and the music serves as a first cue that one is getting closer to each image.
The music was composed by Svetlana Rudenko, currently a Visiting Researcher at Durham, using her expertise on the experience of synaesthesia. Synaesthesia is the phenomenon whereby perceptual experience in one modality leads to spontaneous sensations in another modality, for example, when hearing music or sounds elicits mental images of colours. The music is thus composed to reflect the colours, textures, and narrative of the Alice story and Dalí’s paintings of this in particular.
Tell us how we can get involved…
A launch event for the app was held on Saturday 4 May 2024 at the Botanic Garden. The app will be free to download for all visitors to the Garden for the foreseeable future.
How does an ideal weekend look for you?
Much like the Alice Dalí app, my ideal weekend would involve plenty of music (I’m a violinist), some cultural experiences, and lots of fresh air in beautiful outdoor locations with family.