Paulina Zybinska and Marte Roel first met one night in the Museum fuฬr Gestaltung Zuฬrich. They were both helping set up the ยซPlanet Digitalยป exhibition. While debugging the software of their respective works, they started talking about real and false memories, about the role of perception, about the potential of technology and new media in therapeutic settings, and about deepfakes - a topic that makes schools, media and governments shudder. After a few more late-night discussions, it was clear that Paulina, who had just completed her masterโs degree in interaction design at ZHdK, and Marte, a postdoc at the University of Zurichโs Department of Psychology, would start working together. A year later, the funding application they submitted together with Professor Birgit Kleim from Zurichโs Psychiatric University Hospital (PUK) and other researchers was approved by the Digitalization Initiative of the Zurich Higher Education Institutions (DIZH).
A matter of perspective
Childhood trauma affects people for the rest of their lives. Therapies can have a supportive effect when memories surface and repeatedly trigger the same feelings of grief and loss. This is the starting point for a research project titled ยซThe impact of deep-fakes in virtual reality scenarios for mental health therapy.ยป Contrary to what its lengthy title might suggest, the project does not aim to overturn the status quo, but to subtly extend it. ยซThe question of perspective is fundamental to my work. It is about perceiving others and acknowledging their emotions,ยป says Roel, whose work combines scientific and artistic methods. One therapeutic approach involves patients connecting with their childhood self. There are different ways of doing this, including theatre or rituals. Or - and this approach hasnโt been used often - technologies such as deepfake and virtual reality. These should help those patients in particular who struggle to visualize.
False and genuine memories
The room looks sparse. A few empty tables, a barely used bookshelf: unsurprisingly, Zybinska and her teamโs project office was previously a storage room. Perhaps little else is needed when ideas and concepts concern virtual rather than physical reality. When asked about her motivation, however, Zybinska mentions a host of references and cross-references. ยซMy masterโs thesis explored the phenomenon of false memories and interpreted it artistically in an installation based on deepfakes. Elisabeth Loftusโs experiments were a particularly important point of reference. How false memories can arise when the same thing is repeated often enough, and when others refer to it and confirm it. That is how, talking to my mother, I seemed to remember things that didnโt happen as I believed they had.ยป Zybinska also refers to Julia Shaw, who researches traumatizing experiences from a neuropsychological perspective and argues that the emotional impact is more important than the details. It is less about whether something really happened as we believe it did than about what feelings remembering those events trigger.
A window between present and past
Sat before me is a child-sized doll with a 360-degree cam- era mounted on its forehead. Zybinska is typing while Roxane Frund, a psychology student playing a therapist for the next few minutes, gives me some final instructions. Although I am not in therapy, I must admit that I am a little nervous. I am supposed to imagine the doll in front of me as my younger, despondent self. I need to say something comforting or perhaps even hug the doll. The camera records everything. Shortly afterwards, I put on the VR headset and find myself in a different room. Instead of the doll, a child is sitting in front of me, on whom an AI application, based on a photograph uploaded the previous day, has placed my digitized, childlike face: a deepfake, in other words. I hear my voice utter the words I have just recorded and see the child tap its feet and appear to react with its facial expressions. Then we swap roles: I take the dollโs place and now watch my adult self speak the comforting words. A window of sorts opens up between the present and the past, albeit briefly and without the crucial framework of an in-depth conversation that would be part of a real therapy session. Zybinska and Roelโs research project, which gives the term deepfake a positive connotation, is still in its infancy. If all goes well, the first patients will find some comfort in the virtual encounter with their childhood self in a few yearsโ time. Past suffering will not be forgotten, but it might become slightly more bearable.