Your Mona Lisa

Category: Interactive Installation, Machine Learning Art
Data: Fall 2022
Duration: 4 Weeks
Teammate: Individual Project
Role: Designer, Programmer, Curator

Project Description

Your Mona Lisa" is an networked installation located in Shanghai that captures and transmits the smiles of its visitors. By extracting and analyzing facial features, the interactive artwork deconstructs and reconstructs the emotional expressions of its participants.

With the aid of ml5 and facial recognition technology, this installation transports the radiance of users' smiles to a server built with node.js, ultimately projecting them onto an LCD screen connected to a Raspberry Pi.

This project aims to craft a surreal experience that delves into the depth of emotional connection through facial expressions. It suggests that every person has their own unique Mona Lisa, and that their smile can be the Mona Lisa's smile to someone else.

Project Inspiration

Expanding the Web

I start by randomly selecting one electronic component before the creation. I choose LCD screen, that have multiple meanings under the context of digital media. Inspired by the topic "Expanded Web" from my professor Gottfried Haider at NYU Shanghai, I was thinking about how to expanding the web, making networked objects that construct artistic meanings between the two screens: computer screen displaying the Internet, as well as the LCD screen. To some extent, my goal is about making networked installations.

LCD Screens as Medium of Deconstructivism

Then I realize one way to create meaning is by deconstructing the common pursuit of LCD screens, which is the "ultimate dsiplay", that displaying the eye-catching contents. Inspired by deconstructivism in post-modernism, I choose to apply screens as tool of deconstruction.

Then I think from the users' side, how to promote the participatory experience for users. I realize capturing their facial parts might be one possible approach.

Moodboard

Plaster

Deconstruction

Testing on Facial Capturing

Design Details

Workflow

Web Interface Design

I realiza facial recognition and capturing might be sensitive for certain users, in order to lower the degree of embarrasment while still engaging them by presenting their facial parts on the installation, I intentionally hide the whole capturing of their face, only showing the parts that would be transmitted to the lcd screen.

The other parts follows the "mosaic" styles, which somehow follows the theme of deconstruction.

Physical Installation Design

At first I try to let users drawing on the web page, and transferring the drawing to the lcd screen. However, I realize there is some gap between such interaction system and my design goal to explore the theme of deconstruction by capturing their facial expression.

Then I try to redesign the whole process, focusing on transferring their facial image in real time. I encountered many technical issues when dealing with the LCD screen connected with the Pi (eg. limitation of canvas size/rotation for transferring, serious delay, etc). Luckily those issues are finally solved.

Testing on Facial Capturing

Making cubes/facial parts from Epoxy Resin

The Code Using ml5

Project Prototype & User Testing

Initially, I put all the parts in a box mirroring various parts.

During the user testing, I got the following valuable feedbacls:

Creating a "daily scene" to improve the visual: I may take the box away and spread them as daily components that are embedded in real life.
Casting the LCD screen image.
Capturing multiple users' smile and converge them to the installations that have multiple LCD screens, strenghening the concept of "reconstruction".

These are all valuable feedbacks, and I try to take the box away, spreading the various components.