L’Oréal's Makeup Genius App: The Cosmetics Counter Goes Digital
By Caroline Winter September 11, 2014
Photograph by Andrew Musson
A combo of darker lipstick, eyeliner, and eye shadow. It’s a bit much
Aryanna Dudnat, a beauty adviser at a Duane Reade in Midtown Manhattan, was hired to help women find new cosmetics. Instead, she often cleans up after customers. “Every day, I come in and find that people have unwrapped products,” she says. She doesn’t blame them. “We don’t carry testers, and buying makeup without trying it is risky.” The art of purchasing drugstore cosmetics is wildly imperfect, verging on absurd—you’re expected to choose a foundation by comparing the color of the bottle to the color of your arm and to spend $10 on a lipstick after holding it next to your face in front of a tiny mirror.
L’Oréal (OR:FP), the French beauty giant, thinks there’s a better way. In June the company released Makeup Genius, an app that lets you see yourself in real time wearing products that aren’t actually on your face. When you smile, pucker up, or wink, the virtual cosmetics move along with you. If you turn rapidly to the side, the screen lags a second before the digital mask jostles back into place. You can apply, say, a lip liner with a lipstick or choose from full looks such as Evening Smokey and Jennifer’s Nude (as in J. Lo), created with a bundle of products.
A combo of darker lipstick, eyeliner, and eye shadow. It’s a bit much
Aryanna Dudnat, a beauty adviser at a Duane Reade in Midtown Manhattan, was hired to help women find new cosmetics. Instead, she often cleans up after customers. “Every day, I come in and find that people have unwrapped products,” she says. She doesn’t blame them. “We don’t carry testers, and buying makeup without trying it is risky.” The art of purchasing drugstore cosmetics is wildly imperfect, verging on absurd—you’re expected to choose a foundation by comparing the color of the bottle to the color of your arm and to spend $10 on a lipstick after holding it next to your face in front of a tiny mirror.
L’Oréal (OR:FP), the French beauty giant, thinks there’s a better way. In June the company released Makeup Genius, an app that lets you see yourself in real time wearing products that aren’t actually on your face. When you smile, pucker up, or wink, the virtual cosmetics move along with you. If you turn rapidly to the side, the screen lags a second before the digital mask jostles back into place. You can apply, say, a lip liner with a lipstick or choose from full looks such as Evening Smokey and Jennifer’s Nude (as in J. Lo), created with a bundle of products.
“If you hold your phone here rather than here, the lighting will be different, and the color will look different,” says Guive Balooch, global director of the company’s Connected Beauty Incubator. Hidden in a tiny, windowless section of an office building in Clark, N.J., Balooch’s tech lab houses 15 engineers and scientists. (In addition to Makeup Genius, the team has a few other projects in the works but won’t divulge details.) The office-within-an-office looks like a Silicon Valley gag, with whiteboards, colorful walls, and a silver ceiling that gives off a futuristic vibe. “We completely renovated this little area,” Balooch says, comparing it with the beige cubicle landscape that fills out the rest of the floor.
Story: How L'Oreal Knows What You Look Like Balooch, perfectly groomed with spiky salt-and-pepper hair, has a Ph.D. in biomaterials and came up with the idea for Makeup Genius with a fellow biologist during a brainstorming session 18 months ago. At the time, several cosmetics companies, including L’Oréal, were offering virtual makeup counters on their websites and at drugstores, but most required users to upload a picture and Photoshop lipstick and eye shadow onto the static image—a time-consuming process. “The Walmarts (WMT) don’t have time for high-end makeup artists,” Balooch says. “But there are thousands of colors, and how do women know which will look good on them?”
“There is as much research on cosmetics products as on the space shuttle”
To address that, some brands have created one-shade-fits-all cosmetics. Stila advertises blushes and bronzers that, the company says, react to “each individual’s pH to create a perfect, personalized shade.” Dior (CDI:FP) sells self-adjusting lipstick and blush. Perricone MD has products intended to work with every skin shade, from bloodless white to dark black. And so-called BB (beauty balm) and CC (color correcting) creams have quickly become bestsellers by appealing to women with a broad range of complexions. Some of these products do work, but most fail on dark skin tones. “There’s not one shade that fits everybody,” says Christa Varrone, a skin-care expert for Sephora. “That’s not possible.”
Balooch’s team tested eye, lip, and cheek products on hundreds of models with varying complexions, capturing how each shade and texture transforms under 400 different lighting conditions. The company also collected more than 100,000 images to compare how the makeup looks on the models in real life vs. on-screen. (L’Oréal won’t disclose how much it spent developing Makeup Genius.)
“There is as much research on cosmetics products as on the space shuttle”
To address that, some brands have created one-shade-fits-all cosmetics. Stila advertises blushes and bronzers that, the company says, react to “each individual’s pH to create a perfect, personalized shade.” Dior (CDI:FP) sells self-adjusting lipstick and blush. Perricone MD has products intended to work with every skin shade, from bloodless white to dark black. And so-called BB (beauty balm) and CC (color correcting) creams have quickly become bestsellers by appealing to women with a broad range of complexions. Some of these products do work, but most fail on dark skin tones. “There’s not one shade that fits everybody,” says Christa Varrone, a skin-care expert for Sephora. “That’s not possible.”
Balooch’s team tested eye, lip, and cheek products on hundreds of models with varying complexions, capturing how each shade and texture transforms under 400 different lighting conditions. The company also collected more than 100,000 images to compare how the makeup looks on the models in real life vs. on-screen. (L’Oréal won’t disclose how much it spent developing Makeup Genius.)
The result is mesmerizing but can look over-the-top: I tried it when I was home sick, and it only emphasized my sunken eyes and sallow complexion. The woman staring back at me from my iPhone looked like a more ghoulish version of Joan Jett. The company says the app has been downloaded more than 1.7 million times in the U.S. and France, the two countries where it’s available. Of course, L’Oréal hopes all that tapping will lead to purchases, either through its built-in e-commerce platform or at a local store, but it won’t share numbers on sales driven by the app.
During development, L’Oréal worked with Image Metrics, a creator of facial recognition software for video games and movies, including 2008’s The Curious Case of Benjamin Button, in which the title character, played by Brad Pitt, ages in reverse. (The only other option would have been the U.S. Department of Defense, Balooch says, joking that “inspired by the TSA” would have made a catchy tag line.) “Did we see ourselves devoting this amount of effort to makeup two years ago? Certainly not,” says Ron Ryder, chief executive officer of Image Metrics. “But working on something this transformative is fantastic. In 10 years, will anyone buy makeup without a digital, perfect try-on first?”
L’Oréal, which holds more than 35,000 patents, spends 3.7 percent of its revenue—more than $1 billion annually—on research and development, about twice the industry standard, according to Bloomberg analyst Deborah Aitken. That budget funds an army of more than 4,000 scientists working in about 50 countries, all of them hunting for the next big thing in beauty. Perhaps something as groundbreaking as commercial hair dye, which L’Oréal says it created in 1907, or sunscreen, which the company introduced in 1936. Its Global Hair Research Center in a suburb of Paris is the size of four football fields and houses rooms full of robots that do things such as wash hair over and over again. “There is as much research on cosmetics products as on the space shuttle,” Laurent Attal, L’Oréal’s executive vice president for research and innovation, told Women’s Wear Daily last year.
During development, L’Oréal worked with Image Metrics, a creator of facial recognition software for video games and movies, including 2008’s The Curious Case of Benjamin Button, in which the title character, played by Brad Pitt, ages in reverse. (The only other option would have been the U.S. Department of Defense, Balooch says, joking that “inspired by the TSA” would have made a catchy tag line.) “Did we see ourselves devoting this amount of effort to makeup two years ago? Certainly not,” says Ron Ryder, chief executive officer of Image Metrics. “But working on something this transformative is fantastic. In 10 years, will anyone buy makeup without a digital, perfect try-on first?”
L’Oréal, which holds more than 35,000 patents, spends 3.7 percent of its revenue—more than $1 billion annually—on research and development, about twice the industry standard, according to Bloomberg analyst Deborah Aitken. That budget funds an army of more than 4,000 scientists working in about 50 countries, all of them hunting for the next big thing in beauty. Perhaps something as groundbreaking as commercial hair dye, which L’Oréal says it created in 1907, or sunscreen, which the company introduced in 1936. Its Global Hair Research Center in a suburb of Paris is the size of four football fields and houses rooms full of robots that do things such as wash hair over and over again. “There is as much research on cosmetics products as on the space shuttle,” Laurent Attal, L’Oréal’s executive vice president for research and innovation, told Women’s Wear Daily last year.
Photograph by Andrew MussonL’Oréal’s Balooch at his lab in New Jersey
Makeup Genius is L’Oréal’s most advanced digital product to date, but it already has competition. ModiFace, a Toronto company that’s spent seven years building 2D and 3D simulations for cosmetic surgeons’ offices, recently created its own real-time virtual makeup mirror. The first prototypes were introduced in February at a Milan Sephora, where two touchscreen mirrors lured customers by zooming in on the closest shopper and automatically painting his or her face with lipstick and eye makeup. “If they’re intrigued, they can walk over and try different shades,” says Parham Aarabi, founder and CEO of ModiFace. “It allows you to try on 100 eye shadows in the time it takes to try one in real life.”
ModiFace’s machine, which works with products from five brands and will soon be rolled out in 40 stores internationally, displays your face as if you’re looking into a mirror. “We always flip the image,” Aarabi says. “If you don’t, it becomes strange for the person.” L’Oréal’s app, by contrast, shows you as you appear to the rest of the world. “It’s the selfie revolution,” Balooch explains. “The interaction we have with our phones is so natural.” ModiFace is also more subtle than Makeup Genius: Almost all of the latter’s preset looks, as one colleague put it, “make you look like a 5-year-old drew on your face.” Balooch is aware of this issue. “We are working on being able to adjust the intensity of the makeup,” he says. “But for phase one, the goal was to provide consumers, on average, the best rendition possible.”
Makeup Genius is L’Oréal’s most advanced digital product to date, but it already has competition. ModiFace, a Toronto company that’s spent seven years building 2D and 3D simulations for cosmetic surgeons’ offices, recently created its own real-time virtual makeup mirror. The first prototypes were introduced in February at a Milan Sephora, where two touchscreen mirrors lured customers by zooming in on the closest shopper and automatically painting his or her face with lipstick and eye makeup. “If they’re intrigued, they can walk over and try different shades,” says Parham Aarabi, founder and CEO of ModiFace. “It allows you to try on 100 eye shadows in the time it takes to try one in real life.”
ModiFace’s machine, which works with products from five brands and will soon be rolled out in 40 stores internationally, displays your face as if you’re looking into a mirror. “We always flip the image,” Aarabi says. “If you don’t, it becomes strange for the person.” L’Oréal’s app, by contrast, shows you as you appear to the rest of the world. “It’s the selfie revolution,” Balooch explains. “The interaction we have with our phones is so natural.” ModiFace is also more subtle than Makeup Genius: Almost all of the latter’s preset looks, as one colleague put it, “make you look like a 5-year-old drew on your face.” Balooch is aware of this issue. “We are working on being able to adjust the intensity of the makeup,” he says. “But for phase one, the goal was to provide consumers, on average, the best rendition possible.”