Hollywood has used the concept of artificial intelligence in films for decades. Now, one A-lister is trying to use AI to make art, instead of just inspire it.
Kristen Stewart is most well-known for her star role in the massively successful (and massively mocked) Twilight movies. Less well-known is her interest in AI, laid out in a new paper on the use of the technology to create art in her screenwriting debut, Come Swim. The paper was released yesterday on ArXiv, an online research repository run by Cornell which publishes papers before they’ve been peer reviewed.
Stewart’s starting inspiration point for Come Swim was one of her own paintings. The paper describes the filmmaker’s experiments with style transfer, a popular use of machine learning that transforms one image into the artistic technique and color profile of another. Stewart and her producers used the technology to turn scenes of Come Swim into the style of Stewart’s own painting. (You can see the images in the paper here [pdf]). Stewart’s co-authors on the paper are a producer at Starlight Studios (which produced the film) and an Adobe employee, whose involvement in the film is unknown.
The paper’s most interesting aspect is its ambition: The team originally tried to tune the algorithm to transfer the sense of emotion in the painting.
“The painting itself evokes the thoughts an individual has in the first moments of waking (fading in-between dreams and reality),” the authors write. “This directly drove the look of the shot, leading us to map the emotions we wanted to evoke to parameters in the algorithm.”
Trying to direct the algorithm into producing an artistically satisfying image proved more difficult than expected, according to the paper. Instead of tinkering with the algorithm itself, they found it easier to modify the images from the film. The team specifically cropped and added blocks of texture to the input images, so the algorithm would be sure to include those influences more heavily when making the final image.
Come Swim is headed to the 2017 Sundance Film Festival later this month and has been described as “a diptych of one man’s day; half impressionist and half realist portraits.”
Stewart isn’t the only person interested in using AI for art: Google has a team working on it called Magenta, and startup CreativeAI keeps track of projects that use code to make music, imagery, and other artistic expressions.