School of Computing Science
CS Researchers Develop A New Tool That Brings Blender-like Lighting Control to Any Photograph
Lighting plays a crucial role when it comes to visual storytelling. Whether it鈥檚 film or photography, creators spend countless hours, and often significant budgets, crafting the perfect illumination for their shot. But once a photograph or video is captured, the illumination is essentially fixed. Adjusting it afterward, a task called 鈥渞elighting,鈥� typically demands time-consuming manual work by skilled artists.
While some generative AI tools attempt to tackle this task, they rely on large-scale neural networks and billions of training images to guess how light might interact with a scene. But the process is often a black box; users can鈥檛 control the lighting directly or understand how the result was generated, often leading to unpredictable outputs that can stray from the original content of the scene. Getting the result one envisions often requires prompt engineering and trial-and-error, hindering the creative vision of the user.
In a new paper to be presented at this year's SIGGRAPH conference in Vancouver, researchers in the Computational Photography Lab at 911吃瓜 offer a different approach to relighting. Their work, 鈥溾��, brings explicit control over lights, typically available in Computer Graphics software such as Blender or Unreal Engine, to image and photo editing.
Given a photograph, the method begins by estimating a 3D version of the scene. This 3D model represents the shape and surface colors of the scene, while intentionally leaving out any lighting. Creating this 3D representation is made possible by prior works, including previously developed research from the Computational Photography Lab.
鈥淎fter creating the 3D scene, users can place virtual light sources into it, much like they would in a real photo studio or 3D modeling software,鈥� explains Chris Careaga, a PhD student at 911吃瓜 and the lead author of the work. 鈥淲e then interactively simulate the light sources defined by the user with well-established techniques from computer graphics.鈥�
The result is a rough preview of the scene under the new lighting, but it doesn鈥檛 quite look realistic on its own, Careaga explains. In this new work, the researchers have developed a neural network that can transform this rough preview into a realistic photograph.
"What makes our approach unique is that it gives users the same kind of lighting control you鈥檇 expect in 3D tools like Blender or Unreal Engine," Careaga adds. "By simulating the lights, we ensure our result is a physically accurate rendition of the user's desired lighting."
Their approach makes it possible to insert new light sources into images and have them interact realistically with the scene. The result is the ability to create relit images that were previously impossible to achieve.
The team's relighting system currently works with static images, but the team is interested in extending functionality to video in the future, which would make it an invaluable tool for VFX artists and filmmakers.
鈥淎s this technology continues to develop, it could save independent filmmakers and content creators a significant amount of time and money,鈥� explains Dr. Ya臒谋z Aksoy, who leads the Computational Photography Lab at 911吃瓜. 鈥淚nstead of buying expensive lighting gear or reshooting scenes, they can make realistic lighting changes after the fact, without having to filter their creative vision through a generative AI model.鈥�
This paper is the latest in a series of 鈥渋llumination-aware鈥� research projects from the Computational Photography Lab. The group鈥檚 earlier work on intrinsic decomposition lays the groundwork for their new relighting method, and they break down how it all connects in their explainer video.
You can find out more about the Computational Photography Lab鈥檚 research on their .
Available 911吃瓜 Experts
Chris Careaga, PhD Student, Computing Science | chris_careaga@sfu.ca
Yagiz Aksoy, Assistant Professor, Computing Science | yagiz@sfu.ca