Expanding Light Sources with Generative AI in Photoshop

Independent Research and Practice Project

Marie Klischat

5/3/2025

For the video “A Feeling of Anger and Rebellion,” I had to remove part of the background. Due to limited space during filming, the video footage always included a portion of the room that was not supposed to appear in the final shot.

In general, AI-based background removal was already quite effective at the time of the experiment (as of March 2025), particularly for static video footage. In my case as well, the camera was stationary. However, a significant limitation arose due to the non-static light source, which caused the lighting conditions to change from frame to frame.

An initial approach involved enhancing the video using generative video AI, specifically Runway AI’s Inpaint tool, which was one of the most popular tools for video-based AI editing at the time. However, due to the changing lighting conditions, this approach did not produce consistent results, making it unsuitable for the project.

So I decided to do it manually, frame by frame, in Photoshop. To do this, I selected the area of the image I wanted to change and generated the new background using a prompt. Although this workflow worked in principle, it was extremely time-consuming.

Frame mit störendem Hintergrund.

Frame mit KI-Überarbeitung.

Video that has been edited frame by frame.

For these reasons, I decided not to use this technique in the final project and instead to document the process as a research experiment. The work serves as an example of the limitations of AI-assisted video editing in March 2024, particularly with regard to lighting consistency and stability across frames.

To speed up the process, I also experimented with a Photoshop script that at least automated the process of regenerating a background for the selection. However, this approach also had significant limitations: visible transitions in the image, hard edges, and the occasional, random addition of unwanted objects by the AI.

Ultimately, it was necessary to create a custom selection for each frame and adjust the prompt again to achieve the smoothest possible result. Despite this significant effort, there was still some slight flickering in the video that could not be completely eliminated.

+491708232303

MARIE.KLISCHAT{at}PROTONMAIL.COM

Legal Notice

Contact

50937 Köln