The code is open-sourced here: https://github.com/EarlywormTeam/qckfx-ads.
The coolest aspects of this was the work I did on ComfyUI workflows to try to get the companies’ products to appear correctly in the generated images. If you look at the code you’ll see tons of iterations on Comfy workflows over the duration of the product.
At first I had a goal to get the image model to properly draw the companies’ products in the first image. I had several corrective steps after that to clean up misdrawings. This actually worked a lot better than I thought thanks to the impressive fine-tuning ability of the flux models.
However the size and shape of the products was not very deterministic and I could not figure out how to fix this besides just generating a ton of images and hoping that one turned out okay.
So I switched to trying to get the model to draw a placeholder product of about the correct proportion and then essentially paste in the actual product image and use inpainting to clean up around it. I believe this is the approach many other companies are using. It’s quite challenging to get things to blend correctly and error prone if the initial placeholder product dimensions are far off from the actual product.
I tried tons of other approaches, but couldn’t find anything that worked super reliably. I think at this point the image models have improved and some providers are probably already offering this kind of functionality on their platforms.
NOTE: The product images below were results of experiments with the code and this is not meant to represent the brand pictured in any way shape or form. It’s just a demonstration of the application applied to real world products.
