Source
- URL: https://x.com/RouaniJihad/status/1870960478572499239
- Author: 🕹️G🅰MESTUFFS🎮 (@RouaniJihad)
- Posted: 2024-12-22 22:31:53
Branch
1/ @RouaniJihad
@NOTimothyLottes
I’m not going to pretend I understand a lick of this, which is why I have a humble request, well 2 actually:
1- Is there a way for you to finish working on STP.
2- Can you create an open source denoiser for say, a real-time Path Tracer 👉👈
Reading this is enough as well… =]
2/ @NOTimothyLottes
@RouaniJihad
My recommendation: ask Unity to open source STP under a MIT license so any external person could finish it. And then both Unity and anyone else could benefit from it’s long term evolution.
3/ @NOTimothyLottes
@RouaniJihad
One fundamental problem with bespoke high-tech solutions like these are that in many cases they become abandonware when all the original people involved in the projects no longer work at the company (which is the case here)
4/ @NOTimothyLottes
@RouaniJihad
There are very few people who could work on it, and after Unity shed it’s high tech rendering staff during it’s downsizing efforts to stay live during it’s stock crash, who knows when they will have an environment that fosters that kind of R&D again
5/ @NOTimothyLottes
@RouaniJihad
As for STP it wasn’t ever designed to denoise per say, it does get a little denoising as a side effect of temporal filtering though, but things like stochastic GI on disocclusions will show a mismatch in noise with other parts of the image that get good temporal feedback
6/ @NOTimothyLottes
@RouaniJihad
The cases where people have merged scaling TAAs and denoisers, they do so by augmenting the TAA with extra side channel information from the G-buffer {normals, albedo, etc} and then keep a lot more recurrent data in feedback across frames, this massively increases costs
7/ @NOTimothyLottes
@RouaniJihad
Traditionally those who have managed to improve temporal feedback in cases of {rendered mirror reflections, SSR, or even RT} have needed to generate a section reprojection vector of where the mirrored reflection was in the prior frame, this is something Unity doesn’t have today
8/ @NOTimothyLottes
@RouaniJihad
There is another interesting option that a scaling-TAA typically tracks the amount of screen convergence, and one could pass that information back into an input denoiser before the TAA that denoises less where things have converged and a lot where things would be disoccluded
9/ @NOTimothyLottes
@RouaniJihad
Can watch videos like this https://www.youtube.com/watch?v=K3ZHzJ_bhaI … and you start to see that no one has actually come to a great solution for screen-space denoise+scaling+TAA, even given the ML solutions …
10/ @NOTimothyLottes
@RouaniJihad
Suggests maybe it would ultimately be better to push the RT usage back into the environment probe space and do the spatial temporal denoise there, and then use those probes to light the scene instead.