Source
- URL: https://x.com/no1zex/status/1216882677476884480
- Author: Kamil N (@no1zex)
- Posted: 2020-01-14 00:40:13
Thread
1/ @no1zex
@EricLengyel
trying to use the structure buffer described in FGED2 and having big troubles with the eye space depth storage.. I’m using OpenGL but should not be much different, yet I get a very visible banding when visualising buffer (and also problems in SSAO and other techs)
2/ @no1zex
@EricLengyel
Any idea what could be wrong there? I’m using GL counterparts to convert between float<>uint, splitting it with a mask then reconstructing, but the banding persists - I spent week on this and I’m running out of ideas.. the depth I use is: -(viewMatrix * (modelMatrix * vertPos)).z
3/ @no1zex
@EricLengyel
4/ @EricLengyel
@no1zex
Please try using
float eyeDepth = 1.0 / gl_FragCoord.w;
just to make sure you’ve got the right values. This is what I use in the Tombstone Engine.
5/ @no1zex
@EricLengyel
Ah, I used it before and just today I converted it to the formula I posted above - I wasn’t 100% sure I’m using the right output. So I can confirm that I get exact same results when using 1.0 / gl_FragCoord.w :/ Something else must be off - vectorScale calculation?
6/ @no1zex
@EricLengyel
7/ @no1zex
@EricLengyel
Filled it with intermediate results:
8/ @EricLengyel
@no1zex
Hmm. Your depth image looks right to me, so I would think the problem is completely in the SSAO part. For reference, here are some quick captures I just made. The first shows the depth in the structure buffer, and the second shows the values in the occlusion buffer.
9/ @EricLengyel
@no1zex
10/ @EricLengyel
@no1zex
I would characterize what I see in your occlusion image more as “streaking” instead of “banding” because it looks like the values are inaccurate as opposed to merely imprecise. Let me look a little closer, and I’ll see if I can come up with some things to check…
11/ @no1zex
@EricLengyel
Sure, thanks a lot! I can provide debug outputs from any value in SSAO shader, but it mainly breaks in the sampling of one of 4th neighbours. And like I mentioned, changing 1.0/scale to 1.0/z0 actually gives me any results, instead of very incorrect artifacts
12/ @no1zex
@EricLengyel
Also worth mentioning that what you see is using the scale = 1.0 / z0, which is different than in books. When I use scale - 1.0 / scale (line after normal calculation) it breaks really bad, I’ll show video
13/ @no1zex
@EricLengyel
14/ @no1zex
@EricLengyel
The above is with
15/ @EricLengyel
@no1zex
I checked your value of vectorScale, and it is fine. That’s not the problem.
16/ @no1zex
@EricLengyel
What is really interesting is that the output from SSAO changes really abruptly when moving camera while using scale=1/scale line, the calculated v vector for the sample is also very unstable, changing with every camera movement.. I assume the pixelCoord is just 0..1 frag uv?
17/ @EricLengyel
@no1zex
No, pixelCoord is gl_FragCoord.xy, which ranges over the resolution of the render target at pixel centers. So the x coordinate should be 0.5 through 1679.5, for example.
18/ @EricLengyel
@no1zex
Your shader code is essentially identical to what I use in the Tombstone Engine. I don’t see a problem there.
19/ @no1zex
@EricLengyel
Really weird. But when I use gl_FragCoord I only get blinking screen which changes values for whole render target. So basically pixelCoord = gl_FragCoord.xy
20/ @no1zex
@EricLengyel
I suppose maybe it assumes the texture is in repeat mode?
21/ @EricLengyel
@no1zex
Just to make sure, your structure buffer and occlusion buffers are all being treated as GL_TEXTURE_RECTANGLE textures, right?
22/ @no1zex
@EricLengyel
Ouch.. no, that’s just GL_TEXTURE_2D… I guess I missed some information somewhere then, I’ll quickly write support for this and see how it behaves with GL_TEXTURE_RECTANGLE instead, will get back to you in few minutes.
23/ @no1zex
@EricLengyel
After enabling TEXTURE_RECTANGLE for structure buffer and mapping to [0, 1] range
24/ @EricLengyel
@no1zex
That looks better. (Using rectangle textures is just a lot easier, IMO. You could use 2D textures, but you’d have to divide the texcoords by the render target dimensions each time you accessed the structure buffer.)
25/ @no1zex
@EricLengyel
Ok took me a while as I didn’t have Rect support in my engine… now it looks a bit more stable though the occlusion is uglier than it was, especially when zoomed out a bit:
26/ @no1zex
@EricLengyel
Samplers for these RECTANGLE textures should be GL_NEAREST and GL_CLAMP_TO_EDGE yes?
27/ @no1zex
@EricLengyel
Hmm, got this… looks a bit more like it but still something is off…
28/ @no1zex
@EricLengyel
29/ @no1zex
@EricLengyel
ok, I broke sampling rotations when I changed everything to RECT and used NEAREST instead of LINEAR which helps with blurring… now it looks pretty good!
30/ @no1zex
@EricLengyel
although I still can see some artifacts sneaking in when I zoom out above the castle - you can see the “streaks of “occlusion” that should not be there
31/ @EricLengyel
@no1zex
Looking much better!
To clarify about the rotation texture — You do want that to be 2D, and it will be repeated a lot so that every little 4x4 neighborhood gets the same pattern.
The streaks in the far away shots definitely are not right.
32/ @no1zex
@EricLengyel
Yeah they’re way less invasive than before, and when actually used as final ambient factor they appear as a slightly darker “band” from far away - I’d really like to find what’s causing this. But after switching to rectangle textures it works much more stable and better
33/ @no1zex
@EricLengyel
Thanks for all the help, much appreciated - I would never figure out that switching to rectangle textures can help. Somehow I missed the “rect” mentions in book shaders, or considered this irrelevant as I never used rectangle textures before.
34/ @no1zex
@EricLengyel
from very far away these get much more visible
35/ @EricLengyel
@no1zex
Unfortunately, I don’t have an ideas at the moment about what might be causing those streaks. I’ll let you know if anything pops into my head.
36/ @EricLengyel
@no1zex
This kind of problem is very typical of computer graphics development. Sometimes you just have to stare at it for a while and think about every little detail until you finally stumble upon the answer. Once in a while, I’ll be stumped for days at a time over a simple oversight.
37/ @no1zex
@EricLengyel
Oh yeah, it is :) I think, after looking at all possible ways to debug it and trying things, that the vectorScale is slightly wrong. I tweaked it for radians, for degrees and there are cases where this almost works correctly, without banding.
38/ @no1zex
@EricLengyel
Unfortunately the full equation for vectorScale was never derived in full, could you maybe show how you calculate it in your engine, doesn’t have to be more than the calculation itself + units used (radians/degrees etc if anything like this matters)
39/ @no1zex
@EricLengyel
40/ @EricLengyel
@no1zex
Looking back at your calculation, your fovy calculation is missing some factors of 2, but you don’t really need that. Using Equation (6.1), just calculate g from fovx:
g = aspectRatio * tan(fovx * 0.5);
If fovx = 90 degrees, then g = aspectRatio.
41/ @EricLengyel
@no1zex
I calculate vectorScale as 2.0 / (screenSize.y * g). This is the same thing you’re doing since screenSize.y = screenSize.x / aspectRatio.