Occlusion culling is really tough in systems where users can add content to the world. Especially if there's translucency. As with windows (not Windows), or layered clothing.
You're in a room without windows. Everything outside the room is culled. Frame rate is very high. Then you open the door and go outside into a large city. Some buildings have big windows showing the interior, so you can't cull the building interior. You're on a long street and can look off into the distance. Now keep the frame rate from dropping while not losing distant objects.
Games with fixed objects can design the world to avoid these situations. Few games have many windows you can look into. Long sightlines are often avoided in level design. If you don't have those options, you have to accept that distant objects will be displayed, and level of detail handling becomes more important than occlusion. Impostors. Lots of impostors.
Occlusion culling itself has a compute cost. I've seen the cost of culling big scenes exceed the cost of drawing the culled content.
This is one of those hard problems metaverses have, and which, despite the amount of money thrown at the problem, were not solved during the metaverse boom. Meta does not seem to have contributed much to graphics technology.
Dynamic occlusion culling is pretty common these days now that the GPU can do its own filtering of the draw list. I think it goes like:
Start with a list of objects that were visible last frame. Assume they are visible this frame. Go ahead and draw them.
Then, for the list of things that were Not visible last frame, draw bounding box queries to build a conservative list of objects that might actually not be occluded this frame. This is expected to be a short list.
Take that short list and draw the probably-newly-visible objects.
Have queries attached to all of those draws so you end up with a conservative list of all of the actually-visible objects from this frame.
This obviously has problems with camera cuts. But, it works pretty well in general with no preprocessing. Just the assumption that the scene contents aren’t dramatically changing every frame at 60 FPS.
I was thinking about this problem a few days ago, imagining a semi-online game where players could create a collective city by plotting buildings. The "grid" would be some kind of pre-determined voronoi pattern, in theory making occlusion culling easier.
If the main goal is to limit sight lines, there are a lot of potential tessellations that will give that effect, while also being predictable and infinitely repeatable. For example, imagine a regular city grid, except every roads is a wavy S-curve.
I bet there is a special math term for "tilings that do/don't contains infinite lines", but I wasn't able to find it quickly. It's not the same as (a)periodic, since a periodic tiling could block the lines, and an aperiodic tiling could have one giant seam down the middle.
I've always wondered to what extent these culling techniques still work with raytracing?
A reflective surface can bring a bunch of otherwise-offscreen things into the scene. Its what makes screen-space reflections look so bad sometimes, they can't reflect whats not on-screen.
I'm remember a Tiny Glade talk where they explained that they did reflections by first doing a screen-space pass, and then a ray-tracing pass for all the pixels that didn't get a "hit" in screen space (as in, the reflection needed to show something offscreen in that pixel).
This and with reflection you have tiny different representation of your world.
RTX has TLAS and BLAS for ray traversing, your own tracing can be based on own BVH acceleration structure and SDF form of the world. So you right, you can't properly cull the world but you can have optimized version by having
1. Nice acceleration structure (hardware TLAS BLAS or software BVH/Octree (iirc octree cache-unfriendly))
2. Simple material form representation to sample
3. Short rays
4. Initial simplified rays result like screen space reflection
5. Half screen (less rays) and temporal accumulation (even more less rays!)
I really appreciate this post. It reminds me of a video I watched a couple years ago that does an excellent job of demonstrating how culling works with actual code and visuals
PVS isn't that expensive to compute. Especially nowadays. I assume this is actually referring to the binary space partitioning techniques used in DOOM and improved in Quake, Half-Life, etc in the late 90s, early 2000s.
The BSP tree was also extremely useful for optimizing netcode for games like Quake 3 Arena and games within that family and time period I believe.
Lots of old racing games used a PVS. The way it was calculated was by flying a camera around the track and calculating what was visible every few meters using occlusion culling, then at runtime you just see what section of track you're on and only draw the objects that you know are visible. Recalculating the PVS for that is definitely a slow operation!
PVS requires some hierarchical scene representation with no seams between walls. I know no other way to build such representation other than BSP. But BSP works fine only with pretty low-detail map geometry consisting of brushes. No large detail meshes or terrains can be used with it. If a game has a lot of open spaces or semi-open spaces it's nearly to impossible to build a BSP for it.
In practice many useful representation can be built only in a hierarchical way. Unless you want to force artist/map makers to split their maps in regions manually.
All Source / Source 2 games still use both PVS (bsp/octree) and pre-baked lightmaps. Of course, they’re quite notorious for the staticness of their environments.
There's nothing necessarily static about BSPs - in fact level editors use BSP brushes (other piecces of geometry stored as BSP) - exactly because calculating set operations between BSPs is fast and well defined, and results in another BSP.
In fact, Red Faction, one of the first games to feature destruction on the PS2, used BSPs under the hood instead of voxels/marching cubes.
What was expensive, and made these game levels static is, was the occlusion calculation and light maps, which were relatively expensive at the time, and the fact that the first instances of those engines, like Quake were designed for the Pentium.
Backface culling has been common since the late 1990s when we started using face normals to determine lighting rather than per-vertex lighting. Pretty much every 3D game engine since about 2004 has included and enabled it by default. How is it that you made a game that doesn't use it?
Ahhh. So you used a wrapper or a library? Interesting then. I had assumed that almost every rendering method enables frustrum, occlusion, and backface culling by default if only to clear the number of objects needed to be tracked in memory. One thing I noticed in your game is that it's based on the absolute mouse position, which with a 16:9 window makes it difficult to turn in certain situations because your horizontal movement space is much larger than the vertical movement space and that adversely affects turning speed. Changing so that is based just on horizontal mouse movement or adding keyboard controls might be better.
For the curious readers, backface culling (at least in the way fixed-function OpenGL does it, and probably newer APIs still do) is not based on face normals, it's based on winding order of triangles, so it works even if normals are not used.
Also face normals (flat shading) are generally considered older tech than per-vertex lighting (Gouraud shading). Newer stuff since 2008-ish is generally per-pixel using normal maps for detail.
If I remember my graphics accelerator history correctly per-vertex lighting using Gouraud shading was the method SGI made standard with OpenGL in 1992, before 3DFX and ATI came in and made per-pixel via Blinn-Phong just as computationally efficient in about 2000 or 2001. Used before Gouraud and then alongside per-vertex and per-pixel was per-face flat shading, which kept being used up until the Xbox 360/PlayStation 3/DirectX 10 era. Face normals for per-pixel started being used in 1999 by SEGA, but stopped being common around the same time as normals flat shading got abandoned.
If I'm wrong please feel free to correct any of this, it's been about eight years since I last learned all of the different methods.
Back in the 90s I made a 3d engine (software renderer) and used frustum culling. But computing the frustum intersection every time was too slow. So one technique I did was add a count to each polygon. If the polygon was outside the view frustum, i added a count of N frames. Each frame if the count for a polygon was 0 it would check against the frustum, otherwise it would reduce the count and skip the polygon rendering entirely.
This worked very well but of course if the camera turned quickly, you would see pop-in. Not a modern technique, but an oldschool one.
Occlusion culling is really tough in systems where users can add content to the world. Especially if there's translucency. As with windows (not Windows), or layered clothing.
You're in a room without windows. Everything outside the room is culled. Frame rate is very high. Then you open the door and go outside into a large city. Some buildings have big windows showing the interior, so you can't cull the building interior. You're on a long street and can look off into the distance. Now keep the frame rate from dropping while not losing distant objects.
Games with fixed objects can design the world to avoid these situations. Few games have many windows you can look into. Long sightlines are often avoided in level design. If you don't have those options, you have to accept that distant objects will be displayed, and level of detail handling becomes more important than occlusion. Impostors. Lots of impostors.
Occlusion culling itself has a compute cost. I've seen the cost of culling big scenes exceed the cost of drawing the culled content.
This is one of those hard problems metaverses have, and which, despite the amount of money thrown at the problem, were not solved during the metaverse boom. Meta does not seem to have contributed much to graphics technology.
This is much of why Second Life is slow.
Dynamic occlusion culling is pretty common these days now that the GPU can do its own filtering of the draw list. I think it goes like:
Start with a list of objects that were visible last frame. Assume they are visible this frame. Go ahead and draw them.
Then, for the list of things that were Not visible last frame, draw bounding box queries to build a conservative list of objects that might actually not be occluded this frame. This is expected to be a short list.
Take that short list and draw the probably-newly-visible objects.
Have queries attached to all of those draws so you end up with a conservative list of all of the actually-visible objects from this frame.
This obviously has problems with camera cuts. But, it works pretty well in general with no preprocessing. Just the assumption that the scene contents aren’t dramatically changing every frame at 60 FPS.
Roblox just gave a talk at this exact intersection of topics (user generated content and high performance culling) last month at GDC.
https://schedule.gdconf.com/session/optimizing-a-large-time-...
Yes, although the GDC paper is paywalled.
But see Roblox's own summary at: [1]
[1] https://devforum.roblox.com/t/occlusion-culling-now-live-in-...
I was thinking about this problem a few days ago, imagining a semi-online game where players could create a collective city by plotting buildings. The "grid" would be some kind of pre-determined voronoi pattern, in theory making occlusion culling easier.
If the main goal is to limit sight lines, there are a lot of potential tessellations that will give that effect, while also being predictable and infinitely repeatable. For example, imagine a regular city grid, except every roads is a wavy S-curve.
I bet there is a special math term for "tilings that do/don't contains infinite lines", but I wasn't able to find it quickly. It's not the same as (a)periodic, since a periodic tiling could block the lines, and an aperiodic tiling could have one giant seam down the middle.
I've always wondered to what extent these culling techniques still work with raytracing? A reflective surface can bring a bunch of otherwise-offscreen things into the scene. Its what makes screen-space reflections look so bad sometimes, they can't reflect whats not on-screen.
I'm remember a Tiny Glade talk where they explained that they did reflections by first doing a screen-space pass, and then a ray-tracing pass for all the pixels that didn't get a "hit" in screen space (as in, the reflection needed to show something offscreen in that pixel).
This and with reflection you have tiny different representation of your world. RTX has TLAS and BLAS for ray traversing, your own tracing can be based on own BVH acceleration structure and SDF form of the world. So you right, you can't properly cull the world but you can have optimized version by having 1. Nice acceleration structure (hardware TLAS BLAS or software BVH/Octree (iirc octree cache-unfriendly)) 2. Simple material form representation to sample 3. Short rays 4. Initial simplified rays result like screen space reflection 5. Half screen (less rays) and temporal accumulation (even more less rays!)
I always wonder about this IRL...I'm at work rn, is my apartment still rendered?
Unless it's being observed externally it is in a state of quantum uncertainty, I think.
have to ask the Maintainer of Time
https://twilightzone.fandom.com/wiki/A_Matter_of_Minutes
https://www.youtube.com/watch?v=e9XFRfeGBVI
The old philosophical question remains, "If a tree falls in a forest and no one is around to hear it, does it make a sound?" [1]
[1] https://en.wikipedia.org/wiki/If_a_tree_falls_in_a_forest_an...
Yes, the other trees hear it. Easy :)
> Quake made PVS famous. It’s still useful in some indoor games where the scene geometry is static and bake time is acceptable.
It was used extensively in outdoor games like Jak and Daxter.
I really appreciate this post. It reminds me of a video I watched a couple years ago that does an excellent job of demonstrating how culling works with actual code and visuals
https://www.youtube.com/watch?v=CHYxjpYep_M
PVS isn't that expensive to compute. Especially nowadays. I assume this is actually referring to the binary space partitioning techniques used in DOOM and improved in Quake, Half-Life, etc in the late 90s, early 2000s.
The BSP tree was also extremely useful for optimizing netcode for games like Quake 3 Arena and games within that family and time period I believe.
Lots of old racing games used a PVS. The way it was calculated was by flying a camera around the track and calculating what was visible every few meters using occlusion culling, then at runtime you just see what section of track you're on and only draw the objects that you know are visible. Recalculating the PVS for that is definitely a slow operation!
PVS requires some hierarchical scene representation with no seams between walls. I know no other way to build such representation other than BSP. But BSP works fine only with pretty low-detail map geometry consisting of brushes. No large detail meshes or terrains can be used with it. If a game has a lot of open spaces or semi-open spaces it's nearly to impossible to build a BSP for it.
PVS does not require a hierarchical representation. You can use any representation you want. In fact the one in the article itself is not hiearchical.
In practice many useful representation can be built only in a hierarchical way. Unless you want to force artist/map makers to split their maps in regions manually.
All Source / Source 2 games still use both PVS (bsp/octree) and pre-baked lightmaps. Of course, they’re quite notorious for the staticness of their environments.
There's nothing necessarily static about BSPs - in fact level editors use BSP brushes (other piecces of geometry stored as BSP) - exactly because calculating set operations between BSPs is fast and well defined, and results in another BSP.
In fact, Red Faction, one of the first games to feature destruction on the PS2, used BSPs under the hood instead of voxels/marching cubes.
What was expensive, and made these game levels static is, was the occlusion calculation and light maps, which were relatively expensive at the time, and the fact that the first instances of those engines, like Quake were designed for the Pentium.
Is portal culling still used today? I thought it's an old technic used only by some very old games like Thief.
Love this, I will now use backface culling for my game:
https://slitherworld.com
Backface culling has been common since the late 1990s when we started using face normals to determine lighting rather than per-vertex lighting. Pretty much every 3D game engine since about 2004 has included and enabled it by default. How is it that you made a game that doesn't use it?
I didn't use a game engine
Ahhh. So you used a wrapper or a library? Interesting then. I had assumed that almost every rendering method enables frustrum, occlusion, and backface culling by default if only to clear the number of objects needed to be tracked in memory. One thing I noticed in your game is that it's based on the absolute mouse position, which with a 16:9 window makes it difficult to turn in certain situations because your horizontal movement space is much larger than the vertical movement space and that adversely affects turning speed. Changing so that is based just on horizontal mouse movement or adding keyboard controls might be better.
Thanks for the feedback, I’ll try to get that sorted out.
For the curious readers, backface culling (at least in the way fixed-function OpenGL does it, and probably newer APIs still do) is not based on face normals, it's based on winding order of triangles, so it works even if normals are not used.
Also face normals (flat shading) are generally considered older tech than per-vertex lighting (Gouraud shading). Newer stuff since 2008-ish is generally per-pixel using normal maps for detail.
If I remember my graphics accelerator history correctly per-vertex lighting using Gouraud shading was the method SGI made standard with OpenGL in 1992, before 3DFX and ATI came in and made per-pixel via Blinn-Phong just as computationally efficient in about 2000 or 2001. Used before Gouraud and then alongside per-vertex and per-pixel was per-face flat shading, which kept being used up until the Xbox 360/PlayStation 3/DirectX 10 era. Face normals for per-pixel started being used in 1999 by SEGA, but stopped being common around the same time as normals flat shading got abandoned.
If I'm wrong please feel free to correct any of this, it's been about eight years since I last learned all of the different methods.
Thanks for clarifying, so I guess I already have it on then.
Great post. Looking forward to the followup article about lights and shadows :)
Very good read and visualizations, thank you for writing it
Back in the 90s I made a 3d engine (software renderer) and used frustum culling. But computing the frustum intersection every time was too slow. So one technique I did was add a count to each polygon. If the polygon was outside the view frustum, i added a count of N frames. Each frame if the count for a polygon was 0 it would check against the frustum, otherwise it would reduce the count and skip the polygon rendering entirely.
This worked very well but of course if the camera turned quickly, you would see pop-in. Not a modern technique, but an oldschool one.
Dooope!