Articles / blog posts index

Categories

Image processing

General Machine Learning, Data Science and Analysis

Photography concepts in graphics

General graphics techniques

Low level / hardware / numerical analysis

Technical leadership, process, team management, tools

Photography

Personal projects / updates / misc / random

Note: In each category, post are sorted in reverse chronological order. I have marked posts that I think are particularly interesting and stood the test of time (by looking at stats, or using my subjective judgement) with bold and a star ⭐ .

Image processing

Bilinear texture filtering – artifacts, alternatives, and frequency domain analysis ⭐ April 2020 – bilinear filtering is “bread and butter” of all real-time computed graphics and one of the most commonly used filtering techniques ever – to large extent because of native GPU hardware acceleration. In this post I analyze star-shaped “bilinear artifacts”, shortcomings of the filter, some commonly used alternatives, and spectral (Fourier) properties of all of those.

Using JAX, numpy, and optimization techniques to improve separable image filters March 2020 – using “optimization” techniques and gradient descent in numpy and JAX to improve low-rank separable image filters (see below) and remove some of their visual artifacts (ringing, negative values, unpleasant corners).

Separate your filters! Separability, SVD and low-rank approximation of 2D image processing filters February 2020 – how to tell if an image processing 2D filter is separable, and finding separable approximations (up to a rank k – k multiple passes of separable filter) of some common filters commonly used in graphics, e.g. circular or hexagonal bokeh.

Local linear models and guided filtering – an alternative to bilateral filter September 2019 – how fitting linear models locally (through least-squares in a small window around a pixel) can be a great cheap alternative to bilateral filter – both for filtering a single image, but also for joint / guided bilateral, and bilateral upsampling. I show the technique using SSAO filtering example.

Checkerboard rendering, rotated anti-aliasing and grid frequencies May 2018 – description of what is “checkerboard rendering”, Fourier domain analysis of aliasing, reconstruction and why it often makes sense to rotate your sampling grid.

Separable disk-like depth of field August 2017 – a very short post where I came back from Siggraph and seeing  “Circularly symmetric convolution and lens blur” by Olli Niemitalo presented as “Circular Separable Convolution Depth of Field” by Kleber Garcia blew my mind so much that I had to implement it myself in Shadertoy for demonstration. 🙂

Machine Learning, Data Science and Analysis

Compressing PBR material texture sets with sparsity and k-SVD dictionary learning August 2020 – I look further into compressing whole PBR materials / texture sets (as opposed to single textures) and provide a gentle intro to dictionary learning, sparsity, and k-SVD.

Dimensionality reduction for image and texture set compression May 2020 – how you can use SVD and PCA dimensionality reduction and the correlation between channels for aggressively compressing whole sets of PBR textures/materials.

Analyze your own activity data using Google Takeout – music listening stats example January 2020 – how one can “take out” personal data from big internet service providers and then analyze it offline and create cool visual charts. Example on creating personal annual music listening stats for Google Play Music.

Photography concepts in graphics

Image dynamic range September 2016 – what is the dynamic range? What is gamma curve/compression, how does it relate to EOTF – Electro Optical Transfer Function? How do viewing conditions impact the dynamic range and why sRGB is different from Rec709? Clearing up many misconceptions and confusions around those topics.

Localized tonemapping – is global exposure and global tonemapping operator enough for video games? August 2016 – a blog post on why “global tonemapping” is not enough and games need either lighting “hacks”, or some sort of localized tonemapping. Combines some photography experiments, screenshots from God of War and introduction to general tonemapping theory.

White balance and physically based rendering pipelines. Part 1 – introduction. October 2015 – what is “white balance” as used in photography, and why it matters in real-time graphics. Effects of white balance on perceptual effects (gold/blue dress!), WB adaptation, and in some of the famous works of art. Part 2 discusses some problems that white balance can cause, some of the still unsolved! (e.g. partial / localized white balance).

Anamorphic lens flares and visual effects March 2015 – short post on where the (in)famous “JJ Abrams” anamorphic lens flares come from, how they are used in real time graphics, and how one can achieve such an effect.

Bokeh depth of field – going insane! part 1 April 2014 – A blog post where I describe what is “bokeh”, why we care about it in graphics / computational photography, and an “insane” (as in computationally expensive and unreasonable for those times) approach I used in The Witcher 2.

General graphics techniques

“Optimizing” blue noise dithering – backpropagation through Fourier transform and sorting April 2020 – Using optimization / backpropagation and numpy/JAX/colab to generate “blue noise” dithering patterns and matrices directly – highly customizable approach in which we design a loss function and run gradient descent on it.

Cull that cone! Improved cone/spotlight visibility tests for tiled and clustered lighting April 2017 – a very simple, pretty efficient “hybrid” method of testing geometric cone (used for e.g. spotlights, but also potentially AI) intersection suitable for optimizing real-time tiled lighting. Includes some pretty sweet visualizations and is linked quite a bit.

Dithering in games – mini series October 2016 – three post series on how dithering can and should be used in video games – why it matters, how it can improve the visual results, what is Golden ratio based low discrepancy sequence, what is blue noise – includes numerical, visual and Fourier-domain comparisons. Quite popular introductory material.

Fixing screen-space deferred decals Mach 2015 – problem caused by the texture derivatives (used to compute mip-map levels) in the case of analytical/procedural effects that convert from screen to texture/world space, like screen-space deferred decals, and how one can solve them.

Designing a next-generation post-effects pipeline December 2014 – a blog post on the post-processing (camera effects, tonemapping, bloom, blurs, but also particle rendering) pipeline that I designed for Far Cry 4 (and later used almost exactly same one on God of War) with the goal of unifying/generalizing all those effects together for fast, half resolution processing.

Hair rendering trick(s) July 2014 – a few multi-pass tricks to render character hair (dense, includes transparent parts, as well as complicated BRDFs) on older hardware. A technique we played with for The Witcher 2/3 games.

Temporal supersampling pt. 2 – SSAO demonstration April 2014 – where I extend the blog post about temporal anti-aliasing to the actual super-sampling part, with some examples from Assassin’s Creed 4. Little did I know at that time, that soon everyone will be using temporal super-sampling for everything, and that many years later I will lead a project at Google that does multi-frame super-resolution for camera zoom.

GDC follow-up: Screenspace reflections filtering and up-sampling March 2014 – quite (out)dated post now on how one can filter and upsample screen-space reflections. See instead my guided filtering post!

Temporal supersampling and antialiasing March 2014 – most popular blog post of mine! Introduction to temporal techniques, timed “perfectly” – around when it switched from being an experimental technique, to contemporary ubiquitous state of the art. It’s not very relevant anymore, as state of the art is much better than that, but shows a fun piece of the graphics history. 🙂

The future of screenspace reflections January 2014 – a blog post about screen-space reflections I implemented for Assassin’s Creed 4: Black Flag, in hindsight probably quite lame (describing one of the earliest approaches to solving the problem), but conversely in hindsight also quite “visionary” post, as it directly precedented the emergence of a huge SSR hype – to the point that people were joking that one of the main differentiators of PS4/XboxOne games were screen-space reflections and shiny surfaces. 🙂

Low level / hardware / numerical analysis

Small float formats – R11G11B10F precision April 2017 – description of some “small float” formats (IEEE floating point with less than 32 bits per single channel) – with main focus on 3 channel, R11G11B10F GPU hardware-accelerated float format – analysis of numerical precision, problems how to avoid color banding, comparison to 8 and 10bit sRGB curves. Highly recommended, as I dispel some myths that are still alive – e.g. that pre-exposure can improve color precision (with floats, it won’t!).

GPUs and GCN – two ways of latency hiding and wave occupancy March 2014 – description of how modern GPUs hide latency – essentially the main difference between their, and CPU memory model. I recommend it to anyone who ever considers writing GPU code, and I include some assembly analysis based on AMDs popular GCN GPU architecture.

Technical leadership, process, team management, tools

How (not) to test graphics algorithms August 2019 – how to test graphics techniques? Why “golden testing” and end to end testing is – in my personal opinion – often a terrible idea? How to incorporate graphics testing in your programming and team workflows.

Tech and scientific writing – graphics, diagram and graph creation tools October 2017 – a survey of what other graphics programmers use for creating diagrams, graphs and visualizations (spoiler: everyone uses something different! 🙂 but I still give some recommendations).

Short names are short September 2016 – Opinionated post on programming variable and function naming and why I think that short names are not great for code readability / understandability, and there is no place for them in the XXI century.

Technical debt… or technical weight? June 2016 – a blog post on “technical weight” or “technical burden”, which is not the same as “technical debt”. Sometimes great, clean solutions can slow you and your progress down, and it can be better to do things in more crude, simple, or even hacked way if it allows for faster long term progress.

On pursuit of (good) free mathematics toolbox and Python as scientific toolbox – 8 months later September 2014 – old posts on how I was trying to find a good free / open-source replacement for Mathematica. Still kind-of-relevant, as my recommendation of using numpy and Jupyter notebooks still stands, but this time the choice is much simpler – use Google Colab!

Runtime editor-console connection in The Witcher 2 May 2014 – An idea / tooling we used on Witcher 2 for real-time connection between game editor, and the console, allowing for real-time editing and preview on the target platform (Xbox 360 at that time). I believe now it is quite “standard” in all engines, but was novel to me at that time.

Compare it! February 2014 – Importance of having reference implementation and comparing against it. Might seem “obvious” now, but it wasn’t so obvious to most of my colleagues back then and I believe still isn’t a common practice among most games graphics programmers!

Why big game studios (usually) use single main 3d software environment? February 2014 – I got asked why most game studios force a single main editing program (like Maya and 3D Studio Max) and I provide probably all the possible answers – there are many reasons!

Photography

Processing scanned/DSLR photos of film negatives in Lightroom February 2015 – How to process scanned film negatives (and turn them to positives), without leaving convenient Adobe Lightroom.

Voigtlander Nokton Classic 40mm f1.4 M on Sony A7 Review August 2014 – Review of a classic-like, manual focus beautiful lens perfect for a mirrorless digital camera. Interestingly, a post that seems “random” to my blog, but is also the most popular I ever wrote! 🙂

Coming back to film photography June 2014 – how I tried to get back to film photography (I have a long, personal relationship with it), some photos, and thoughts about it. In hindsight, my attempt to get back to it didn’t succeed heh.

Sony A7 review July 2014 – a quick review of my first mirrorless camera.

Personal projects / updates / misc / random

New debugging options in CSharpRenderer framework and CSharpRenderer Framework update October 2014 – Some updates to my CSharpRenderer, in hindsight has a few cool, usable ideas for debugging, as well as “bilateral vectors” – optimization for precomputing optimal UVs for depth-guided bilateral upsampling ahead of time.

Review: “Multithreading for Visual Effects”, CRC Press 2014 September 2014 – a review of a book about multi-threading use in the VFX industry.

C#/.NET graphics framework on GitHub + updates June 2014 – update post, but includes IMO a cool and somewhat novel idea -> “scriptable” constant / uniform buffers with procedural, scripted rules describing how they are derived. Allows for coupling together all shader logic in a single file, while still allowing for CPU pre-computation and optimization.

C#/.NET graphics framework April 2014 – Description of my own, open-source DX11 graphics prototyping framework, motivations, some implementations details and ideas that can be incorporated in graphics engines.

My first visit to Cuba January 2014 – my first blog post ever! About my personal travel to Cuba.