Fast and High-quality Image Denoising via Malleable Convolutions
To achieve spatial-varying processing without significant overhead, we present Malleable Convolution (MalleConv), as an efficient variant of dynamic convolution. The weights of MalleConv are dynamically produced by an efficient predictor network capable of generating content-dependent outputs at specific spatial locations. Unlike previous works, MalleConv generates a much smaller set of spatially-varying kernels from input, which enlarges the network’s receptive field and significantly reduces computational and memory costs. These kernels are then applied to a full-resolution feature map through an efficient slice-and-conv operator with minimum memory overhead.
Procedural Kernel Networks
B. Wronski, arXiv preprint / technical report
In this work, we introduce Procedural Kernel Networks (PKNs), a family of machine learning models which generate parameters of image filter kernels or other traditional algorithms. A lightweight CNN processes the input image at a lower resolution, which yields a significant speedup compared to other kernel-based machine learning methods and allows for new applications. The architecture is learned end-to-end and is especially well suited for a wide range of low-level image processing tasks, where it improves the performance of many traditional algorithms. We also describe how this framework unifies some previous work applying machine learning for common image restoration tasks.
Image Stylization: From Predefined to Personalized, IET Research Journal 2020
https://arxiv.org/abs/2002.10945 Arxiv preprint
We present a framework for interactive design of new image stylizations using a wide range of predefined filter blocks. Both novel and off-the-shelf image filtering and rendering techniques are extended and combined to allow the user to unleash their creativity to intuitively invent, modify, and tune new styles from a given set of filters.
Handheld Multi-Frame Super-Resolution, ACM Siggraph 2019 Technical Paper
We present a multi-frame super-resolution algorithm that supplants the need for demosaicing in a camera pipeline by merging a burst of raw images. In the above figure we show a comparison to a method that merges frames containing the same-color channels together first, and is then followed by demosaicing (top). By contrast, our method (bottom) creates the full RGB directly from a burst of raw images. This burst was captured with a hand-held mobile phone and processed on the device. Note in the third (red) inset that the demosaiced result exhibits aliasing (Moiré), while our result takes advantage of this aliasing, which changes on every frame in the burst, to produce a merged result in which the aliasing is gone but the cloth texture becomes visible.
Volumetric fog: Unified, compute shader based solution to atmospheric scattering, ACM Siggraph 2014
Bartlomiej Wronski, ACM Siggraph 2014
PPTX Version – 83MB (with movies)
This talk presents “Volumetric Fog”, a novel technique developed by Ubisoft Montreal for Assassin’s Creed 4: Black Flag for next-gen consoles and PCs.
The technique addresses problem of calculating in unified, coherent and optimal way various atmospheric effects related to the atmospheric scattering:
- Fog, smoke and haze with varying participating media density
- „God rays”
- Volumetric lighting and shadows
Developed technique supports varying density of participating media, multiple light sources, is compatible with both deferred and forward shading and is faster than existing ray marching approaches.
Assassin’s Creed 4: Road to Next-gen Graphics, GDC 2014
Bartlomiej Wronski, GDC 2014
PDF Version – 4.11MB
PDF Version with presenter notes – 9.63MB
PPTX Version with the movies – 164MB
Movies separate download
This talk will describe the novel techniques and easy-to-integrate effects of Assassin’s Creed IV that contribute to the next-gen look. It will provide attendees with basic information about porting various GPU effects to next-gen consoles. The talk is divided into four parts. First, there will be a description of deferred normalized irradiance probes – a GI technique that is based on FarCry 3 deferred radiance transfer volumes. The next part will describe volumetric fog – a novel technique that simulates various light scattering phenomena through the use of compute shaders and light accumulation in volumetric textures. The third part will include information about atmospheric and material effects such as GPU-simulated rain particles and raymarched screenspace reflections that can easily contribute to the next-gen look of the game without many content or pipeline changes. Finally, there will be a brief description of AMD Southern Islands architecture and practical lessons learned while developing/porting/optimizing those GPU effects to PlayStation 4 and Xbox One.
Assassin’s Creed 4: Lighting, weather and atmospherics, Digital Dragons 2014
Bartlomiej Wronski, Digital Dragons 2014
PPTX Version, 226MB – but worth it (tons of videos!)
GPU Pro 6: Advanced Rendering Techniques
A K Peters/CRC Press
Published September 11, 2015
Reference – 586 Pages – 279 Color Illustrations
ISBN 9781482264616 – CAT# K24427
Deferred Normalized Irradiance Probes, John Huelin, Benjamin Rouveyrol, and Bartlomiej Wronski
Volumetric Fog and Lighting, Bartlomiej Wronski