Title: Tutorial 4: 2D OpenGL rendering and visualization

Integration with 2D OpenGL rendering and visualization: the GLASS effect

In this tutorial, we demonstrate how to add a simple glass effect to an image: we draw some text over an input image, with a glass-like material. The tutorial demonstrates some graphical capabilities of Quasar as well as the writing of a simple kernel function that is executed in parallel on the GPU.

First, we open an input image, which can be achieved using the function imread:

im = imread("lighthouse.png")

Then, we prepare an image with the text “GLASS”. For this purpose, we create a special qvectorlayer object (which can be regarded as a vector layer in photo editing programs like Adobe Photoshop or Corel Paint Shop Pro), and we render the text “GLASS” to it (with font Arial and size 48):

% Create a mask with some text
surf = new(qvectorlayer)
surf.setfont("Arial", 48)
surf.drawstring("GLASS", [30,180], [224,80])

In general, vector layers are useful for rendering various layers with changing details, for overlaying on video renderings and/or plotting functions, although in this tutorial we are interested in a rasterized representation of the vector layer. Rasterization can be obtained by:

im_mask = surf.rasterize(size(im,0), size(im,1)) 

Correspondingly, im_mask will be an image with the same dimensions as im. Next, we blur the input image using a gaussian filter with blurring parameter 1:

im_mask = gaussian_filter(im_mask, 1)

This results in the following image:

tutorial4_glasstext.jpg

To proceed, we write a kernel function to apply the glass effect to the input image im. A kernel function is a function that is applied (mostly in parallel) to every position in the image. For this purpose, the kernel function has a special position argument pos. The glass effect can be obtained by calculating the horizontal and vertical partial derivatives of the mask, multiplying these derivatives with a constant (strength) and using the resulting values as components of a displacement vector. In this way, every pixel is displaced according to the gradient of the mask image at that position.

function [] = __kernel__ displacement_kernel(im_in : cube'hwtex_linear'mirror, im_out : cube, im_mask : cube, _
strength : scalar, pos : ivec3)

        dx = (im_mask[pos+[0,1,0]]-im_mask[pos]) * strength
        dy = (im_mask[pos+[1,0,0]]-im_mask[pos]) * strength
        im_out[pos] = im_in[pos[0] + dy, pos[1] + dx, pos[2]]    
end

im_out = zeros(size(im))
parallel_do(size(im_out), im, im_out, im_mask, strength, displacement_kernel)

Note in particular the type modifiers 'hwtex_linear and 'mirror applied to im_in. In Quasar, these type modifier allows using special features of the accelerator/GPU device:

Including importing the necessary library (imfilter.q), the final program is:

import "imfilter.q"

function [] = main()
    im = imread("lighthouse.png")
    strength = 0.6
   
    % Create a mask with some text
    surf = new(qvectorlayer)
    surf.setfont("Arial", 48)
    surf.drawstring("GLASS", [30,180], [224,80])
    im_mask = surf.rasterize(size(im,0), size(im,1)) % values get clipped between 0 and 255    
    
    % Blur the mask
    im_mask = gaussian_filter(im_mask, 1)        
    imshow(im_mask), title("Mask")

    function [] = __kernel__ displacement_kernel(im_in : cube'hwtex_linear'mirror, im_out : cube, im_mask : cube, strength : scalar, pos : ivec3)        
        dx = (im_mask[pos+[0,1,0]]-im_mask[pos]) * strength
        dy = (im_mask[pos+[1,0,0]]-im_mask[pos]) * strength
        im_out[pos] = im_in[pos[0] + dy, pos[1] + dx, pos[2]]        
    end

    im_out = zeros(size(im))
    parallel_do(size(im_out), im, im_out, im_mask, strength, displacement_kernel)
    imshow(im_out), title("Processed image")
end

The resulting image is as follows:

tutorial4_result.jpg

So, with only 22 lines of code, we already have a fairly complex effect!

Readers familiar with OpenGL/DirectX pixel shaders will note that the same can easily be achieved with writing a simple pixel shader program. In fact, the content of the above kernel function can be seen as a pixel shader program. The advantages of Quasar in this respect are:

  1. The kernel function can easily be executed on heterogeneous compute devices (CPU, CUDA GPU, OpenCL GPU, …)
  2. The kernel function and the surrounding “driver” code are written in the same language
  3. Kernel functions offer low-level access to some GPU synchronization primitives (like the block/group position, thread synchronization). These primitive allow writing aggregation operations (like the sum, min/max) which are usually hard to express in OpenGL/DirectX pixel shaders.