08 September 2013

Digging into SOIL Library for OpenGL

soil image loading library for opengl
When I was (re)implementing PhotoAlbum sample application I got stuck in one place. Program was about to load a list of textures (like 16 maybe) and suddenly I noticed that the whole process takes quite long time. More disappointing thing was that I got memory access errors for some images and configurations. This stopped the "development" for a time and I decided that I need to dig inside my image loading library: dig into SOIL.

About...

SOIL library is a well known, public domain, very easy to use and small library for loading images. It is designed to work with OpenGL, usually as a static library. Simple case:
tex_ID = SOIL_load_OGL_texture("test.jpg", 
                               SOIL_LOAD_AUTO, 
                               SOIL_CREATE_NEW_ID,
                               SOIL_FLAG_POWER_OF_TWO | 
                               SOIL_FLAG_MIPMAPS |
                               SOIL_FLAG_COMPRESS_TO_DXT);
I like the library because of it simplicity. There is almost no need to learn new API. For example SOIL returns raw GLuint for texture objects so it can be easily embedded into your existing wrapper for Texture objects. It is multiplatform as well so it aligns well with freeGLUT for instance.
the question

What was wrong?

The First

Let's go to the mentioned memory access violation errors. It occurred when I tried to load RGB image of size 725x544. The problem was very simple to fix, but I took me some time to solve. 
I started debugging:
  • To be able to go inside SOIL_load_OGL_texture it had to be recompiled with DEBUG flag. SOIL comes with the source code by default so I quickly setup solution under VS 2012 and press recompile.
  • Soil.c is the main file for the library. It is quite large - around 2000 lines - and it is written in C style. Unfortunately the C style means, in this case,  large functions (for instance 400 lines in once example!) and usually no or "little" SRP. Fortunately after a while the code seems to be readable.
  • SOIL_load_OGL_texture is a simple function that loads data from a file and then redirects all the work to SOIL_internal_create_OGL_texture. Alternatively for DDS files SOIL_direct_load_DDS is used. When loading the data Soil can force additional channels (from RGB to RGBA for instance)
  • SOIL_internal_create_OGL_texture is this long 400-line function. It does almost everything :). Its main purpose is to create OpenGL texture object and push the previously loaded data into OpenGL. To do that it needs to support various SOIL flags like INVERT_Y, NTSC_SAFE_RGB, MULTIPLY_ALPHA...
  • For some flags GL extension are loaded - like for NPOT textures or texture rectangles. The work is done by query_**_capability() functions. Basically GL extensions are loaded "manually", no separate library is used for that.
  • After the pixel transformations are done GL texture object is created. 
  • Then data is uploaded to OpenGL. In most cases glTexImage is called.
  • When compression is selected then SOIL can compress loaded pixel using DXT1 or DXT5 custom implementations and then use glCompressedTexImage to push the data.
  • Another step is to create mipmaps. This step is done using custom "rescaling" algorithms and mipmaps are generated for POT textures only.
  • The last step involves setting proper texture parameters and a memory cleanup.
  • If everything goes as expected then the function returns valid ID (GLuint) to OpenGL Texture Object.
Below there is a simple diagram of the described path
SOIL_load_OGL_texture diagram

In the end I got one idea regarding my memory errors: use proper data alignment! Simply, I had to setup glPixelStore properly because data from my image was not aligned to 4 bytes. I wrongly assumed that SOIL can deduce the alignment. Silly thing, but it motivated me to debug the code and learn library internals.

The second

It was related to the performance. One image loaded quite fast, but when I needed to upload 16 images it was not. I tried to change flags in the loader. Usually I pass SOIL_FLAG_MIPMAPS | SOIL_FLAG_INVERT_Y. When, for instance, I removed mipmap generation it worked faster! My images was not that big: around 540x700, so I thought they should be read quickly.
But while debugging those memory errors several ideas came to my mind about the SOIL and its performance:
  • When MIPMAP flag is passed SOIL does not only create mipmaps but before that rescales the texture to be POT. Simply: it cannot make mipmaps out of NPOT texture directly. In my case for an image 540x700 it was upscaled to 1024x1024! This could take some time!
  • Algorithms for scaling, mipmapping, etc are all in software, no hardware acceleration is used.

Proposed changes

The debugging and learning a bit of SOIL internals made me to think about the following upgrades that could be done to the library:
  • Add ability to use glGenerateMipmap. This requires only single line in code to call, works for NPOT textures and should be hardware accelerated. Moreover nowadays it should be supported almost everywhere. Basically GL_EXT_framebuffer_object is only needed.
  • Use modern style of loading extensions. Use glGetStringi. This should make SOIL to work in Core GL profile. Additionally GL_CLAMP needs to be changed into GL_CLAMP_TO_BORDER.
  • Add ability to use immutable texture storage.
  • Add some more examples and tests.
Next time I will try to describe changes that I tried to make to the library. For now you can take a look at my github repository SOIL_ext.
Content By Bartlomiej Filipek, Blogger platform
Any opinions expressed herein are in no way representative of those of my employers.
Disclaimer: This site contains ads or referral links, which provide me with a small commission for each sale/click. Thank you for your understanding.