Browsed by
Category: Programming

OpenGL ES 2.0/3.0 Offscreen rendering with glRenderbuffer

OpenGL ES 2.0/3.0 Offscreen rendering with glRenderbuffer

Rendering to offscreen surfaces is a key component to any graphics pipeline. Post-processing effects, deferred rendering, and newer global illumination strategies all use it. Unfortunately, implementing offscreen rendering on OpenGL ES is not well documented. OpenGL ES is often used on embedded/mobile devices, and until recently, these devices haven’t typically had the graphics bandwidth to keep up with new rendering techniques. Compound this with the fact that many mobile games have simple gameplay/small screens that do not need such complex lighting models, many people now use off the shelf engines for their games, and that there is still a good amount of mobile hardware out there that doesn’t even support render to offscreen surfaces, and it is no surprise that few people use the technique and it’s not well discussed.

In implementing offscreen rendering for OpenGL ES, I turned to the very good OpenGL ES Programming book as it has a whole chapter on framebuffer objects. When I tried the samples in the book, however, I was having a lot of difficulty getting it working on my linux-based mobile device. A lot of the implementation examples use a technique of creating framebuffer objects using textures, but you can also use framebuffer objects via something called render buffers. One reason this is good to know is because many hardware vendors support very few render-to-texture formats. You can often find yourself struggling with your implementation not working because the output formats aren’t supported.

Thankfully, I found this article and thought I’d copy the information here since it’s the only place I’ve seen working code that demonstrated the technique. It also includes the very important step of reading the output format and uses glReadPixels() so you can validate that you were writing correctly to the offscreen renderbuffer surface.

In my case, on an Intel graphics part, I found that the format (which is also the most recommended one) that worked  was GL_RGB/GL_UNSIGNED_SHORT_5_6_5. Steps 1-8 is standard OpenGL ES setup code that is included so you can verify your setup. Step 9 is where the glFrameBuffer and glRenderBuffer objects are created.

 

    #define CONTEXT_ES20

    #ifdef CONTEXT_ES20
        EGLint ai32ContextAttribs[] = { EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE };
    #endif

    // Step 1 - Get the default display.
    EGLDisplay eglDisplay = eglGetDisplay((EGLNativeDisplayType)0);

    // Step 2 - Initialize EGL.
    eglInitialize(eglDisplay, 0, 0);

    #ifdef CONTEXT_ES20
    // Step 3 - Make OpenGL ES the current API.
    eglBindAPI(EGL_OPENGL_ES_API);

    // Step 4 - Specify the required configuration attributes.
    EGLint pi32ConfigAttribs[5];
    pi32ConfigAttribs[0] = EGL_SURFACE_TYPE;
    pi32ConfigAttribs[1] = EGL_WINDOW_BIT;
    pi32ConfigAttribs[2] = EGL_RENDERABLE_TYPE;
    pi32ConfigAttribs[3] = EGL_OPENGL_ES2_BIT;
    pi32ConfigAttribs[4] = EGL_NONE;
    #else
    EGLint pi32ConfigAttribs[3];
    pi32ConfigAttribs[0] = EGL_SURFACE_TYPE;
    pi32ConfigAttribs[1] = EGL_WINDOW_BIT;
    pi32ConfigAttribs[2] = EGL_NONE;
    #endif

    // Step 5 - Find a config that matches all requirements.
    int iConfigs;
    EGLConfig eglConfig;
    eglChooseConfig(eglDisplay, pi32ConfigAttribs, &eglConfig, 1,
                                                    &iConfigs);

    if (iConfigs != 1) {
        printf("Error: eglChooseConfig(): config not found.n");
        exit(-1);
    }

    // Step 6 - Create a surface to draw to.
    EGLSurface eglSurface;
    eglSurface = eglCreateWindowSurface(eglDisplay, eglConfig,
                                  (EGLNativeWindowType)NULL, NULL);

    // Step 7 - Create a context.
    EGLContext eglContext;
    #ifdef CONTEXT_ES20
        eglContext = eglCreateContext(eglDisplay, eglConfig, NULL,
                                               ai32ContextAttribs);
    #else
        eglContext = eglCreateContext(eglDisplay, eglConfig, NULL, NULL);
    #endif

    // Step 8 - Bind the context to the current thread
    eglMakeCurrent(eglDisplay, eglSurface, eglSurface, eglContext);
    // end of standard gl context setup

    // Step 9 - create framebuffer object
    GLuint fboId = 0;
    GLuint renderBufferWidth = 1280;
    GLuint renderBufferHeight = 720;

    // create a framebuffer object
    glGenFramebuffers(1, &fboId);
    glBindFramebuffer(GL_FRAMEBUFFER, fboId);

    // create a texture object
    // note that this is commented out/not used in this case but is
    // included for completeness/as example
    /*  GLuint textureId;
     glGenTextures(1, &textureId);
     glBindTexture(GL_TEXTURE_2D, textureId);
     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);                             
     //GL_LINEAR_MIPMAP_LINEAR
     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
     glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP_HINT, GL_TRUE); // automatic mipmap
     glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, renderBufferWidth, renderBufferHeight, 0,
                  GL_RGB, GL_UNSIGNED_BYTE, 0);
     glBindTexture(GL_TEXTURE_2D, 0);
     // attach the texture to FBO color attachment point
     glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
                         GL_TEXTURE_2D, textureId, 0);
     */
     qDebug() << glGetError();
     GLuint renderBuffer;
     glGenRenderbuffers(1, &renderBuffer);
     glBindRenderbuffer(GL_RENDERBUFFER, renderBuffer);
     qDebug() << glGetError();
     glRenderbufferStorage(GL_RENDERBUFFER,
                           GL_RGB565,
                           renderBufferWidth,
                           renderBufferHeight);
     qDebug() << glGetError();
     glFramebufferRenderbuffer(GL_FRAMEBUFFER,
                               GL_COLOR_ATTACHMENT0,
                               GL_RENDERBUFFER,
                               renderBuffer);

      qDebug() << glGetError();
      GLuint depthRenderbuffer;
      glGenRenderbuffers(1, &depthRenderbuffer);
      glBindRenderbuffer(GL_RENDERBUFFER, depthRenderbuffer);
      glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16,     renderBufferWidth, renderBufferHeight);
      glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthRenderbuffer);

      // check FBO status
      GLenum status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
      if(status != GL_FRAMEBUFFER_COMPLETE) {
          printf("Problem with OpenGL framebuffer after specifying color render buffer: n%xn", status);
      } else {
          printf("FBO creation succeddedn");
  }

  // check the output format
  // This is critical to knowing what surface format just got created
  // ES only supports 5-6-5 and other limited formats and the driver
  // might have picked another format
  GLint format = 0, type = 0;
  glGetIntegerv(GL_IMPLEMENTATION_COLOR_READ_FORMAT, &format);
  glGetIntegerv(GL_IMPLEMENTATION_COLOR_READ_TYPE, &type);

  // clear the offscreen buffer
  glClearColor(1.0,0.0,1.0,1.0);
  glClear(GL_COLOR_BUFFER_BIT);

  // commit the clear to the offscreen surface
  eglSwapBuffers(eglDisplay, eglSurface);

  // You should put your own calculation code here based on format/type
  // you discovered above
  int size = 2 * renderBufferHeight * renderBufferWidth;
  unsigned char *data = new unsigned char[size];
  printf("size %d", size);

  // in my case, I got back a buffer that was RGB565
  glReadPixels(0,0,renderBufferWidth,renderBufferHeight,GL_RGB, GL_RGB565, data);

  // Check output buffer to make sure you cleared it properly.
  // In 5-6-5 format, clearing to clearcolor=(1, 0, 1, 1)
  // you get 1111100000011111b = 0xF81F in hex
  if( (data[0] != 0x1F) || (data[1] != 0xF8))
      printf("Error rendering to offscreen buffern");

  QImage image(data2, renderBufferWidth,  renderBufferHeight,renderBufferWidth*2, QImage::Format_RGB16);
  image.save("result.png");
PIX for Windows is back!

PIX for Windows is back!

PIX is a performance tuning and debugging tool for game developers – that hadn’t been updated in years for the desktop. It survived on in three generations of Xbox consoles, but there was no desktop love. No longer! Microsoft just announced PIX beta is now available for analyzing DirectX 12 games on Windows.

PIX on Windows provides five main modes of operation:

  • GPU captures for debugging and analyzing the performance of Direct3D 12 graphics rendering.
  • Timing captures for understanding the performance and threading of all CPU and GPU work carried out by your game.
  • Function Summary captures accumulate information about how long each function runs for and how often each is called.
  • Callgraph captures trace the execution of a single function.
  • Memory Allocation captures provide insight into the memory allocations made by your game.

Go to the Microsoft blog to download it for free.

 

Lane Finding

Lane Finding

My first homework assignment for my self-driving automotive class was to find lanes on still, then on captured video. Here was my first attempt, which seems to have come out pretty well. I still want to improve it a bit with some more frame-to-frame smoothing.

Useful links/techniques:

Module to cause linux kernel panic

Module to cause linux kernel panic

In doing some robustness testing, I needed to crash my Linux VM intentionally to see if the rest of the system survived. I was thinking of writing a kernel module that just did something silly like dereference null or divide by zero, but turns out you can do it much easier than that. Just call panic.

panic.c:
#define __NO_VERSION__
#include <linux/version.h>
#include 
#include 

int init_module(void)
{
    panic(" insert lame excuse here");
    return 0;
}

Build with gcc -I/usr/src/linux/include -D__KERNEL__ -DMODULE -o panic.o -c panic.c
Here's a good link about building kernel modules without full kernel source.

When you run: insmod panic.o 
Bang - your system will kernel panic and crash.
Credit to Paul's Journal
“Hello World” OpenGL on Linux

“Hello World” OpenGL on Linux

Writing the OpenGL context creation plumbing on Linux isn’t 100% intuitive. It’s always good to have a handy getting started guide. With developers increasingly using engines such as Unity, the number of people writing directly to OpenGL is shrinking – and so are some of the online resources to get over this initial step. I decided to collect a few links and copy critical sample code here for when I need to whip something up quickly.

 

Some links to good ‘hello world’ Linux GL samples:
https://github.com/jckarter/hello-gl
https://www.opengl.org/wiki/Tutorial:_OpenGL_3.0_Context_Creation_(GLX)
http://www-f9.ijs.si/~matevz/docs/007-2392-003/sgi_html/ch04.html

 

Code:
Taken from: https://www.opengl.org/sdk/docs/man2/xhtml/glXIntro.xml

Below is a minimal example of creating an RGBA-format X window that’s compatible with OpenGL using GLX 1.3 commands. The window is cleared to yellow when the program runs. The program does minimal error checking; all return values should be checked.

#include < stdio.h >
#include < stdlib.h >
#include < GL/gl.h >
#include < GL/glx.h >

int singleBufferAttributess[] = {
    GLX_DRAWABLE_TYPE, GLX_WINDOW_BIT,
    GLX_RENDER_TYPE,   GLX_RGBA_BIT,
    GLX_RED_SIZE,      1,   /* Request a single buffered color buffer */
    GLX_GREEN_SIZE,    1,   /* with the maximum number of color bits  */
    GLX_BLUE_SIZE,     1,   /* for each component                     */
    None
};

int doubleBufferAttributes[] = {
    GLX_DRAWABLE_TYPE, GLX_WINDOW_BIT,
    GLX_RENDER_TYPE,   GLX_RGBA_BIT,
    GLX_DOUBLEBUFFER,  True,  /* Request a double-buffered color buffer with */
    GLX_RED_SIZE,      1,     /* the maximum number of bits per component    */
    GLX_GREEN_SIZE,    1, 
    GLX_BLUE_SIZE,     1,
    None
};


static Bool WaitForNotify( Display *dpy, XEvent *event, XPointer arg ) {
    return (event->type == MapNotify) && (event->xmap.window == (Window) arg);
}
int main( int argc, char *argv[] )
{
    Display              *dpy;
    Window                xWin;
    XEvent                event;
    XVisualInfo          *vInfo;
    XSetWindowAttributes  swa;
    GLXFBConfig          *fbConfigs;
    GLXContext            context;
    GLXWindow             glxWin;
    int                   swaMask;
    int                   numReturned;
    int                   swapFlag = True;

    /* Open a connection to the X server */
    dpy = XOpenDisplay( NULL );
    if ( dpy == NULL ) {
        printf( "Unable to open a connection to the X servern" );
        exit( EXIT_FAILURE );
    }

    /* Request a suitable framebuffer configuration - try for a double 
    ** buffered configuration first */
    fbConfigs = glXChooseFBConfig( dpy, DefaultScreen(dpy),
                                   doubleBufferAttributes, &numReturned );

    if ( fbConfigs == NULL ) {  /* no double buffered configs available */
      fbConfigs = glXChooseFBConfig( dpy, DefaultScreen(dpy),
                                     singleBufferAttributess, &numReturned );
      swapFlag = False;
    }

    /* Create an X colormap and window with a visual matching the first
    ** returned framebuffer config */
    vInfo = glXGetVisualFromFBConfig( dpy, fbConfigs[0] );

    swa.border_pixel = 0;
    swa.event_mask = StructureNotifyMask;
    swa.colormap = XCreateColormap( dpy, RootWindow(dpy, vInfo->screen),
                                    vInfo->visual, AllocNone );

    swaMask = CWBorderPixel | CWColormap | CWEventMask;

    xWin = XCreateWindow( dpy, RootWindow(dpy, vInfo->screen), 0, 0, 256, 256,
                          0, vInfo->depth, InputOutput, vInfo->visual,
                          swaMask, &swa );

    /* Create a GLX context for OpenGL rendering */
    context = glXCreateNewContext( dpy, fbConfigs[0], GLX_RGBA_TYPE,
				 NULL, True );

    /* Create a GLX window to associate the frame buffer configuration
    ** with the created X window */
    glxWin = glXCreateWindow( dpy, fbConfigs[0], xWin, NULL );
    
    /* Map the window to the screen, and wait for it to appear */
    XMapWindow( dpy, xWin );
    XIfEvent( dpy, &event, WaitForNotify, (XPointer) xWin );

    /* Bind the GLX context to the Window */
    glXMakeContextCurrent( dpy, glxWin, glxWin, context );

    /* OpenGL rendering ... */
    glClearColor( 1.0, 1.0, 0.0, 1.0 );
    glClear( GL_COLOR_BUFFER_BIT );

    glFlush();
    
    if ( swapFlag )
        glXSwapBuffers( dpy, glxWin );

    sleep( 10 );
    exit( EXIT_SUCCESS );
}

 

The value of ESP was not properly saved across a function call…

The value of ESP was not properly saved across a function call…

Windows can trigger this error, but sometimes it’s not easy to figure out what’s going on.

I recently got this error when trying to use the OpenGL ES ANGLE library on Windows 10. When compiling against the ANGLE library, the error came when trying to call into the ANGLE and used eglGetProcAddress().

eglGetProcAddress() returns function pointers for important GL extensions, so I couldn’t just ignore it or work around it.

In looking around, the obvious first step is to make sure you’re defining the function pointers correctly, but that turns out not to be my problem.

In looking at this article, I realized I probably had a mismatch of compiler and linker settings. The Visual Studio projects (VS2008) that came with ANGLE required a certain set of compiler and linker flags that were not standard. I had migrated the Visual Studio projects to VS2015, so that also added an element of uncertainty. I simply opened both project settings up next to each other and compared the settings for the ANGLE library build and the final project and found a few mismatches. I change a number of them to be the same, and things worked great.

Summary:
Check the linking AND compiling flags for not only your project, but the project files that generate the libraries you’re linking against. Differences in compiler settings can cause this error.

Practice your coding skills

Practice your coding skills

For those of us that work in the industry, one of the difficult parts about working in high tech is constantly keeping on top of all the new developments and technology.

One of the things I’ve noticed after you get a lot of years of work under your belt is that you naturally start specializing into certain areas. These specializations are good in themselves, but often they utilize only slivers of the original breadth of computer science, algorithms, and data structures. If one is not careful, you can lose that breadth that is essential to your adaptability.

LeetCode is a great website with literally hundreds of coding problems that can help you brush up on your algorithms, data structures, and coding skills.  Give it a whirl!

‘Audibilization’ of sorting algorithms

‘Audibilization’ of sorting algorithms

Visualization and “audibilization” of 15 Sorting Algorithms in 6 Minutes.
Sorts random shuffles of integers, with both speed and the number of items adapted to each algorithm’s complexity.
The algorithms are: selection sort, insertion sort, quick sort, merge sort, heap sort, radix sort (LSD), radix sort (MSD), std::sort (intro sort), std::stable_sort (adaptive merge sort), shell sort, bubble sort, cocktail shaker sort, gnome sort, bitonic sort and bogo sort (30 seconds of it).

Google more information via the “Sound of Sorting”.

CMake with compilers that need custom parameters

CMake with compilers that need custom parameters

When using CMake on non-GCC/non-Microsoft compilers – you often run into interesting problems. Especially true for embedded devices/cross-compilers.

One thing that can bite you is the fact that CMake requires the compiler to pass a ‘smoke’ test. Unfortunately, if there are required parameters for your compiler, the smoke test part will fail.

There are a few ways to solve this, the ‘recommended’ way seems to be via the CMAKE_FORCE_C_COMPILER/CMAKE_FORCE_CXX_COMPILER flags. This allows you to tell CMake what the compiler is and pass the smoke test.

CMAKE_FORCE_C_COMPILER(/usr/mycc/cxint86 GNU)
CMAKE_FORCE_CXX_COMPILER(/usr/mycc/cxxint86 GNU)

Here are the docs for more information
https://cmake.org/Wiki/CMake_Cross_Compiling#The_toolchain_file