Properly minimizing an OpenGL view

[arekkusu]

(this is something to put in the FAQ)

Did you ever notice that when you minimize the window of an OpenGL game, sometimes the window turns all white while it is genie-ing into the dock? Or, the dock icon is all white?

Nearly all of the uDevGame entries had this problem, so here's the answer for Cocoa people. You need to read the GL framebuffer and draw it into the underlying Quartz view:


// window delegate methods
- (void)windowWillMiniaturize:(NSNotification *)notification {
[self copyGLtoQuartz];
[[self window] setOpaque:NO]; // required to make the Quartz underlay and the window shadow appear correctly
}


- (void)windowDidMiniaturize:(NSNotification *)notification {
[[self window] setOpaque:YES];
}

// NSOpenGLView subclass methods
- (BOOL)isFlipped {
return YES; // required for proper minimization, must sync ortho view
}


- (void)copyGLtoQuartz {
NSSize size = [self frame].size;
GLfloat zero = 0.0f;
long rowbytes = size.width * 4;
rowbytes = (rowbytes + 3)& ~3; // ctx rowbytes is always multiple of 4, per glGrab
NSBitmapImageRep *minicon = [[NSBitmapImageRep alloc]
initWithBitmapDataPlanes:nil
pixelsWide:size.width
pixelsHigh:size.height
bitsPerSample:8
samplesPerPixel:3
hasAlpha:NO
isPlanar:NO
colorSpaceName:NSDeviceRGBColorSpace
bytesPerRow:rowbytes
bitsPerPixel:32];

[ctx makeCurrentContext];
glFinish(); // finish any pending OpenGL commands
glPushAttrib(GL_ALL_ATTRIB_BITS); // reset all properties that affect glReadPixels, in case app was using them
glDisable(GL_COLOR_TABLE);
glDisable(GL_CONVOLUTION_1D);
glDisable(GL_CONVOLUTION_2D);
glDisable(GL_HISTOGRAM);
glDisable(GL_MINMAX);
glDisable(GL_POST_COLOR_MATRIX_COLOR_TABLE);
glDisable(GL_POST_CONVOLUTION_COLOR_TABLE);
glDisable(GL_SEPARABLE_2D);

glPixelMapfv(GL_PIXEL_MAP_R_TO_R, 1, &zero);
glPixelMapfv(GL_PIXEL_MAP_G_TO_G, 1, &zero);
glPixelMapfv(GL_PIXEL_MAP_B_TO_B, 1, &zero);
glPixelMapfv(GL_PIXEL_MAP_A_TO_A, 1, &zero);

glPixelStorei(GL_PACK_SWAP_BYTES, 0);
glPixelStorei(GL_PACK_LSB_FIRST, 0);
glPixelStorei(GL_PACK_IMAGE_HEIGHT, 0);
glPixelStorei(GL_PACK_ALIGNMENT, 4); // force 4-byte alignment from RGBA framebuffer
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
glPixelStorei(GL_PACK_SKIP_PIXELS, 0);
glPixelStorei(GL_PACK_SKIP_ROWS, 0);
glPixelStorei(GL_PACK_SKIP_IMAGES, 0);

glPixelTransferi(GL_MAP_COLOR, 0);
glPixelTransferf(GL_RED_SCALE, 1.0f);
glPixelTransferf(GL_RED_BIAS, 0.0f);
glPixelTransferf(GL_GREEN_SCALE, 1.0f);
glPixelTransferf(GL_GREEN_BIAS, 0.0f);
glPixelTransferf(GL_BLUE_SCALE, 1.0f);
glPixelTransferf(GL_BLUE_BIAS, 0.0f);
glPixelTransferf(GL_ALPHA_SCALE, 1.0f);
glPixelTransferf(GL_ALPHA_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_RED_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_RED_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_GREEN_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_GREEN_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_BLUE_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_BLUE_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_ALPHA_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_ALPHA_BIAS, 0.0f);
glReadPixels(0, 0, size.width, size.height, GL_RGBA, GL_UNSIGNED_INT_8_8_8_8, [minicon bitmapData]);
glPopAttrib();

[self lockFocus];
[minicon drawInRect:frame];
[minicon release];

[self unlockFocus];
[[self window] flushWindow];
}


Note 1: This setup requires the viewport coordinate system to have the origin in the upper left. For example in an orthographic projection:


glViewport(0, 0, frame.size.width, frame.size.height);
gluOrtho2D(0, frame.size.width, frame.size.height, 0); // must be flipped to match the NSView, origin at TOP left


If your origin is in the lower left, like in Quartz, then you have to flip the NSImage upside down in CopyGLtoQuartz.

Note 2: This doesn't handle a GL view which has a translucent background. This is only an issue if you are overlaying GL content on Quartz content like in the UnderlaySurface example. See the GLUT source code if you need to do this, it involves fixing up the alpha component of each pixel.

Note 3: If your app uses GLUT, then all of this is already done for you.

Note 4: You'll also have to do this if you want to print your NSOpenGLView.

(Edit: fixed typos induced by lack of sleep)

[codemattic]

THANK YOU. this drove me crazy.

[codemattic]

where is ctx defined/initialized?

thanks, Codemattic

[arekkusu]

// NSOpenGLView subclass header
NSOpenGLContext *ctx;


Depending how you construct your view, you either make this yourself, or you can get it from the system with [NSOpenGLContext currentContext];

Don't forget, if you have multiple contexts or use threads, you must set the current context before ANY gl calls (resize drawRect, dealloc, etc);

[Bachus]

Ah thank you. I keep forgetting to add code like this to Gaichu. Mental note...

[geezusfreeek]

Somebody should add this to the source code section of the site. It would be a good addition.

[DoG]

If we only had it in Carbon, too. Though I am sure there are some Apple examples on this, somebody should extract the relevant code.

[OneSadCookie]

It's pretty easy in Carbon, too -- Make a GWorld as if loading a texture via QuickTime, ReadPixels into the GWorld, then CopyBits the GWorld to the window.

[Programmer]

Strange -- I dropped this into my (non-game) app to try it and it doesn't even call those methods upon miniturization. Does this have to be a root view, or will it work with a pair of NSOpenGLView panes in a larger window?

[Sohta]

Make sure that your glView (or whatever object you put the code in) is the window's delegate, then the code should be called when the window is minimised.

[arekkusu]

You'll have to do a little more work if you have multiple GLviews, or overlapping views, or etc. This simplified code is geared towards games where the window is one big GL context.

See the GLUT source for a complete recursive hierarchical copy.

[Sohta]

I've finally added that sweet piece of code to my project (works great)

For those who don't want to change their coordinate system (like me). You can also simply flip the image as you copy it to the window. Here's a simple way to do it:

NSAffineTransform *mirror = [NSAffineTransform transform];
[mirror scaleXBy:1 yBy:-1];
[mirror translateXBy:0 yBy: -size.height];
[mirror concat];
[minicon drawInRect:[glWindow frame]];
[mirror invert];
[mirror concat];

Thanks a lot!

[aaronsullivan]

Anybody ever try to get this to work with an SDL app? I'm assuming it would go into SDLMain.m, but I'm shaky on Cocoa and I could be wrong. Also, I'm not sure how to get the ctx from SDL. Any help on this would be great!

[arekkusu]

I believe SDL is built on top of CGL, not NSGL. So you can use CGLGetCurrentContext() at any time after the context is created and active. Substitute the proper SDLImage stuff for NSBitmapImageRep stuff too.

[DoG]

Since NSGL (and AGL, for that matter)is a layer above CGL, you can probably use CGL calls no matter how the context was created.

[arekkusu]

The example I posted at the top of the thread is written assuming an NSOpenGLContext instance variable "ctx" which is set once when the view is created and never changes. But of course you can use any flavor GL interface to do the same thing.

[arekkusu]

Revisiting this after some recent work on PCSX:

One of the inefficiencies in the code at the top of this thread is the repeated pixel swizzling:
* the GL framebuffer is always ARGB format
* glReadPixels swizzles that to RGBA
* NSImage must be created with RGBA
* NSImage swizzles to ARGB (NSCachedImageRep) during draw

That isn't so great. So here's an updated version using CG to eliminate the swizzles:

- (void)copyGLtoQuartz {
const void *get_byte_pointer(void *bitmap) { return bitmap; }
NSSize size = [self frame].size;
GLfloat zero = 0.0f;
long rowbytes = size.width * 4;
rowbytes = (rowbytes + 3)& ~3; // ctx rowbytes is always multiple of 4, per glGrab
unsigned char* bitmap = malloc(rowbytes * size.height);

[ctx makeCurrentContext];
glFinish(); // finish any pending OpenGL commands
glPushAttrib(GL_ALL_ATTRIB_BITS); // reset all properties that affect glReadPixels, in case app was using them
glDisable(GL_COLOR_TABLE);
glDisable(GL_CONVOLUTION_1D);
glDisable(GL_CONVOLUTION_2D);
glDisable(GL_HISTOGRAM);
glDisable(GL_MINMAX);
glDisable(GL_POST_COLOR_MATRIX_COLOR_TABLE);
glDisable(GL_POST_CONVOLUTION_COLOR_TABLE);
glDisable(GL_SEPARABLE_2D);

glPixelMapfv(GL_PIXEL_MAP_R_TO_R, 1, &zero);
glPixelMapfv(GL_PIXEL_MAP_G_TO_G, 1, &zero);
glPixelMapfv(GL_PIXEL_MAP_B_TO_B, 1, &zero);
glPixelMapfv(GL_PIXEL_MAP_A_TO_A, 1, &zero);

glPixelStorei(GL_PACK_SWAP_BYTES, 0);
glPixelStorei(GL_PACK_LSB_FIRST, 0);
glPixelStorei(GL_PACK_IMAGE_HEIGHT, 0);
glPixelStorei(GL_PACK_ALIGNMENT, 4); // force 4-byte alignment from RGBA framebuffer
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
glPixelStorei(GL_PACK_SKIP_PIXELS, 0);
glPixelStorei(GL_PACK_SKIP_ROWS, 0);
glPixelStorei(GL_PACK_SKIP_IMAGES, 0);

glPixelTransferi(GL_MAP_COLOR, 0);
glPixelTransferf(GL_RED_SCALE, 1.0f);
glPixelTransferf(GL_RED_BIAS, 0.0f);
glPixelTransferf(GL_GREEN_SCALE, 1.0f);
glPixelTransferf(GL_GREEN_BIAS, 0.0f);
glPixelTransferf(GL_BLUE_SCALE, 1.0f);
glPixelTransferf(GL_BLUE_BIAS, 0.0f);
glPixelTransferf(GL_ALPHA_SCALE, 1.0f);
glPixelTransferf(GL_ALPHA_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_RED_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_RED_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_GREEN_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_GREEN_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_BLUE_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_BLUE_BIAS, 0.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_ALPHA_SCALE, 1.0f);
glPixelTransferf(GL_POST_COLOR_MATRIX_ALPHA_BIAS, 0.0f);
glReadPixels(0, 0, size.width, size.height, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, bitmap);
glPopAttrib();

[self lockFocus];
// create a CGImageRef from the memory block
CGDataProviderDirectAccessCallbacks gProviderCallbacks = { get_byte_pointer, NULL, NULL, NULL };
CGDataProviderRef provider = CGDataProviderCreateDirectAccess(bitmap, rowbytes * size.height, &gProviderCallbacks);
CGColorSpaceRef cs = CGColorSpaceCreateDeviceRGB();
CGImageRef cgImage = CGImageCreate(size.width, size.height, 8, 32, rowbytes, cs,
kCGImageAlphaNoneSkipFirst, provider, NULL, NO, kCGRenderingIntentDefault);

// composite the CGImage into the view
CGContextRef gc = [[NSGraphicsContext currentContext] graphicsPort];
CGContextDrawImage(gc, CGRectMake(0, 0, size.width, size.height), cgImage);

// clean up
CGImageRelease(cgImage);
CGDataProviderRelease(provider);
CGColorSpaceRelease(cs);
free(bitmap);

[self unlockFocus];
[[self window] flushWindow];
}


For 32bpp framebuffers, this is about twice as fast as the NSImage version, so the short pause before minimization is reduced. It still works fine for 16bpp framebuffers, but, like the old version, it will have to convert during glReadPixels. (If you know your framebuffer might be 16bpp and you care, you could optimize for that case.)

[Edit: fixed get_byte_pointer type for forward-compatibility.]

[codemattic]

I have a weird thing going on with my G4/400 Rage128. When an app calls glReadPixels it freezes my entire system for just over 30 seconds (just timed it). This only happens the first time - even if I relaunch the app it wont happen again. Only if I reboot the machine will a call to glReadPixels trigger the pause again.

I know the Rage128 is pretty old - so its not a big concern. But does anyone here still using a Rage128 see the same thing or is it just me?

Thought Id mention it since it was weird.

[arekkusu]

30 seconds? Are you sure you're not trying to allocate a gig of RAM or something, and swapping? What does Shark say is happening the whole time?

I definitely don't see that happen on the Rage128 iMacs I've tested on, although I haven't looked with 10.3.7 yet.

The pause I see (for a reasonably sized window on the order of 1024x768) is about 0.1 seconds for NSImage and 0.05 seconds for the new CG code.