CVDisplayLink - ObjectiveC & C++

Apprentice
Posts: 10
Joined: 2012.06
Post: #1
I'm trying to create a simple 2d game using Xcode and ObjectiveC++. I have worked with OpenGL before but only with GLUT. I'm trying to jump to using something better. I'm just getting started so I haven't done hardly anything yet. I wanted to use a CVDisplayLinkRef like it's shown in this article by Apple: http://developer.apple.com/library/mac/#...index.html



Here is the code they provide (and the code I used):

Code:
@interface MyView : NSOpenGLView
{
    CVDisplayLinkRef displayLink; //display link for managing rendering thread
}
@end

- (void)prepareOpenGL
{
    // Synchronize buffer swaps with vertical refresh rate
    GLint swapInt = 1;
    [[self openGLContext] setValues:&swapInt forParameter:NSOpenGLCPSwapInterval];

    // Create a display link capable of being used with all active displays
    CVDisplayLinkCreateWithActiveCGDisplays(&displayLink);

    // Set the renderer output callback function
    CVDisplayLinkSetOutputCallback(displayLink, &MyDisplayLinkCallback, self);

    // Set the display link for the current renderer
    CGLContextObj cglContext = [[self openGLContext] CGLContextObj];
    CGLPixelFormatObj cglPixelFormat = [[self pixelFormat] CGLPixelFormatObj];
    CVDisplayLinkSetCurrentCGDisplayFromOpenGLContext(displayLink, cglContext, cglPixelFormat);

    // Activate the display link
    CVDisplayLinkStart(displayLink);
}

// This is the renderer output callback function
static CVReturn MyDisplayLinkCallback(CVDisplayLinkRef displayLink, const CVTimeStamp* now, const CVTimeStamp* outputTime, CVOptionFlags flagsIn, CVOptionFlags* flagsOut, void* displayLinkContext)
{
    CVReturn result = [(MyView*)displayLinkContext getFrameForTime:outputTime];
    return result;
}

- (CVReturn)getFrameForTime:(const CVTimeStamp*)outputTime
{
    // Add your drawing codes here

    return kCVReturnSuccess;
}

- (void)dealloc
{
    // Release the display link
    CVDisplayLinkRelease(displayLink);

    [super dealloc];

I've been trying to use that as a bridge to a simple class I wrote in C++. I tried to call my class' drawing function from the getFrameForTime functions above, but every time I try to make my NSOpenGLView class implementation file a .mm file I get errors on these two lines of code:


Code:
CGLContextObj cglContext = [[self openGLContext] CGLContextObj];
CGLPixelFormatObj cglPixelFormat = [[self pixelFormat] CGLPixelFormatObj];

the errors are:
error: invalid conversion from 'void*' to '_CGLContextObject*'
error: invalid conversion from 'void*' to '_CGLPixelFormatObject*'




the code works fine before I change the file extension to .mm so I assume that the compiler is suddenly seeing these two lines differently because I told it to compile c++ code as well. I'm sorry that I know so little. I'm extremely new to Objective-C. Thanks so much!
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #2
That's a C++ error (that would not normally be present for C/ObjC); you can't implicitly cast from void* to another pointer type in C++. You just need to add a cast.

Avoid CVDisplayLink. It's not worth the trouble (multithreading, lack of multimonitor support, etc.)
Quote this message in a reply
Apprentice
Posts: 10
Joined: 2012.06
Post: #3
Ok, thanks for the input! Would I use an NSTimer instead? What other options are there. Again thank you for your response.

Also, is this your article? http://blog.onesadcookie.com/2007/12/xco...orial.html

If it is...you got me started with OpenGL. Thank you Smile
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #4
NSTimer is good. If you set your minimum OS version to 10.7 or later then you'll automatically get vsync; if you set it to 10.6 or earlier then you should explicitly enable it.

Yes, that's my article. Constantly outdated :/
Quote this message in a reply
Apprentice
Posts: 10
Joined: 2012.06
Post: #5
Awesome, I guess even if I need to use CVDisplayLink in the future, I might as well start with something I better understand, like NSTimer.

I'm still running OSX 10.6.8 so I better enable it. I'm assuming this is enabling vsync. Am I correct?

Code:
// Synchronize buffer swaps with vertical refresh rate
- (void)prepareOpenGL
{
    GLint swapInt = 1;
    [[self openGLContext] setValues:&swapInt          forParameter:NSOpenGLCPSwapInterval];
}

I've looked all over the place for information about the pros and cons of CVDisplayLink and NSTimer. It seems that most people prefer CVDisplayLink, so I'm interested in what you prefer to use for a render loop if you avoid CVDisplayLink. Do you just use an NSTimer?

Thanks again! Your help is greatly appreciated.
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #6
Yes, that enables vsync. You can do it in one line if you prefer:

Code:
[[self openGLContext] setValues:(GLint[]){1} forParameter:NSOpenGLCPSwapInterval];

I use NSTimer.
Quote this message in a reply
Moderator
Posts: 3,571
Joined: 2003.06
Post: #7
(Jun 21, 2012 07:16 AM)Turtle Labs Wrote:  I've looked all over the place for information about the pros and cons of CVDisplayLink and NSTimer. It seems that most people prefer CVDisplayLink, so I'm interested in what you prefer to use for a render loop if you avoid CVDisplayLink.

From my own experience, I suspect that most of the ones who "prefer" DisplayLink in forum posts and mailing lists haven't actually developed with DisplayLink for long. I think they are fixated on the last few percent of performance they can squeeze out of it, no matter the effort and/or cost, rather than being practical, IMHO. I say that because that's where I was when I "preferred" display link myself back in the day.

DisplayLink pros:
- smoothest possible animation I've been able to achieve on a Mac

DisplayLink cons (as already mentioned by OSC):
- doesn't run on the main thread
- no multiple display support

I use display link for my own development, to help keep me on my multi-threading toes, and only because I have an existing path developed, but I am afraid to deal with what kinds of bugs there will be out in the wild. I haven't released software which uses it on the Mac yet, although we are considering a release in the next couple of months. I haven't decided if I will release it with DisplayLink -- I doubt that I will.

I have DisplayLink and the NSTimer path pretty much the same so I can optionally choose either one. This is really important IMHO to be able to isolate multithreading bugs with the DisplayLink path. If a bug is discovered and I'm unsure then I fall back to NSTimer for a bit to see if it was because of threading.

In practice, from my own experience, the single DisplayLink advantage of smooth animation is offset by three things, basically:

1) When the game is in motion with all kinds of stuff, NSTimer animation performance is "good enough" that you don't really notice much difference while playing. If you're making something like Pong where the ball is basically the only moving object, then you will more likely notice it, but it still doesn't ruin the gameplay.

2) Since it's in another thread, any time you touch your engine code from the main thread, such as from a Cocoa menu item you will crash miserably and perhaps randomly, unless you bracket the call with a lock. I have one project which has a ridiculous number of locks needed because I have so much Cocoa interface interacting with the engine. I don't think it's worth it anymore to have all those extra lock calls polluting otherwise elegant code.

3) Since it's in another thread ... you get multi-threaded bugs!!! If you haven't done this kind of debugging, be forewarned that it is definitely not for beginners, and multithreaded debugging in Obj-C can be a new experience in reality bending confusion on top of that. It can slow development down greatly, because you really have to do your development and testing in very small chunks, where you change a few lines, test, and repeat. Going back to find a prior existing bug and not knowing where to start looking can be maddening.

One has to constantly ask themselves: Is it really worth the extra hassle?

Then some would argue, well, it couldn't be *that* much hassle, could it? Heh... Let's examine that! Here is pseudocode setting up an NSTimer path:

Code:
- (id)initWithFrame:(NSRect)frameRect
        useDisplayLink:(BOOL)yn
        coreName:(const char *)coreName
        coreConfiguration:(TQcoreConfig *)configuration
{

... yada yada

    renderTimer = [[NSTimer alloc] initWithFireDate:[NSDate distantFuture]
                interval:0.001 // must use with vbsynch on, or you waste lots of CPU!
                target:self
                selector:@selector(renderTimerCallback:)
                userInfo:nil
                repeats:YES];
    [[NSRunLoop currentRunLoop] addTimer:renderTimer forMode:NSEventTrackingRunLoopMode];
    [[NSRunLoop currentRunLoop] addTimer:renderTimer forMode:NSDefaultRunLoopMode];
    [[NSRunLoop currentRunLoop] addTimer:renderTimer forMode:NSModalPanelRunLoopMode];
    [renderTimer release];

... yada yada

    return self;
}

- (void)renderTimerCallback:(NSTimer*)theTimer
{
    // lets the OS call drawRect for best window system synchronization
    [self display];
}

- (void)drawRect:(NSRect)rect
{
    ... draw stuff ...

    [currentContext flushBuffer];
}

That's pretty straight forward. Now let's examine the pseudocode for setting up the NSTimer and parallel CVDisplayLink paths:
Code:
- (id)initWithFrame:(NSRect)frameRect
        useDisplayLink:(BOOL)yn
        coreName:(const char *)coreName
        coreConfiguration:(TQcoreConfig *)configuration
{

... yada yada

    if (useDisplayLink)
    {
        CVDisplayLinkCreateWithActiveCGDisplays(&displayLink);
        CVDisplayLinkSetOutputCallback(displayLink, displayLinkCallback, self);
        CGLContextObj cglContext = [[self openGLContext] CGLContextObj];
        CGLPixelFormatObj cglPixelFormat = [[self pixelFormat] CGLPixelFormatObj];
        CVDisplayLinkSetCurrentCGDisplayFromOpenGLContext(displayLink, cglContext, cglPixelFormat);
    }
    else
    {
        renderTimer = [[NSTimer alloc] initWithFireDate:[NSDate distantFuture]
                    interval:0.001 // must use with vbsynch on, or you waste lots of CPU!
                    target:self
                    selector:@selector(renderTimerCallback:)
                    userInfo:nil
                    repeats:YES];
        [[NSRunLoop currentRunLoop] addTimer:renderTimer forMode:NSEventTrackingRunLoopMode];
        [[NSRunLoop currentRunLoop] addTimer:renderTimer forMode:NSDefaultRunLoopMode];
        [[NSRunLoop currentRunLoop] addTimer:renderTimer forMode:NSModalPanelRunLoopMode];
        [renderTimer release];
    }

... yada yada

    return self;
}

- (void)lockFrameMutex
{
    pthread_mutex_lock(&myFrameMutex);
}

- (void)unlockFrameMutex
{
    pthread_mutex_unlock(&myFrameMutex);
}

static CVReturn displayLinkCallback(CVDisplayLinkRef displayLink, const CVTimeStamp *now,
                            const CVTimeStamp *outputTime, CVOptionFlags flagsIn,
                            CVOptionFlags *flagsOut, void *displayLinkContext)
{
    tqMacView *macView = (tqMacView *)displayLinkContext;
    
    // this is not on the main thread, so need to lock or crash will ensue
    [macView lockFrameMutex];
    [macView drawFrame];
    [macView unlockFrameMutex];
    
    return kCVReturnSuccess;
}

// similar to above, except called by regular timer, not display link
- (void)renderTimerCallback:(NSTimer*)theTimer
{
    // lets the OS call drawRect for best window system synchronization
    [self display];
}

- (void)drawRect:(NSRect)rect
{
    if (useDisplayLink)
    {
        // if we're using Display Link then the only time drawRect will be called is when the OS windowing
        // system needs to redraw
        [self lockFrameMutex];
        [self drawFrame];
        [self unlockFrameMutex];
    }
    else
    {
        // note no need for lock because we're already on the main thread with NSTimer
        [self drawFrame];
    }
}

- (void)drawFrame
{
    NSAutoreleasePool *pool = NULL;
    if (useDisplayLink)
    {
        pool = [[NSAutoreleasePool alloc] init];
    }

    [currentContext makeCurrentContext];
    
    // must lock GL context because display link is threaded
    CGLLockContext([currentContext CGLContextObj]);

    ... draw stuff ...

    [currentContext flushBuffer];
    CGLUnlockContext([currentContext CGLContextObj]);
    
    if (pool != NULL)
    {
        [pool release];
    }
}
Quote this message in a reply
Apprentice
Posts: 10
Joined: 2012.06
Post: #8
AnotherJake, thanks for that explanation. I definitely did not hear about many of the disadvantages of using CVDisplayLink on the forums I visited, so this really helped me put it all in perspective.
Quote this message in a reply
⌘-R in Chief
Posts: 1,248
Joined: 2002.05
Post: #9
BTW, found this quote from the Apple engineer:

"Hmm.... even being the person who wrote CVDisplayLink originally, if I were writing a game-like app, I'd probably just use a continuously firing NSTimer and let the VBL stuff throttle the actual frame rate."
Quote this message in a reply
Apprentice
Posts: 5
Joined: 2012.10
Post: #10
Hi All,

I'm a bit late to this thread, but hope someone is still paying attention to it. I have a rather unusual app I am developing at work that requires 5 displays. We have a MacPro with 2 ATI 5770 cards, and 5 nice Dell LCDs. 4 displays are for displaying content, and the 5th is basically a control window.

The app has to display 4 completely different scenes/animations that have to be mathematically calculated for each frame. I have the code to generate the 4 texture buffers and display them as desired. I am using 4 windows each with a subclassed NSOpenGLView to display my content.

Main Problem, of course, is performance and getting smooth animation at 60fps on each screen. When running on one screen, it looks beautiful, two is just passable, 3 and 4 trigger seizures in most humans. While each frame is unique, they calculation is not that difficult, (basically generating a B&W pattern) and I do not think they are not getting done in time to display.

I have tried both the CVDisplayLink approach and NSTimer. I see now, after more thorough research, that CVDisplayLink is not necessarily the better mouse-trap that Apple seems to suggest, especially with multiple windows/displays.

But NSTimer doesn't quite do it either, although it is better when testing with 2 displays going. Further, I have seen in the Apple docs that NSTimer is discouraged for anything requiring less than 50ms resolution, so this would seem to disqualify it for 60fps applications. But it does seem to do much better than 50ms, as with 2 displays it is still pretty smooth. I have the NSTimer in the init for each window, though they all run on main loop.

The 5th (control) window will have some activity on it as well, mainly UI input, but I can't use up all my CPU cycles on the 4 NSTimers

I would love to hear anyone's thoughts on how best to approach this problem.

Many thanks in advance!
Quote this message in a reply
Moderator
Posts: 3,571
Joined: 2003.06
Post: #11
NSTimer works fine for 60 fps applications. I've used it for 120 Hz applications without any problems. It depends on how much CPU you require during display refresh.

I am not sure I understand exactly what you are doing, as it does sound a bit unusual, but it seems to me (guessing) that if you are drawing on the main thread for four different, large views at the same time with different scenes, you could suck up a lot of CPU quickly and performance will suffer. Threaded rendering of some sort sounds like it might be the way to go in your case.

Also, when using NSTimer, a lot of examples suggest to use a 60 Hz timer, but I like to set the timer to 1000 Hz and make sure to have VBL synch enabled, which automatically throttles that back to 60 Hz, helping to avoid missing frames better than a 60 Hz timer, without revving the CPU. That has shown to be the best performance for me over the years (when using NSTimer).

Also, be certain that you have no views overlapping other displays than the ones they are intended to be on, or else you will pay dearly in terms of performance because they have to synch between the two displays.
Quote this message in a reply
Sage
Posts: 1,482
Joined: 2002.09
Post: #12
Also, it's not quite clear how your rendering works. It sounds like you are doing software rendering on the CPU then uploading all of that into huge screen sized textures to blit them.

Have you done profiling to see where your bottlenecks are?

Scott Lembcke - Howling Moon Software
Author of Chipmunk Physics - A fast and simple rigid body physics library in C.
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #13
CVDisplayLink doesn't work with multiple displays.

Your best bet is one GL context per display, one thread per context, and vsync on all of them, but I've heard complaints that doesn't work either Sad
Quote this message in a reply
Apprentice
Posts: 5
Joined: 2012.10
Post: #14
Thanks all for the replies...

Caveat: While I have been developing apps for OS X for several years (mostly number crunching and data conversion apps), I am fairly new at OpenGL, and have not yet had to manage multi-threaded applications.

Some clarification: The actual image that I need displayed on each LCD is computed on the GPU using OpenCL. My very first iteration of this app just used CPU, and it was clearly a bottleneck. Now I have the GPU calculating a single row of 1920 pixel in RGBA, and this row is then vertically repeated in GL by repeating the pattern (which is what I want displayed). (Note: I tried to get the OpenCL/OpenGL shared context setup to work to avoid bringing the image buffer back from the GPU, but couldn't get it to work with the multiple windows. Possibly threading issues that I couldn't get a handle on. Seems like this would be the most efficient if I could get it to work simultaneously with 4 displays and 2 GPUs).

I am setting the VBL in prepareOpenGL. So, if I understand correctly, with the VBL set OpenGL will only render and output at the refresh rate, regardless of the timer frequency. YES? With each timer tick I run the OpenCL kernel, create an OpenGL texture, and glFlush(). So, with the timer set > 60fps perhaps the OpenGL call creating the texture from the image buffer is getting stepped on as I read back the image buffer from the OpenCL kernel...? I am not sure how to figure what is causing the stuttering.

As far as threading each rendering window, this makes sense to me, as the "control" window UI, and all the timers seem to be fighting when I begin working the UI. I've been trying to wrap my head around threading issues in OpenGL and OpenCL contexts, but I;ve mostly just been pulling my threads out Smile If anyone can recommend some resources that might help with the threading of the rendering windows, that would be great.

I will post some code snippets in a min.
Hope this is the right way to post code...
This is the OpenGL code in my render loop. The imageBufferLinear in glTexImage2D below is what comes back from the OpenCL kernel just prior to executing this OpenGL code.
Code:
    glEnable( TextureTarget );
    glBindTexture( TextureTarget, _textureId );
    
    glTexParameteri(TextureTarget, GL_TEXTURE_WRAP_S, GL_REPEAT);
    glTexParameteri(TextureTarget, GL_TEXTURE_WRAP_T, GL_REPEAT);
    glTexParameteri(TextureTarget, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
    glTexParameteri(TextureTarget, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

    glTexImage2D(TextureTarget, 0, TextureInternal, _textureWidth, 1, 0, TextureFormat, TextureType, imageBufferLinear);

    glBegin( GL_QUADS );
    {
        glTexCoord2f( 0.0f, 0.0f );
        glVertex3f( -1.0f, -1.0f, 0.0f );
        
        glTexCoord2f( 0.0f, 1.0f );
        glVertex3f( -1.0f, 1.0f, 0.0f );
        
        glTexCoord2f( 1.0f, 1.0f );
        glVertex3f( 1.0f, 1.0f, 0.0f );
        
        glTexCoord2f( 1.0f, 0.0f );
        glVertex3f( 1.0f, -1.0f, 0.0f );
    }
    glEnd();

    glFlush();

Edited by AnotherJake: code tags use brackets here. There is a "#" symbol above the text box when posting which will insert them for you.
Quote this message in a reply
Moderator
Posts: 3,571
Joined: 2003.06
Post: #15
(Oct 2, 2012 09:17 PM)dauerbach Wrote:  So, if I understand correctly, with the VBL set OpenGL will only render and output at the refresh rate, regardless of the timer frequency. YES?

Yes, that is correct. As an example, a timer with a 1000 Hz frequency will be automatically blocked until time to draw at the display's refresh rate, which for an LCD is typically about 60 Hz.

I don't know anything about OpenCL, but I am sure someone here does. The round trip from the GPU to get your texture then re-upload it (if I am understanding you correctly) is definitely going to be a performance killer. Surely there is a better way to do that.

Also, you should use glTexSubImage2D for texture updates, which is a little faster after you've created the initial image with glTexImage2D.

Also, although it probably doesn't matter for just one quad of texture, immediate mode with glbegin/end is obsolete.
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  CVDisplayLink instead of NSTimer OptimisticMonkey 6 7,217 Nov 18, 2009 02:49 PM
Last Post: SethWillits
  Interesting Articel on CVDisplayLink OptimisticMonkey 1 4,011 Dec 28, 2008 11:15 PM
Last Post: AnotherJake