Water Reflection

[Marjock]

This seems like something that would have come up before, but the two pages of search results did not cover what I wanted to know.

So, I am attempting to create a basic water reflection, such as might be found on the surface of a pond. I have successfully set up a PBuffer, and can render my scene to it, and then create a texture from this, which can be applied to on-screen meshes.

The problem comes in with Texture Coordinates. If you apply the reflection texture the way you would any other texture, then OpenGL distorts it, so as to create perspective. However, water reflections do not get distorted with perspective, in this way.

So what I have tried to devise is an equation will will calculate new Texture Coordinates based on the angle I am from the water. The idea being that I would calculate the texture coordinates in such a way that it would counter-act OpenGL's perspective distortion.

In short, I've failed. And so I thought I'd come and make a post, and see if anybody is able to explain the kind of techniques that are generally employed in situations such as this.

I'll post my latest code below, it's probably almost impossible to follow, so feel free to question bits of it.
However, at the same time, I wouldn't be surprised if it proves easier to scrap this code and try something new, so feel free to suggest new methods, not just ways of fixing up my current one. :-)

-(void)calculateWaterTexCoords
{
//The number of faces for a model is equal to the number of *floats* for vertex data or tex coord data or normal data,
//and so the numFaces variable can be uses as it is below.

float * tempTexCoordArray = (float*)malloc(sizeof(float)*[waterModel numFaces]);
unsigned int i;
float cameraAngle;
float distance = 0.0f;
float furtherestAway = 0.0f;
int index = 0;
myVector directionVector;
myVector vertexVector;

//Set up a vector for the direction in which the camera is pointing. (xPosition is the camera's xPosition)
directionVector.x = -[self xPosition];
directionVector.y = 0.0f;
directionVector.z = -[self zPosition];

[vectorOutlet normalise:&directionVector];


//Find which vertex of the water model is furtherest away (in the direction that we're looking) from the camera

for(i=0;i<[waterModel numFaces];i+=3)
{
vertexVector.x = [waterModel vertexArray][i];
vertexVector.y = [waterModel vertexArray][i+1];
vertexVector.z = 0.0f;
[vectorOutlet normalise:&vertexVector];

distance = [vectorOutlet dotProduct:directionVector
with:vertexVector];

if(distance > furtherestAway)
{
furtherestAway = distance;
//This index now points to the Vertex, the Normal and the Tex Coord (Depending on which array it is used in) of the furtherest away vertex.
index = i;
}
}

//Get the angle of the camera from the water surface.
cameraAngle = atan([self yPosition]/sqrt([self xPosition]*[self xPosition] + [self zPosition]*[self zPosition]));

cameraAngle = (cameraAngle/GL_PI)*180.0f;

//Go through every texture coordinate and apply a transformation to it, so that it counteracts OpenGL's perspective distortion
for(i=0;i<[waterModel numFaces];i+=3)
{
if(cameraAngle==0.0f)
cameraAngle = 1.0f;

// i will point to the x float, and i+1 will point to the y float.


//The general idea here is the following. When viewed from straight-on (ie. 90 degrees from the surface), a reflection might cover one quarter of the water surface.
//When we are viewing the water from 45 degrees, we see half as many pixels on-screen. However, we want to see the same number of Texture Pixels (OpenGL tries to half it, along with everything else)
//However, we want the furtherest away portion of the water to have the same texture coordinates (because that is where the reflection originates from.
tempTexCoordArray[i] = ([waterModel texCoordArray][i]/((90.0f/cameraAngle)*directionVector.x)) + ([waterModel texCoordArray][index] - ([waterModel texCoordArray][index]/((90.0f/cameraAngle)*directionVector.x)));
tempTexCoordArray[i+1] = ([waterModel texCoordArray][i+1]/((90.0f/cameraAngle)*directionVector.z)) + ([waterModel texCoordArray][index+1] - ([waterModel texCoordArray][index+1]/((90.0f/cameraAngle)*directionVector.z)));
}

//Set the altereted tex coords to be the ones which are used to draw the water.
[waterModel setTexCoordArray:tempTexCoordArray];



}

Thanks in advance,
Mark Thomson.

[akb825]

This is how I did the textures for my water stuff: I drew everything to a rectangular texture the same size as the viewport, and set it up so it wouldn't need any distortions or flipping or anything if just drawn from the camera's point of view. (aka: if I'm drawing the reflection, everything would already be upside-down) Then, once you have the texture set up, you simply have the texture coordinates be the projected texture position. (at least the x y portion) You have 2 options for this: you can either have a vertex shader set up the texture coordinates as the final projected vertex position, or you can set it up to dynamically change through software, by projecting the vertices to set up new texture coordinates each time the camera changes.

[Marjock]

Hmm, I don't quite understand what you mean by the "projected texture position", or the projected vertex position.

-Mark

[akb825]

For projected texture position, I mistyped and meant projected vertex position. What I mean by the projected vertex position is the vertex's position multiplied by the modelview and projection matrices. Essentially, that projects the vertex to the screen's coordinates, so the x and y coordinates are the same as the x and y coordinates for the screen and the z is the depth value. Essentially, all that work is already done in the texture (except the z value is lost) assuming it's a rectangular texture, and you're simply projecting the vertices to your water to then grab the respective portion from the texture.

Edit: actually, to clarify, after you multiply by the matrices, the x and y coordinates will range from -1 to 1. In order to get them useable, you'll need to add 1 and divide by 2 to each both the x and y coordinates, and then multiply by the width and height of the viewport respectively. You can also use gluProject to make your life easier if you're doing it in software rather than in the shader. Also, if you were to do more advanced effects using pixel shaders, you can merely use the fragment's position.

[Marjock]

Well, first off, my texture is a square, but it has the same edges as it would if it were a rectangle, so I should just be able to resize things accordingly.

I think using gluProject looks like my best bet. If I do that, what steps does it eliminate? Seems like it would multiply by my viewport for me, but not do the whole (+1)/2 step.
When using it, I seem to get heaps and heaps of tiny tiny versions of my texture over the surface of the water (So tiny they look almost like noise until you zoom in close, and can then make out vague maybe-shapes).

Thanks for your help.

-Mark

[unknown]

can you just render the water as a stencil and then draw a fullscreen quad with the pbuffer texture on it?

[akb825]

Well, first off, my texture is a square, but it has the same edges as it would if it were a rectangle, so I should just be able to resize things accordingly.

I think using gluProject looks like my best bet. If I do that, what steps does it eliminate? Seems like it would multiply by my viewport for me, but not do the whole (+1)/2 step.
When using it, I seem to get heaps and heaps of tiny tiny versions of my texture over the surface of the water (So tiny they look almost like noise until you zoom in close, and can then make out vague maybe-shapes).

Thanks for your help.

-Mark
Yes, it already does the job of multiplying by the viewport. in that case. When you're rendering your texture, if you use a rectangular texture (with GL_TEXTURE_RECTANGLE_ARB instead of GL_TEXTURE_2D), you can keep the viewport the exact same size when you render the reflection. Unknown's suggestion works, too, but it won't work if you wanted to add distortions or other effects to the rendered water by distorting the texture coordinates, adding other textures on top, etc.

[unknown]

if you were going for wavey rippling water you would need to render 6 sides of a cube for an environment map, or somthing similar.

[akb825]

No, you wouldn't. The simple reflection will do, assuming you aren't planning on doing monstrous near-vertical waves complete with reflection. On a per-vertex scale, you can't do full rippling water, but you could do a larger wave pattern for choppy water.

[Marjock]

Hmm, simply changing GL_TEXTURE_2D to GL_TEXTURE_RECTANGLE_ARB didn't seem to work, are there further complications I'm unaware of?

Also, after projecting the vertices of my water, I get values ranging between -2000 and 9,000. Which seems just a little bit wrong. :-p

Perhaps I am misusing gluProject?

glGetDoublev(GL_MODELVIEW_MATRIX, modelMatrix);
glGetDoublev(GL_PROJECTION_MATRIX, projMatrix);
glGetIntegerv(GL_VIEWPORT, viewport);

if(gluProject((GLdouble)[waterModel vertexArray][i],
(GLdouble)[waterModel vertexArray][i+1],
(GLdouble)[waterModel vertexArray][i+2],
modelMatrix,
projMatrix,
viewport,
&screenX,
&screenY,
&screenZ))
{
NSLog(@"%f, %f, %f", screenX, screenY, screenZ);
[waterModel texCoordArray][i] = screenX;
[waterModel texCoordArray][i+1] = screenY;
[waterModel texCoordArray][i+2] = screenZ;
}
else
{
NSLog(@"Error at gluProject for index: %d",i);
}


Thanks again,
Mark

[akb825]

You appear to be using gluProject correctly. However, the texture coordinates should be in pairs, not in triplets. (so all you need is the x and y coordinates) It's possible that you're assuming that it's in 3s, but it's only looking in 2s, so the texture coordinates would be messed up.

[Marjock]

Well, I have to pass and receive three values from gluProject, and then I was glDrawElements to draw my sprites, so it pulls the information straight from my texCoord Array (Whereby the Z component is 0.0).

Perhaps I'm misunderstanding what you mean, but I don't think there's any way to use gluProject with only X and Y coordinates.

-Mark

[akb825]

You do use gluProject with all 3 coordinates. However, you only need to put the resulting x and y coordinates in the texture array. The resulting z coordinate is useless, since the texture is 2D, and in fact is detrimental since it only expects 2 texture coordinates for each vertex, not 3.

[Marjock]

Changing the code so that the third texCoord value is set to 0.0f doesn't fix the problem, sadly. :-/

I can't understand why the X and Y coordinates would have such bizzare values after being projectd (between -2,000 and 9,000, as I said.)

-Mark

[akb825]

No, you don't set it to 0. You don't set it at all. As I said (twice, now), it is only looking for 2 values packed together, so when you provide 3, it screws up and doesn't process the values correctly. Instead, the array should have the number of vertices*2 for the number of elements, and you should only set i and i + 1 each time.

It's fine that you get values of -2000 to 9000: it just means that you have portions that are off screen. (the values that are on screen would be in the range of 0 - width for the x coordinate, and 0 - height for the y coordinate)

[Marjock]

But an array packed with three values for Texture Coordinates has worked for all of my other objects.

-Mark

[akb825]

Are you using vertex arrays or just immediate mode (with glVertex3f, glVertex2f/3f)? If it's the former, what I said is the problem. If it's the latter, it doesn't really make as much of a difference, since it's going to only look at the first 2 anyway, but you really should only use the 2 rather than 3. Can you be a little more specific as to what the problem is and how the water looks at this point?

[Marjock]

I'm using vertex arrays. However, when I draw other objects using Vertex arrays, I still supply three texture coordinates, and it still works. glTexCoordPointer() seems to take a parameter which would alter this. However, it seems stupid to ignore you when you're kindly helping me, so I'll try with just two coordinates shortly. :-)

Here are some pictures of the water, as it is now.

From far away:

http://i76.photobucket.com/albums/j37/Attraption/FarAway.png


From a middle distance:

http://i76.photobucket.com/albums/j37/Attraption/MidWay.png


From close up:

http://i76.photobucket.com/albums/j37/Attraption/CloseUp.png


The quad floating in the middle is textured from what is being drawn to the Pbuffer:

http://i76.photobucket.com/albums/j37/Attraption/PBufferTexture.png

It looks suspiciously like I'm getting a huge number of tiny versions of my texture over the surface of the water.

-Mark

[akb825]

So are you sure that you changed every instance of GL_TEXTURE_2D to GL_TEXTURE_RECTANGLE_ARB that has to do with that texture, and you've kept the viewport the same when rendering the water as with the scene? Having many tiny parts of it repeated would certainly make sense, though, if every other texture coordinate had a 0 in it from having the coordinates be set up incorrectly. When you set up your texture coordinate array to have only 2 coordinates, be sure that you glTexCoordPointer call has 2 as it's first parameter. If it's 3 right now, it won't make much of a difference (except for the array being 50% larger than it needs to be), since it will at least look at the right intervals. I would say the most likely cause is the use of GL_TEXTURE_2D in some critical areas, though, but the viewport suggestion, if it isn't already implemented, will cause you some problems after you fix that part.

[Marjock]

Okay, well, it turns out my problem with GL_ARB_TEXTURE_RECTANGLE was that I didn't realise that the texture coordianmtes went [0...width],[0....height]. But I do now, and so have made GL_TEXTURE_RECTANGLE work (My floating textured quad proves this for me.)
However, the water surface is still very resistant to working. Below are the latest pictures. You can see in some places that there are things that almost start to look like reflections, but in other places there's nothing at all.
The bendy coloured lines are axis lines drawn to the PBuffer (they're straight when they're drawn, obviously, so something in the texture placement is bending them).

So, here are the pictures from four different angles.

http://i76.photobucket.com/albums/j37/Attraption/AngleOne.png

http://i76.photobucket.com/albums/j37/Attraption/AngleTwo.png

http://i76.photobucket.com/albums/j37/Attraption/AngleThree.png

http://i76.photobucket.com/albums/j37/Attraption/AngleFour.png

Once again, thanks heap for the help,
Mark

[unknown]

Edit: :O WOW!

[akb825]

Something tells me you're doing something to the texture coordinates you don't have to. If you simply use the results of gluProject, it already does the work of going 0 - width and 0 - height for you. This is assuming you draw the reflection right: essentially, from the camera's perspective, but below the water with everything upside-down, redrawing each screen.

[Marjock]

Yeah, the texture is exactly the same as the real world, except it's upside down.

I really can't see anything else that I'm doing to the Texture Coordiantes.

Let's see, I draw the water with the following code:

-(void)drawPondWater
{
int i;
int a;
float matrixArray[16];

for(i=0;i<[pondScene numWater];i++)
{


glBindTexture(GL_TEXTURE_RECTANGLE_ARB, pBufferTexture);
glEnable(GL_TEXTURE_RECTANGLE_ARB);
glDisable(GL_TEXTURE_2D);



glPushMatrix();
glRotatef(270.0f,1.0f,0.0f,0.0f);
glTranslatef([pondScene waterProperties][i*6]*SCENE_SCALE,[pondScene waterProperties][i*6+1]*SCENE_SCALE,[pondScene waterProperties][i*6+2]*SCENE_SCALE);



for(a=0;a<16;a++)
{
matrixArray[a] = [pondScene waterMatrix][(i*16)+a];
}
glMultMatrixf(matrixArray);

[self calculateWaterTexCoords];
[waterModel drawModel];

glPopMatrix();
}

}

-calculateWaterTexCoords looks like this:

-(void)calculateWaterTexCoords
{
double screenX, screenY, screenZ;
unsigned int i;
GLdouble modelMatrix[16];
GLdouble projMatrix[16];
GLint viewport[4];



for(i=0;i<[waterModel numFaces];i+=3)
{

glGetDoublev(GL_MODELVIEW_MATRIX, modelMatrix);
glGetDoublev(GL_PROJECTION_MATRIX, projMatrix);
glGetIntegerv(GL_VIEWPORT, viewport);

if(gluProject((GLdouble)[waterModel vertexArray][i],
(GLdouble)[waterModel vertexArray][i+1],
(GLdouble)[waterModel vertexArray][i+2],
modelMatrix,
projMatrix,
viewport,
&screenX,
&screenY,
&screenZ))
{
[waterModel pixelCoordArray][i] = (int)screenX;
[waterModel pixelCoordArray][i+1] = (int)screenY;
[waterModel pixelCoordArray][i+2] = (int)0.0f;

NSLog(@"%d: %d, %d, %d",i, [waterModel pixelCoordArray][i], [waterModel pixelCoordArray][i+1], [waterModel pixelCoordArray][i+2]);

}
else
{
NSLog(@"Error at gluProject for index: %d",i);
}


}

}


Everything really looks fine. The only thing that is that when I shit my calculateTexCoords call to before the glTranslate, the texture is drawn in a different place, but still suffers the same odd zig-zaggy distortion.

-Mark

[akb825]

Just out of curiosity, what is waterMatrix supposed to represent? Multiplying the modelview matrix by another matrix would certainly mess things up in this way.

[Marjock]

It's a matrix that I pull from Blender which defines the size and rotation of the model currently being drawn.

-Mark

[akb825]

I see. I think it may be a problem with where you draw the reflection: if you mess up your camera calculations (such as by inverting an axis, which then causes it to move in an opposite way) it won't show what you meant to, possibly even nothing if it's in the right place.

[Marjock]

Hmm, well, I know I'm drawing everything correctly to the PBuffer, as I can texture a little 300x150pixel quad, and it looks fine.

Are there any obvious cures to this problem that you can think of, such as multiplying something else by my waterMatrix?
It took me a very long time to get things drawing right straight from setting up a scene in Blender, and the only way seemed to be by using matrices, so I'm very reluctant to remove matrix multiplication from the drawing code.

-Mark

[akb825]

I'm going to have to look into it, since I can't find any obvious reasons as to why it would fail. However, I can make a suggestion right now: you should move the glGetDoublev and glGetIntegerv calls in calculateWaterTexCoords outside the loop: there's no reason to get those matrices more than once. Also, for your texture coordinate array, it may be better if you were to use floats. It shouldn't matter too much, due to the scale, but it couldn't hurt to have a little more precision in there, especially when it keeps them as floats internally anyway. Also, it might be easier to diagnose if you provide the project for me to look at, assuming this isn't some super-secret code you're working on. :p

[Marjock]

Hmm, okay, thanks heaps for all of your help, it really is very good of you. :-)

I was told that when using GL_ARB_TEXTURE_RECTANGLE my tex coords should be ints, but if you're using it successfully with floats, that's actually easier for me. :-)

I'll happily provide the project for you, shortly (When I have access to the other computer, and have fixed up the couple of suggestions you just made).
But, er, I can't promise that the code is at all pretty. :-/ The "SceneLoader" class is especially hideous :-p

So, yeah, thanks heaps, and I'll upload the project shortly.

-Mark

[Marjock]

Wow, it's been a while, and sadly I haven't been able to work on this project at all over the time, let alone fix it.

Now the code and project are probably largely unreadable, I find it real hard to guage how tough it will be for an outside person looking in. Please, just ask anything you want.

The methods that I can see as being of concern are -caculateWaterTexCoords in MyOpenGLView.m and perhaps -drawPondWater in the same class.
-drawModel in ModelLoader.m is another one.

Once again I apologise for the illegibility of the code, and thank you very much for taking the time to do this.

The link is here:

http://www.yousendit.com/transfer.php?action=download&ufid=D048E1101A05F8B0

-Mark