Reading Binary Files

[Nick]

I'm trying to develop a small, portable, custom binary file to store things for my game. I found a site that shows how to write out structs to a binary file quite easily. It also showed an easy way to read this structure. Here's what it shows:

struct OBJECT
{
int number;
char letter;
} obj;

obj.number = 15;
obj.letter = ëM�*;

fout.write((char *)(&obj), sizeof(obj));

That would write the entire structure for you! Let�*s move on to input now. Input will be a cinch now because the read() function takes exactly the same parameters as write(), and it works exactly the same.

ifstream fin("file.dat", ios::binary);

fin.read((char *)(&obj), sizeof(obj));


Is there a way to write out a struct like this but then read in the individual parts? For instance I save the example struct but I want to recall the letter and assign it to one thing but the number to another. I don't want to just read in to an identical struct. How would I go about this.

[Steven]

You probably should allocate a temporary instance of your structure and read it, grab the values from that, and then copy them to where you want them to go. If you try to read the numbers in separately you have to deal with not only the byte order but also you have to guess at how the structure is padded and aligned. And by portable I hope you don't mean that you can copy the data files across computers, because for the same reasons you probably can't using this method. YMMV

[Nick]

Why wouldn't I be able to use the file on multiple computers? Even if I load with the temporary struct, it still wouldn't work? Why not?

[sealfin]

@Nick: I'd presume that by "multiple computers" Steven is really talking about multiple architectures/OS (your usual "fun with endianess" issues...)

The code would also likely break on even only the one OS if, at an indeterminate point in the future, you switched compilers* with the new compiler re-ordering or padding the members of the struct to perform it's own, different, optimisations...

This is the reason why you can't just read an individual member from the file – as you don't know what changes the compiler has made to the struct, you don't know where the member you're looking for might be in the file – you're forced to "fin.read((char *)(&obj), sizeof(obj));" the entire object, and then retrieve objects as "char myChar = obj.letter;"



(*although I suppose not that likely on the Mac unless you're already using CodeWarrior.)

[Nick]

So is it a bad way to go to save my files this way? I want the most cross-platform way to save files. I'm going to save my maps and all sorts of things. I'm using just SDL and C/C++ so there are no Cocoa classes to help me. I'm just sorting out some of the "finer" details of programming.

[LongJumper]

It's not a bad way to save files, unless you want it to be cross platform. Now if you are designing a class that does all the work for you, you could write functions that would take into account the endianness of the system you are on and properly write/read the file.

It's definetly a fast and easy way to save files, and I prefer it to writing out long text files of information. Steven's advice is a little misleading, you can copy them across computers, just not ones with different architectures or OS's.

[Nick]

Hm. Interesting. And here I thought using straight C/C++ would be perfect. Any tips on where to look as to writing a class that does all the work for me? Seems slightly daunting.

[OneSadCookie]

"different architectures" being "the mac" now that we're running on both PowerPC and Intel. Never, ever, ever, do what the code in the OP does. Not only is it guaranteed to fail on whichever of PowerPC and Intel didn't write the original file, but it's likely to fail in a move to a different ABI when the alignment of various types changes.

You should always read a specific number of bytes into a buffer of memory, then manually extract the bytes you want, swap them as appropriate, then put them wherever you want them.

If you want a marginally easier way of doing that than lots of pointer math, you could try http://onesadcookie.com/svn/repos/libBinaryIO/ -- a printf/scanf-like API for reading and writing binary data.

[Taxxodium]

Back in the good old pre-Carbon days we used to GetHandle() and BlockMove()

I miss those days :p

[Fenris]

What you need to do is design your own file format. Then you allocate a buffer with malloc or new[] and fill that out. That way you will sidestep all problems with padding, since there is none. (You designed it not to have any) Then, if you have multibyte data in it, you need to detect if you are on a big- or little-endian system. If you're on a system different than the one that made the file, you have to swap the bytes to match. Google for endianness or byteswapping.

As for the first part, it's easy.
Write:

char *buffer = (char *) malloc (100); // Or whatever size you need
char *bufPtr = buffer; // A pointer to the first byte

memcpy (bufPtr, thePlayer->numOfLives, sizeof (int));
bufPtr += sizeof (int);
memcpy (bufPtr, thePlayer->money, sizeof (int));
bufPtr += sizeof (int);
fwrite (as usual, just dump the buffer)


Reading:

char *buffer = (char *) malloc (100); // Or whatever size you need
char *bufPtr = buffer; // A pointer to the first byte

fread (as usual, just read the entire file)
memcpy (thePlayer->numOfLives, bufPtr, sizeof (int));
bufPtr += sizeof (int);
memcpy (thePlayer->mone, bufPtr, sizeof (int));
bufPtr += sizyeof (int);

I'm sure you get the idea :)

[Nick]

Would something like this work? I get a bus error when I run it in the terminal. I'm probably doing the fwrite wrong. I haven't used that function much.


#include <stdio.h>
#include <string.h>
#include <stdlib.h>

struct TestClass
{
int age;
int lives;
};


int main()
{
TestClass myObj;
myObj.age = 24;
myObj.lives = 4;


char *buffer = (char *)malloc(100);
char *bufPtr = buffer;

memcpy(bufPtr, (int *)myObj.age, sizeof(int));
bufPtr += sizeof(int);
memcpy(bufPtr, (int *)myObj.lives, sizeof(int));
bufPtr += sizeof(int);

FILE *testFile;
testFile = fopen("test.dat","wb");
fwrite(bufPtr,sizeof(int),2,testFile);

fclose(testFile);
return 0;
}

[sealfin]

I'd first put in some error detection to check whether the file you're trying to fwrite() to has actually been created; I'd then wonder long and hard about why you're creating a char* variable and using memcpy() to store ints in it... I haven't read through all the source, so this may be a silly question... but why?

Oh, and I haven't played with pointer arithmetic for quite a while, but from my memory you're passing 'bufPtr' to fwrite() without first reseting it's address; that will probably cause a few errors (but reading the code, you won't even get that far...), and you probably want to pass the original 'buffer' instead.

Try something like this...

#ifdef __cplusplus
extern "C" {
#endif

#include <stdio.h>

#ifdef __cplusplus
};
#endif

class TestClass
{
public:
int m_lives;

void Save( void )
{
FILE *file;
if(( file = fopen( "sealfin.bin", "wb" )) == NULL )
{
printf( "Scream! And die!" );
return;
}
fwrite( &m_lives, sizeof( int ), 1, file );
/* Yes, I know I should probably check that the int has actually been written to the file, but I'm in a rush... */
fclose( file );
return;
};

void Load( void )
{
FILE *file;
if(( file = fopen( "sealfin.bin", "rb" )) == NULL )
{
printf( "Scream! And die!" );
return;
}
fread( &m_lives, sizeof( int ), 1, file );
/* ...and here I should be checking that an int has actually been read from the file. */
fclose( file );
return;
};
};


int main()
{
TestClass obj;
obj.Load();
printf( "\nlives: %d\n\n", obj.m_lives );
obj.m_lives = 21;
obj.Save();
return 0;
}

[sealfin]

Uh, I just took a second look; your bus error was caused by this: memcpy(bufPtr, (int *)myObj.age, sizeof(int));
Which should have been this: memcpy(bufPtr, &myObj.age, sizeof(int));

You were dereferencing the variable which you should have been passing the address of (i.e., you were passing it's value as the address); my points on the pointer arithmetic, and the error checking, still stand though ;)

[Nick]

I knew about error checking and such. Normally I do that but I was just trying to figure out writing the files. Thanks for the tips and the fix.

The char thing was just me being absent minded and just waking up, trying to make sense of things I hadn't ever really worked with.

[Nick]

Using your technique, sealfin, how would I save multiple variables and recall them in the proper order?

[sealfin]

If you mean the technique from my original post where I included code, the major difference between my method and Fenris' is that with mine you have to make lots of calls to fwrite()/fread() (but you gain the advantage that you can branch the input code based on the values retrieved), whereas with Fenris' method you have to play with pointer arithmetic (but gain the advantage that you only need one call to fwrite()/fread(), but at the disadvantage that you lose the ability to branch based on values retrieved.)

There is of course no reason why you couldn't mix both methods, loading chunks of values and then inspecting the values loaded up to that point for whether to branch...

#ifdef __cplusplus
extern "C" {
#endif

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#ifdef __cplusplus
};
#endif

class TestClass
{
public:
int m_lives;
char *m_name;

TestClass( void )
{
m_lives = 0;
m_name = NULL;
};

~TestClass( void )
{
if( m_name != NULL )
{
free( m_name );
}
};

void Save( void )
{
FILE *file;
if(( file = fopen( "sealfin.bin", "wb" )) == NULL )
{
printf( "Scream! And die!" );
return;
}
fwrite( &m_lives, sizeof( int ), 1, file );
/* Yes, I know I should probably check that the int has actually been written to the file, but I'm in a rush... */
int length = strlen( m_name );
fwrite( &length, sizeof( int ), 1, file );
fwrite( m_name, sizeof( char ), length, file );
fclose( file );
return;
};

void Load( void )
{
FILE *file;
int length;
if(( file = fopen( "sealfin.bin", "rb" )) == NULL )
{
printf( "Scream! And die!" );
return;
}
fread( &m_lives, sizeof( int ), 1, file );
/* ...and here I should be checking that an int has actually been read from the file. */
fread( &length, sizeof( int ), 1, file );
m_name = ( char* )malloc( length + 1 );
fread( m_name, sizeof( char ), length, file );
m_name[ length ] = '\0';
fclose( file );
return;
};

void GiveName( const char *p_name )
{
m_name = ( char* )malloc( strlen( p_name ) + 1 );
strcpy( m_name,
p_name );
return;
};
};


int main()
{
TestClass obj;
obj.Load();
printf( "\nlives: %d\n", obj.m_lives );
printf( "\nname: %s\n\n", obj.m_name );
obj.m_lives = 21;
obj.GiveName( "Mark Bishop" );
obj.Save();
return 0;
}

The above code is not particularly robust (i.e. calling GiveName() more than once will obviously leak memory, amongst other fun...)