Wishful 3D file format

DoG
Moderator
Posts: 869
Joined: 2003.01
Post: #1
I've noticed a lot of people are looking for an easy to use, yet powerful file format for 3D animation, etc, for use in games.

I realise the requirements vary a lot in terms of complexity, features, learning curve, etc, but it would be nice to see all people want in one place. Post in this thread.

The aim is largely so we can come up with something many people can use, and which is extensible. Furthermore, we can write plugins to export the file format from popular 3D modelers, such as C4D, Maya, or Lightwave. No, no 3DS, it's evil Rasp

Additionally, it would be nice to churn out a few tutorials based on this whole file format business, about writing plugins for 3D modellers, and file parsing in several programming languages.

If you want an open-source file format, speak up now!

EDIT: Wishlist:
  • binary and human readable versions
  • skeletal animation
  • keyframed animation
  • frame-based animation
  • arbitrary additional information
  • exporters for 3D modellers and animators (C4D, Maya, LW, etc)
  • library to parse file and load into memory
  • triangle meshes with per-triangle or per-vertex texture coordinates
  • primitives (spheres, cylinders, boxes, etc)
  • materials with texture references, multi-texturing
Quote this message in a reply
⌘-R in Chief
Posts: 1,260
Joined: 2002.05
Post: #2
Model formats are as varied as the kinds of bananas. (If you didn't know, there are over a thousand different kinds of bananas.) An open one that everyone here could use from the simplest static mesh up to full skeletal animation and was extensible and allowed metadata or something so advanced developers could customize it a bit for their own purposes, as well as freely available code to load and draw in OpenGL would kick major booty.
Quote this message in a reply
Member
Posts: 144
Joined: 2004.07
Post: #3
Sometime last year I got the ball rolling on this by approaching Karl Berg (he wrote a popular .obj loader and is working on his own 3D modeler) about integrating a more standardized (but more capable then .obj) format. He thought it was a great idea and expressed interest in including this format in future versions of Sculpt (I can't speak for him directly though).

I think it's about time as a community we ban together to build a technology we can all share, use, and improve upon.

EDIT: And it's also worthwhile to note DoG and I have experience writing C4D plugins, so we can have that application convered for support.
Quote this message in a reply
⌘-R in Chief
Posts: 1,260
Joined: 2002.05
Post: #4
One idea to throw out there on what the format should support: Binary and Text formats? Both might be handy for manual editing, but if it's not necessary, then I'd go for binary only, and exclude text.
Quote this message in a reply
Moderator
Posts: 1,560
Joined: 2003.10
Post: #5
A few things I'd like to see in the hypothetical perfect 3D file format:
  • Supports both ASCII and binary formats (same info; binary more compact, ASCII human-readable, able to translate between the two without loss of information)
  • Skeletal animation support is a must.
  • Non-skeletal animation support would also be a good thing.
  • Support for arbitrary GL primitive types.
  • Support for arbitrary metadata.
If this ends up going anywhere, it'd be cool if we could provide an open source library for working with the format. Maybe hosted on sourceforge?

- Alex Diener
Quote this message in a reply
Member
Posts: 715
Joined: 2003.04
Post: #6
Microsoft, Adobe, IBM and a dozen other companies are working on this "magic 3D" format that can be used from modeling, to games, to the internet.

Don't know where they are in this quest yet, but "standardized" 3D format is
eventual, although it will take ten years to really make a difference, like PDF has (and PDF is still an annoyance half the time)

Here is a cube with two bones, an animation, and three materials in iGame3D's .WTF format. the files as I went from saving with no bone, to a bone, to materials, to two bones.
In case you need for analysis.
If you have questions then post and I'll send Tobi over to answer them.

There certainly needs to be a home of model format parsing info and source.


Code:
iGame3D Mdl
0
#Author:
#Date:
#Notes:
#PNG: 0

#Bones:
2
top
centroid
0,1
1,1
#Vertices:
10
814.410767,3273.809082,3887.315674,0
814.420715,1623.809204,3887.315674,0
2464.420410,-26.190849,5537.315430,0
2464.420410,3273.809082,5537.315430,1
2464.420410,3273.809082,2237.315674,1
2464.420410,-26.190849,2237.315674,0
-835.579041,-26.190849,5537.315430,0
-835.598572,3273.809082,5537.315430,1
-835.579041,3273.809082,2237.315674,1
-835.579041,-26.190849,2237.315674,0
#Edges:
18
7,2
8,5
7,4
3,7
8,4
6,5
2,6
9,5
5,3
2,5
3,2
4,3
5,4
9,7
6,9
7,6
8,7
9,8
#Materials:
3
Zplanar
11
smooth=0
color=1.000000,1.000000,1.000000,1.000000
ambient=0.774710,0.774710,0.774710,1.000000
diffuse=0.800000,0.800000,0.800000,1.000000
specular=0.000000,0.000000,0.000000,1.000000
shininess=0.000000
emission=0.000000,0.000000,0.000000,1.000000
blend=0
texture=images/brick.png
wire=0.000000
depth=1
4
7,6,2,0.000000,0.000000,0.000000,1.000000,1.000000,1.000000
7,2,3,0.000000,0.000000,1.000000,1.000000,1.000000,0.000000
8,5,9,4158905652365603373056.000000,55922980.000000,7379668075796050358491414528​0.000000,3458224128.000000,211889143808.000000,211889045504.000000
8,4,5,70379458241177514409984.000000,72732507981685305901056.000000,117597211327​9496662670114816.000000,66831941289500383117312.000000,0.000000,1928258805627832​4813702478430208.000000
Xplane
11
smooth=0
color=1.000000,1.000000,1.000000,1.000000
ambient=0.200000,0.200000,0.200000,1.000000
diffuse=0.800000,0.800000,0.800000,1.000000
specular=0.000000,0.000000,0.000000,1.000000
shininess=0.000000
emission=0.637930,0.637930,0.637930,1.000000
blend=0
texture=images/brick.png
wire=0.000000
depth=1
4
5,4,3,0.000000,0.000000,0.000000,1.000000,1.000000,1.000000
5,3,2,0.000000,0.000000,1.000000,1.000000,1.000000,0.000000
9,7,8,0.000000,0.000000,0.999994,1.000000,0.000000,1.000000
9,6,7,0.000000,0.000000,0.999994,0.000000,0.999994,1.000000
Yplane
11
smooth=0
color=1.000000,1.000000,1.000000,1.000000
ambient=0.200000,0.200000,0.200000,1.000000
diffuse=0.800000,0.800000,0.800000,1.000000
specular=0.000000,0.000000,0.000000,1.000000
shininess=0.000000
emission=0.729890,0.729890,0.729890,1.000000
blend=0
texture=images/brick.png
wire=0.000000
depth=1
4
7,4,8,0.000000,1.000000,1.000000,0.000000,0.000006,0.000000
7,3,4,0.000000,1.000000,1.000000,1.000000,1.000000,0.000000
6,9,5,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000
6,5,2,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000
#Animations
1

2
0,0.5
morph=1,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000
0.5,1
morph=1,10,0.000000,0.000000,0.000000,0.000000,0.000000
Quote this message in a reply
Member
Posts: 277
Joined: 2004.10
Post: #7
I like .obj but Skeletal support sounds cool Smile

I've been working on a 3D format where all primitive types were grouped together...
but that would make only one object per-file. Sad
(It would have been in ASCII)

Global warming is caused by hobos and mooses
Quote this message in a reply
DoG
Moderator
Posts: 869
Joined: 2003.01
Post: #8
Well, .obj sucks donkey bollocks, so much said. It's no good trying to extend it. Don't even think about it.

Having a file format be binary/text/xml is secondary to the actual content of. I suppose we could easily add zip compression to a text based file format, too.

Furthermore, by creating a C library for loading the file format, the actual data on disk wouldn't matter as much anyhow.

I have some pretty far-reaching ideas on what the file format should look like logically, but I don't want to show it, yet. I'd like to see some graphs and/or descriptions of how you guys think it should be handled. Remember, flexibility is very important. A parser should be able to ignore information it doesn't support without crapping out. XML would be a very good choice in this regard, as it already provides the tree structure of a scene graph for free, and capable parsers exist for it.

Keep going, I'll edit my first post to show a summary of possible features.
Quote this message in a reply
Moderator
Posts: 3,577
Joined: 2003.06
Post: #9
I agree, I think basing it off .obj isn't really a good idea. I'm not sure about the XML idea though, I think a properly tagged format would be just as fault tolerant. Here are some of my ideas:

-Maybe call it .idg? After a quick search through wotsit, it appears that .idg isn't used as an extension by anybody yet.

-I agree pretty much with ThemsAllTook's suggestions and I like DoG's wishlist. I would also add my emphasis to support for skeletal animation and primarily (strictly?) for use with OpenGL.

-I like iGame3d's # character to denote the start of a tag. I would modify it a bit to be like "#vertex3f 8" though. This would mean that there are 8 vertices with 3 coordinates each to follow. This would make it easy to allocate memory in the loader right there on the spot without having to count lines or calculate faces. I say vertex3f so it follows the OpenGL naming system, plus the format could also be used for 2d objects as well using #vertex2f instead.

-The format should be designed for a simple (easy to teach and learn) and expandable state machine parser.

-It should have C/C++ style comments. Please!
Quote this message in a reply
DoG
Moderator
Posts: 869
Joined: 2003.01
Post: #10
AnotherJake Wrote:I agree, I think basing it off .obj isn't really a good idea. I'm not sure about the XML idea though, I think a properly tagged format would be just as fault tolerant. Here are some of my ideas:

-Maybe call it .idg? After a quick search through wotsit, it appears that .idg isn't used as an extension by anybody yet.

-I agree pretty much with ThemsAllTook's suggestions and I like DoG's wishlist. I would also add my emphasis to support for skeletal animation and primarily (strictly?) for use with OpenGL.

How is the use with OpenGL different than for non-OpenGL with respect to skeletal animation?

Quote:-I like iGame3d's # character to denote the start of a tag. I would modify it a bit to be like "#vertex3f 8" though. This would mean that there are 8 vertices with 3 coordinates each to follow. This would make it easy to allocate memory in the loader right there on the spot without having to count lines or calculate faces. I say vertex3f so it follows the OpenGL naming system, plus the format could also be used for 2d objects as well using #vertex2f instead.

If you are talking about the human radable format, consider that vertices will only ever be stored as 3 floating point numbers. If you only want to deal with 2D, just ignore the last coordinate. Being double or float wouldn't matter, too.

So far, I've used tags with begin and end tags, for example a "begin vertices" / "end vertices" pair. Of course, something a little more robust (like XML) wouldn't hurt.

Quote:-The format should be designed for a simple (easy to teach and learn) and expandable state machine parser.

Agreed.

Quote:-It should have C/C++ style comments. Please!

I dont see any point in such comments. .obj like comments are perfectly sufficient.
Quote this message in a reply
Member
Posts: 715
Joined: 2003.04
Post: #11
.WTF in layman terms

#Vertices:
10 -- ten vertices to this model
814.410767,3273.809082,3887.315674,0 -- X,Y,Z,normal facing

in iGame3d each of the provided floats is divided by 660
ie the first vertice is 1.233956,4.960317,5.889872
I think this is held over from the original meshwork import days

#Edges section tells it how many edges to draw then to draw from integer A to integer B along that list

#Materials:
3 -- there are 3 Materials in this file
Zplanar -- Material Name
11 -- there are eleven properties to this material
smooth=0
color=0.000000,0.000000,0.000000,1.000000
ambient=0.774710,0.774710,0.774710,1.000000
diffuse=0.800000,0.800000,0.800000,1.000000
specular=0.000000,0.000000,0.000000,1.000000
shininess=0.000000
emission=0.000000,0.000000,0.000000,1.000000
blend=0
texture=images/brick.png
wire=0.000000
depth=1
4 -- vertices attached to this material
7,6,2,0.000000,0.000000,0.000000,1.000000,1.000000 ,1.000000
---NO clue what the material data is

In theory its extensible by say adding
#myNewFeature
#primitives
#shaders
#whatever
Quote this message in a reply
Member
Posts: 144
Joined: 2004.07
Post: #12
AnotherJake Wrote:-Maybe call it .idg? After a quick search through wotsit, it appears that .idg isn't used as an extension by anybody yet.

And I think it should stay that way. I groan just a tad when I hear people suggest using an extension like that. Sure it specifies were it originated, but so what, who cares? IMO that's just looking to cause more confusion make us (including me) look bad in creating this format.

I think it's always most appropriate to design and build a product before naming it, the name should come from the wisdom gained by those who went through the development process and poored themselves into it. But if people wanted to name it now, I'd suggest crafting a name around the words 'open, community, free, game', and similar terms to convey what the file is about.

Maybe you guys want to incorporate idevgames into a bunch of things you do to the point where you're naming projects and formats after it, but I find the idea very unappealing.
Quote this message in a reply
Moderator
Posts: 916
Joined: 2002.10
Post: #13
all I care about is:
easy converting to the object
easy translation from 3D format to game usable data. too often people give me this awesome file format, but I just end up using my own format and it seems like such a waste. An already provided code based framework would be wonderful.
Quote this message in a reply
Moderator
Posts: 3,577
Joined: 2003.06
Post: #14
DoG Wrote:How is the use with OpenGL different than for non-OpenGL with respect to skeletal animation?
Whoops! That's totally not at all what I meant to say. It should have been two separate sentences. Skeletal animation is a must. Designing the file format to work primarily with OpenGL should be a must. D'oh!

DoG Wrote:If you are talking about the human radable format, consider that vertices will only ever be stored as 3 floating point numbers. If you only want to deal with 2D, just ignore the last coordinate. Being double or float wouldn't matter, too.
This is true, but what I was thinking is that if all you want to do is draw 2D objects then that extra floating point number to be ignored is just wasting disk space. I realize we're talking about a *3d* file format but I'm always looking for more generic usage (call it a character flaw Wink ).

DoG Wrote:So far, I've used tags with begin and end tags, for example a "begin vertices" / "end vertices" pair. Of course, something a little more robust (like XML) wouldn't hurt.
Hmm... Something still tells me that the XML route is bogus for this, but I'm all ears.

I've been thinking along the lines of a hybrid approach. Something like the header of the file would contain begin and end tags for all the entities contained in the file, but the bulk data (like vertex coordinates) for each entity would be pointed to elsewhere in the file towards the end.

lightbringe Wrote:Maybe you guys want to incorporate idevgames into a bunch of things you do to the point where you're naming projects and formats after it, but I find the idea very unappealing.
That's cool. I don't like to name projects after anything but ME, but this seemed appropriate to point at as being born here. If you're uncomfortable/embarrassed with that I'm all up for whatever anyone else wants, it just sprung to mind at the moment.
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #15
Did it ever occur to y'all that if this were possible, somebody would have done it? There's just too much variation in features required to satisfy everybody with one format, whilst still keeping that format manageable.

this is doomed. DOOMED! DOOOOOOOOOOOOMED!!!!!!!!!!!

on which note, why not simply write a library to load, animate and render MD5s, and Wings and Blender plug-ins to export 'em?
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  what's the most suitable graphic file format when deadling with OpenGL? anthony 8 4,754 Feb 16, 2007 06:25 PM
Last Post: PowerMacX
  Reverse Engineering 3d File Format nabobnick 1 2,951 Aug 13, 2004 12:11 PM
Last Post: nabobnick