ThePlatoon.com
July 30, 2010, 05:13:58 AM *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: Look around and get acquainted to the new settings and features.
 
  Home   Forum   Help Search Calendar Login Register  
  Shop Gallery  
Pages: [1] 2 3   Go Down
  Print  
Author Topic: Ghost Recon maps  (Read 12433 times)
0 Members and 1 Guest are viewing this topic.
Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« on: December 15, 2006, 06:40:41 AM »

I've been gone from the GR scene for a while.  I don't recall if anyone ever posted any information about the file layout for the original Ghost Recon maps.  I've been looking at them lately trying to figure out the storage format to see if I can perhaps create an exporter for MilkShape 3d to build new maps.

The map format seems to look a lot like the 3DS and MAX file formats (lots of embedded text tags).  I've made some headway but being able to find any other information on those would be great.

I seem to remember that Mike Schell (or was it WolfSong) had 3DS Max and made some maps for GR.  But it doesn't appear that he's here anymore.  Anybody know how to get in touch with him?  With some (very) simple maps, I might be able to figure out the map file format.

I don't have access to 3DS Max (I can't afford the $3500) and the plugins only work with R3 or R4 anyway and you can only get R9, now, I think.
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Zazoo
Administrator
Hero Member
*****

Rep: 13
Offline Offline

Posts: 1343
2049.50 credits

View Inventory
Send Money to Zazoo

View Profile WWW
« Reply #1 on: December 18, 2006, 11:37:09 PM »

Quote from: CowBell;27366
I've been gone from the GR scene for a while.  I don't recall if anyone ever posted any information about the file layout for the original Ghost Recon maps.  I've been looking at them lately trying to figure out the storage format to see if I can perhaps create an exporter for MilkShape 3d to build new maps.

The map format seems to look a lot like the 3DS and MAX file formats (lots of embedded text tags).  I've made some headway but being able to find any other information on those would be great.

-------------------------------------------------

I seem to remember that Mike Schell (or was it WolfSong) had 3DS Max and made some maps for GR.  But it doesn't appear that he's here anymore.  Anybody know how to get in touch with him?  With some (very) simple maps, I might be able to figure out the map file format.


I'm still here (Mike Schell).

It's been a while since I've done any GR map work in 3D Studio Max, so I wouldn't know where to begin when it comes to building a map. I do have a map making tutorial I wrote back when I still knew how to mod GR. :tongue:

You can get it here:
http://www.mcschell.com/GR-MAP-Tutorial.zip

I also have partial format specs for the MAP and QOB files. I found most of this info in just the manner you mentioned... building small, very simple maps, and then ripping the files apart to see what everything did. I took the time to write out a rough, partial format spec for each file type... I knew it would come in handy some day!

http://www.mcschell.com/GRMAPFormat.doc
http://www.mcschell.com/GRQOBFormat.doc

The only differences between file formats are in the tags, flags, and layout -- the 3D data and texture chunks themselves are the same in every file (QOB, CHR, MAP, etc.). I figured out the layout and tags for most of the formats, as well as most of the flags for QOB and CHR files. The only real chore left is to figure out the flags for the MAP file. It won't be hard, just tedious since it involves a lot of jumping between 3D Studio, GR, and a hex editor/programming suite.

It's been 5 years since I've touched this stuff (that long? Wow), so I probably won't be able to offer much more help than this. I do have code left on another backup disk. I have a few basic GR file parsers and object models I built that may be useful to you, I'll see if I can find them. What language(s) are you using?

Good luck!
~Mike
Logged

Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #2 on: December 19, 2006, 03:13:51 AM »

Heh, you fooled me with that Swedish_Seb thing! Cheesy

I'm using Visual C++.  I'm still on V6.0 but moving (slowly and reluctantly to V8.0 VS 2005).

What I'm building right now is a MAP parser. I've started an object framework that handles each part as objects (all labels are preceded by the label length, etc.).  From there each label is handled as an object type that may have child objects.  (materiallist->materials  materiallist->textures  lightlist->lights).  The geometry list appears to be mostly an opaque entity at present.

I'll check out the stuff at those links.  That might get me a lot further than I am.  I'm trying to tackle one file type at a time.  The QOB files are objects that get placed in the MAP, I think.

If I can figure out the entire structure of the MAP files, I can write an exporter/editor for MilkShape (or something else) and maybe even a viewer so we can see what the map will look like in-game.  It would also be cool to be able to take existing maps and relight them.

Thanks, Mike!! Thumbs up 2  You are still, Da Man!
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #3 on: December 19, 2006, 04:05:14 AM »

Okay, I looked at your MAP layout document.  I think that will be very helpful.  But the material entries do not appear to follow the layout you've given.  Some material entries are 0x63 (99) bytes in size with the size integer included in the size (the way Microsoft has the structure size embedded at the beginning of many of their structures) and some are 0x64 (100) bytes in size.  Apparently only the first 10 are 99 bytes in size because of the double digits in the names after number 9 (0 through 9 are first).

The textures are listed in a separate list following the materials.  The next value after the materials list is the number of textures.  There's a size entry for each texture, too (about 0x4C), right where it should be.

The map I'm looking at is the main_menu map for the original OGR missions.  It's quite small and appears to be representative.  I'm also looking at the "training" map for OGR. It just has more data in it and appears to follow the "mainmenu" map layout exactly.

The other entries look like they are probably correct from a cursory examination.  But I won't get to them until I'm sure I can read the materials and textures properly.

I'm thinking that the four byte integer that you call a "chunk id" prior to the length of each text label might be a data type (read structure type) id.  Each list label is preceded by a different type "chunk id".  Also each material is preceded by the _same_ "chunk id" (0x2).  Each texture also appears to have the same chunk id (0x26).

The reason I say that the type id (chunk id) preceeds the text label is because that value changes just before the first texture entry.  The value 2  (0x02) preceeds each material entry.  And the value 38 (0x26) preceeds each texture entry.

The version number of the materials that I'm seeing also are 2 rather than 1.  I'm thinking that that may be why these are different.

Were you using Max 3 or Max 4 for your testing?  The plug-ins for each might have different versions of various entities and thus different formats.

What you've given me is quite valuable.  I thank you for that.  Since what I'm trying to do is both have a way to look at existing GR maps AND be able to create new ones, I need to understand both versions.  So I will code different structures for the different apparent versions of the data.  If you have the old maps you exported, I would like to have those, too so I can test against those as well.

I've already made a couple of leaps just from reading your file.  Thanks again!! Thumbs up 2 Star :cookie: cool

I'll report back tomorrow with some updates on what I've found.  Some other things are starting to click, now (like the purpose of the second 4 byte integer after the BeginMapv4.0).
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Zazoo
Administrator
Hero Member
*****

Rep: 13
Offline Offline

Posts: 1343
2049.50 credits

View Inventory
Send Money to Zazoo

View Profile WWW
« Reply #4 on: December 19, 2006, 06:50:29 AM »

Quote from: CowBell;27496
Okay, I looked at your MAP layout document.  I think that will be very helpful.  But the material entries do not appear to follow the layout you've given.  Some material entries are 0x63 (99) bytes in size with the size integer included in the size (the way Microsoft has the structure size embedded at the beginning of many of their structures) and some are 0x64 (100) bytes in size.  Apparently only the first 10 are 99 bytes in size because of the double digits in the names after number 9 (0 through 9 are first).


The sizes of the individual material chunks are given by "Material Chunk Size" (the first 4 bytes of each entry). Since the entries allow for variable-length zero-terminated strings you'll get different size values for some entries (i.e. the double digit entries will have one extra byte in their material name strings, thus one additional byte in their material entry). None of the GR formats are padded to any kind of boundary, so parsers have to be prepared to read variable-length strings

Quote

I'm thinking that the four byte integer that you call a "chunk id" prior to the length of each text label might be a data type (read structure type) id.  Each list label is preceded by a different type "chunk id".  Also each material is preceded by the _same_ "chunk id" (0x2).  Each texture also appears to have the same chunk id (0x26).

The reason I say that the type id (chunk id) preceeds the text label is because that value changes just before the first texture entry.  The value 2  (0x02) preceeds each material entry.  And the value 38 (0x26) preceeds each texture entry.


Yes, I included a small list of "list" and "entry" ID's at the beginning of the document which give the values assigned to each chunk.

Quote
The version number of the materials that I'm seeing also are 2 rather than 1.  I'm thinking that that may be why these are different.


I believe this may refer to the old RS formats.

Quote
Were you using Max 3 or Max 4 for your testing?  The plug-ins for each might have different versions of various entities and thus different formats.


Not sure.

Also, I updated the commenting in my code far more often than I updated this document. I may have more info in my code -- assuming I can find it, you'd think I would have been smart and backed these all up together... :tongue:

~Mike
Logged

Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #5 on: December 19, 2006, 10:56:33 PM »

Yeah, I was posting the obvious/already stated for some of that.  The Chunk type appears to preceed the chunk, though rather than trail it.  Or is that what you said?

I figured out the bytes immediately following the BeginMapv4.0.

The first four byte integer is a date/time stamp.  This is apparently when the map file was created.  It's a "time_t" that can be turned into a string by using ctime().

The second four byte integer is the size of the material list (the entire material list).

The third four byte integer is the type code for the material list.

I figured out the reason for the different lengths of the chunks after I posted that.  It is indeed because of the numbers following some of the tags.

I'm still working on the other stuff.  Getting it to read and identify all the chunks properly in any map file is my first goal.  Figuring out the formats for all the versions of the various chunks is the next.

It appears that the files in the game (v1.40) are fairly different than the files created using the plugins.

I'm printing out (to the console) information about the map file as it processes it.  When it can read all the chunks, it should serve as a way to see which and many resources are used by a map.

The main_menu map, by the way, doesn't have an env or other files associated with it except for a single background texture (the one that backs the soldiers in the team selection/equipment screen).

I think the last four bytes of the map files being zero, indicate that there are no more chunks to read.  Ergo, end of file.

I'll post an updated version of that document when I'm sure that my information is correct.

I appreciate your help with this, Mike. Thumbs up 2
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #6 on: December 21, 2006, 10:50:25 PM »

I've made some progress.  I've also updated the MAP format document you gave me, Mike.  I'm also converting the file to HTML format so that I can post it on the web.  No sense keeping any of that a secret.  I'll post that up with a link when I get it done.  I have my own domain and I can host it there and not dig into the platoon's bandwidth (not that there will be a stampede for it or anything :wink: ).

I've got a solid feel for the chunk layouts and I should have a program that will fully parse a map file (at the chunk macro level, anyway) soon.  It might be today but I don't want to promise anything.

Begin diatribe:

I'm building the code with Visual C++ V6.  I have the 90 day trial of Visual Studio 2005 but I have a problem with having to buy the whole "studio" when all I want is the C/C++ compiler.  I don't want the other languages since they're MS specific.  Especially since I have to pay a lot more for stuff I don't want.

I'm not enamored of the interface changes between VC6 and VC8.  When you undocked the windows inside VC6 they just became child windows with max/min boxes and all.  The new interface just makes them children with a close button but you have to resize them with the borders and you can't min/max them (or I don't know how if you can).  Consequently, the interface feels very cluttered.  Even on a 21 inch monitor I like to have my working windows maximized.  Partly so I can as much as possible of what I'm working on and partly so I'm not distracted by what's on the periphery. There seems to be a definite shift by many to this "peephole" use of windows that just seems like a waste to me.  Why display a bunch of stuff you don't currently need and obscure parts of what you're actually focused on at the moment?  I don't get it.

I guess I'll have to move to VC8 (part of VS 2005) eventually since DX9 and above doesn't work with VC6.  But I can be pretty stubborn about such things.  I probably won't make the move until I have to.

End diatribe:

Anyway, progress made.  Work continues.  More later.
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #7 on: December 23, 2006, 05:22:01 AM »

Got the MaterialList processing objects created and working.  It's only working on the RSE made maps, now.  I'm pretty sure it won't work on the materials/textures from the plugins because of the different versions.

It does go through the entire map files, now and identifies and reports on all the list chunks.  I found two list chunks that may not exist in the maps created with the plugins (or maybe the ones that Mike made just didn't have those).  The "new" list chunks are "Occluder List" and "VFogList".

Updated the Map info document/webpage a bit more today, too.  Once I get the HTML conversion complete, I'll post it on my website and include links to it here.  It won't be complete by then, but at least anyone interested will be able to look at it and possibly offer ideas.

I'll probably put a link to the source code and executable on my page, too, and update that as often as makes sense.

Here's the dump I just ran from the main menu map for Desert Siege.
===============================================
Opening: F:GamesGhost ReconModsMp1MapMain_menud_mainmenu.map
Success!
Map size: 3116 bytes.
Map buffer allocated.
Map loaded into memory.
Map Date/Time: Thu Jan 31 13:28:26 2002
F:GamesGhost ReconModsMp1MapMain_menud_mainmenu.map is a valid map file.
Chunk Size: 172 bytes.
Chunk Type: 8
Version: 3
Name: MaterialList
Count: 1
Handling Material: 1
Chunk Size: 99 bytes
Chunk Type: 2
Texture File: Material0
Integer value  1: 3F800000
Integer value  2:        0
Integer value  3: 3F800000
Integer value  4: 3F800000
Integer value  5: 3F800000
Integer value  6: 3F800000
Integer value  7: 3F800000
Integer value  8: 3F800000
Integer value  9: 3F800000
Integer value 10: 3F800000
Integer value 11: 3F666666
Integer value 12: 3F666666
Integer value 13: 3F666666
Integer value 14: 3F800000
Float? : 0.050000
Byte ? : 0
Texture Count: 1
Handling Texture: 1
Chunk Size: 65 bytes
Chunk Type: 38
Texture Name: Texture0
Char value:  0
Texture File: shot8.rsb
Integer value  1:        3
Integer value  2:        0
Integer value  3: 3F800000
Chunk Size: 1395 bytes.
Chunk Type: 7
Version: 1
Name: GeometryList
Count: 1
Chunk Size: 4 bytes.
Chunk Type: 9
Version: 1
Name: PortalList
Count: 0
Chunk Size: 4 bytes.
Chunk Type: 40
Version: 1
Name: OccluderList
Count: 0
Chunk Size: 979 bytes.
Chunk Type: 13
Version: 2
Name: LightList
Count: 5
Chunk Size: 4 bytes.
Chunk Type: 55
Version: 1
Name: VFogList
Count: 0
Chunk Size: 4 bytes.
Chunk Type: 10
Version: 5
Name: ObjectList
Count: 0
Chunk Size: 154 bytes.
Chunk Type: 11
Name: RoomList
Count: 1
Chunk Size: 4 bytes.
Chunk Type: 34
Name: TransitionLineList
Count: 0
Chunk Size: 16 bytes.
Chunk Type: 24
Name: PlanningLevelList
Count: 1
===============================================
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Zazoo
Administrator
Hero Member
*****

Rep: 13
Offline Offline

Posts: 1343
2049.50 credits

View Inventory
Send Money to Zazoo

View Profile WWW
« Reply #8 on: January 05, 2007, 04:15:38 AM »

How's it going? Any progress?

~Mike
Logged

Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #9 on: January 05, 2007, 04:51:05 AM »

I've made some.  But the geometry section in the maps I'm looking at are a different version and are formatted differently.  Without some sort of key data to look at, it's slow going trying to figure the different format out.

I'm trying different ideas to see if I can figure out what the data represents.  There isn't a one to one correspondence with some of the stuff.  I think I've identified the vertices.  But there are some odd bits of data after that that _might_ be polygons or triangles.

The map file for the main_menus is relatively small.  If you look at the geometry section and have any ideas to offer about it's layout, I'm all ears (so to speak). Cheesy  I don't believe there are any doors in that map.  And I'm pretty sure it's just one room.

I haven't worked on it in a few days.  Letting my "back brain" work on the problem for a little while.  I'll be back on it whenever I finish up with R6 Vegas (whether or not I will finish the game is still unknown). :hmm:
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Zazoo
Administrator
Hero Member
*****

Rep: 13
Offline Offline

Posts: 1343
2049.50 credits

View Inventory
Send Money to Zazoo

View Profile WWW
« Reply #10 on: January 05, 2007, 10:52:09 AM »

I found some of my old code, and was able to manipulate it into a partial parser/converter.

After the room entry headers, there are two bytes, the purpose -- who knows. But after that, is a long integer that gives the number of vertices in the current room, following that is the vertex array itself (triplets of 4-byte floating point numbers -- XYZ for each vertex).
After the vertex array is another long integer indicating the number of polygon groups in the room.
Each polygon group starts with 39+ bytes of data, the purpose of which I haven't a friggin clue :tongue:. sometimes it's more than 39 (always by a multiple of 4 indicating a long int or float). The only thing I do know, is that three bytes after the start of this data is a long integer that indicates which material goes with the curent polygon group (an index into the material array).

Following that mystery data in each polygon group is the "meat" of the polygon group itself. First is a long integer indicating the number of polygons in the group. After this are three arrays (each containing a number of elements equal to the number of polygons).
The first array contains the surface normals (four 4-byte floats per polygon A, B, C, D)
The second array contains the indices for the polygon's vertices (3 short integers per polygon, one index for each corner of triangle)
The third array contains the texture/vertex normals indices (3 short integers per polygon, these indices refer to the texture attributes arrays that follow)

After these arrays are a few more arrays, these based on texture data.
First is a long integer indicating the number of texture attributes
Following that another long integer, usually "1", I thin it indicates the number of texture maps for each face, but it's allways "1", so no way to tell.
Anyway, the texture attributes section contains 3 arrays (each containing a number of elements equal to the number of texture assignments).
The first array contains the texture/vertex normals (3 floating point values per entry)
The second array contains the XY maps for the texture (two 4-byte floating point values per entry -- position the texture on the face)
The last array contains RGBI data for the face (four 4-byte floating point values per entry -- Red, Green, Blue, Intensity or Alpha)

After this the next polygon group data begins (staring with a new 39+ byte mystery listing, etc.)

I'm fairly certain that what I have identified is correct. I've managed to extract some rooms from the maps:

Red Square (room 1):


Training (room 1):


I'm off to bed pretty soon, but I'll get together my code with comments tomorrow, and type up a basic spec (there are a lot of errors in the docs I linked to earlier... I think those docs are my notes on plug-in generated maps, the default maps seem to have a different layout).

For now, I did zip up the OBJ/MTL file for the first room of the training map, as well as a basic dump of the first room's readable data. Same for the mainmenu map Hopefully this will help:

http://www.mcschell.com/training_map_dump.zip
http://www.mcschell.com/mainmenu_map_dump.zip

~Mike
Logged

Zazoo
Administrator
Hero Member
*****

Rep: 13
Offline Offline

Posts: 1343
2049.50 credits

View Inventory
Send Money to Zazoo

View Profile WWW
« Reply #11 on: January 05, 2007, 04:37:56 PM »

Managed to load the full training map. Can get most of the geometry out of the other maps. A few still choke.

The mystery data seems to be texture/material references. The problem I was having before (with different sized entries) seems to come up when the parser encounters a piece of geometry that isn't textured. This is easy enough to bypass though.

(click for full image)


Full map in 3DS format (~4.2 mb):
http://www.mcschell.com/training.zip

More to follow...
~Mike
Logged

Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #12 on: January 05, 2007, 07:25:10 PM »

That's awesome, Mike! Thumbs up 2 I'm going to work on this tonight.  Once I get the geometry/materials tied together like you have it, I'll look at the other data sections.

Some, like the Occluders, I'm not really sure what they're for.  Maybe LOS blocking of building interiors.

Looks to me like you got pretty far with your work.  You should have published this.  I'm sure it would have gotten a LOT of interest.

I think one reason the modding of OGR didn't go much further than it did was because it required 3DS Max to make new maps.  That's one reason (the other is that I just enjoy doing this sort of stuff) that I'm doing this.  I'm so disappointed with GRAW for the PC that I want to try to revive interest in OGR by making it possible to make maps without a $3500 piece of software.
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #13 on: January 06, 2007, 08:33:05 PM »

@Mike

I worked on my map extractor last night and got the basic mainmenu.map to come out with the same data you have.

However, when I try it on the training.map file, I get very different results than yours.  I'm thinking our training.map files are different.

In the geometry list, the first list of vertices has a different number of vertices (I have about 10 more than you) and they don't appear to be in the same order ,either.

The rest of the data is different, too.  My code works on the first polygon group.  But when it comes to that 39 byte section, there's still data to be read when it thinks it's done.  But I don't know what that data is.  The number don't look like they should for what they're supposed to be, either.

For example, in the vertex texture coordinates, the T value (of S and T) is the same on all of them in the mainmenu map.  That doesn't sound at all right.  

Also, a normal is a vector.  Vectors don't need more than three components.  So why do they seem to have four?  I don't know what those values are, but I don't think they're normals.  They could just have a scale tacked on.  But you don't need that.  Normals don't have to be unit normals.  But things work better if they are.

I'm still going over this geometry data and wondering why it seems that it's close but not the same.

Do you have GR,DS and IT loaded?  Do you have the 1.40 patch loaded?  I do. I'm wondering if that's why some of our data seems to come out differently.

I'm also trying to figure out why it seems like there's data left to process when I think I'm supposed to be at the point to skip over that 39 byte chunk of unknown data.  I still don't have any idea about what that's for.

If that four byte integer in that 39 bytes three bytes into the data is the texture number,  it appears not to be zero based.  It could be but I don't understand why it has a one there when there's only one material/texture (i.e. mainmenu.map) unless they are using a 1 based system.   That would be fairly odd for C or C++ programmers to do when dealing with data that is not end-user visible.

What I haven't figured out is how the materials get linked to the textures (or vice-versa).  I'm looking into that, too.  There isn't always a one to one correspondence there.

It certainly seems like your algorithms for pulling out the data into 3DS files works.  But when I model mine on what you've written it only works on mainmenu.map.  I'm probably missing something or just misunderstanding what you wrote.

The data does seem to be formulaic rather than fully chunk organized (i.e. chunks have structure).  There should be some reason to tell when there's still data to process when I think I'm done with a polygon.  I'm still looking at that.

By the way, my training.map file from the origmiss/map/training directory is 2092938 bytes in size.  If yours is different, that would explain why we get different data from it.
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Zazoo
Administrator
Hero Member
*****

Rep: 13
Offline Offline

Posts: 1343
2049.50 credits

View Inventory
Send Money to Zazoo

View Profile WWW
« Reply #14 on: January 06, 2007, 09:55:26 PM »

Quote from: Ronin;28344

Also, a normal is a vector.  Vectors don't need more than three components.  So why do they seem to have four?  I don't know what those values are, but I don't think they're normals.  They could just have a scale tacked on.  But you don't need that.  Normals don't have to be unit normals.  But things work better if they are.


It's likely not a normal. That was simply the best thing I could come up with. Hadn't put much thought into it as whatever it is, I've been ignoring it on the exporting end up until now (with no major ill-effects on the geometry). The normals I dumped into the OBJ file are the normals given in the texture assignment area (standard ijk vector).
Smoothing data maybe.

Quote
Do you have GR,DS and IT loaded?  Do you have the 1.40 patch loaded?  I do. I'm wondering if that's why some of our data seems to come out differently.


No. I did when I originally worked on this stuff a few years ago. I'm thinking that may be the reason for all the "chokes" my old code hit while parsing the map files. I don't remember having this problem before.

Quote
I'm also trying to figure out why it seems like there's data left to process when I think I'm supposed to be at the point to skip over that 39 byte chunk of unknown data.  I still don't have any idea about what that's for.

If that four byte integer in that 39 bytes three bytes into the data is the texture number,  it appears not to be zero based.  It could be but I don't understand why it has a one there when there's only one material/texture (i.e. mainmenu.map) unless they are using a 1 based system.   That would be fairly odd for C or C++ programmers to do when dealing with data that is not end-user visible.

What I haven't figured out is how the materials get linked to the textures (or vice-versa).  I'm looking into that, too.  There isn't always a one to one correspondence there.


I figured this out today.
That small grouping of data at the begining of each polygon group is where the material/texture assignments are stored.

The data is arranged like this:

Byte - Unknown (always 1)
Byte - Unknown (always 0)
Byte - Unknown (always 0)
Long - Material Index
Long - Number of Textures Indices
Long - Texture Indices (repeated according to the number above)
Long - Unknown (always -1)
Long - DarkMap Index
Long - Unknown (always -1)
Long - Unknown (always -1)
Long - Unknown (always -1)
Long - Unknown (always -1)

The bytes are flags of some sort I'm guessing, but they are never any different in any of the maps I've loaded.
Also, in every map I've loaded thus far the long unknown data is always the same. It seems to be unused.
I have yet to see a grouping that uses more than one texture index (may not be supported by engine).
The darkmap index is an index into the texture array.

Quote

By the way, my training.map file from the origmiss/map/training directory is 2092938 bytes in size.  If yours is different, that would explain why we get different data from it.


My file size is different. What version is attached to the "BeginMap" string at the start of the file?

I've managed to get my parser working so that it can export the geometry and textures for all of the maps in the original GR (no patches/expansions):


I realy hope the format hasn't changed that much between updates. It doesn't seem like they ever added any new "features" to the maps themselves that would require screwing with the format/re-exporting.
If they are different that's going to be a pain in the ass.

I'm still putting together a quick and dirty "readable" parser/exporter from the giant cluster f*ck of code I have now. When I finish it I'll post the source here.
~Mike
Logged

Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #15 on: January 06, 2007, 10:27:58 PM »

Quote from: Zazoo;28346
I figured this out today.
That small grouping of data at the begining of each polygon group is where the material/texture assignments are stored.

The data is arranged like this:

Byte - Unknown (always 1)
Byte - Unknown (always 0)
Byte - Unknown (always 0)
Long - Material Index
Long - Number of Textures Indices
Long - Texture Indices (repeated according to the number above)
Long - Unknown (always -1)
Long - DarkMap Index
Long - Unknown (always -1)
Long - Unknown (always -1)
Long - Unknown (always -1)
Long - Unknown (always -1)

The bytes are flags of some sort I'm guessing, but they are never any different in any of the maps I've loaded.
Also, in every map I've loaded thus far the long unknown data is always the same. It seems to be unused.
I have yet to see a grouping that uses more than one texture index (may not be supported by engine).
The darkmap index is an index into the texture array.

Yeah, I was starting to think that's what those were.  The other values are probably unused texture indices.  They probably planned to have multiple textures but just didn't get around to it.  They might have also been planning to do bump mapping or something.

Quote from: Zazoo;28346
My file size is different. What version is attached to the "BeginMap" string at the start of the file?

It says BeginMapv4.0 on it.  But they may just not bother changing the version number for different versions.

Quote from: Zazoo;28346
I've managed to get my parser working so that it can export the geometry and textures for all of the maps in the original GR (no patches/expansions):


I realy hope the format hasn't changed that much between updates. It doesn't seem like they ever added any new "features" to the maps themselves that would require screwing with the format/re-exporting.
If they are different that's going to be a pain in the ass.

Mike, my friend, you have a gift for understatement.  Cheesy That Red Square map room looks terrific.  Thumbs up 2  Are you loading those into MAX and taking shots from there?

Quote from: Zazoo;28346
I'm still putting together a quick and dirty "readable" parser/exporter from the giant cluster f*ck of code I have now. When I finish it I'll post the source here.
~Mike

Can't wait to see it.  I guess we'll have to keep comparing notes on the different versions of the game.  Although I can load different setups on a couple of machines here and just parse them over the LAN.

I'll be working on this some more this evening.  I'm going to have a nap.  I didn't get to bed until about 4 AM.  And my neighbor woke me up at 8 mowing his lawn (in January).

What programming language and compiler are you using?  I'm still VC 6 but I just installed Visual Studio 2005 Demo.  I'm slow to change what works.  But a lot of stuff with DirectX 9.0 won't compile properly under VC 6.  Oh, well.
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Zazoo
Administrator
Hero Member
*****

Rep: 13
Offline Offline

Posts: 1343
2049.50 credits

View Inventory
Send Money to Zazoo

View Profile WWW
« Reply #16 on: January 06, 2007, 10:40:58 PM »

Quote from: Ronin;28347
It says BeginMapv4.0 on it.  But they may just not bother changing the version number for different versions.


Can you upload a copy of the map file somewhere so I can look at it (I don't have IT/DS available right now)?

Quote
Are you loading those into MAX and taking shots from there?


Yes.

Quote
What programming language and compiler are you using?  I'm still VC 6 but I just installed Visual Studio 2005 Demo.  I'm slow to change what works.  But a lot of stuff with DirectX 9.0 won't compile properly under VC 6.  Oh, well.


Where I haven't used VC regularly for about 5 years now I went with VB 6.
~Mike
Logged

Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #17 on: January 07, 2007, 02:33:48 AM »

Quote from: Zazoo;28348
Where I haven't used VC regularly for about 5 years now I went with VB 6.
~Mike


I think this kind of stuff is actually harder using VB 6 than VC 6.  But hey, whatever works for you. Cheesy  

I can fix you up with DS and IT if you want.  I think I have 3 or 4 copies of the game (can't recall).  At least one (GR Gold) I've never even opened.  Let me know and I'll hook you up. Cheesy
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #18 on: January 07, 2007, 05:58:26 AM »

I think I figured out the discrepancy in the array data.

The size and number of arrays are correct if there's only a single texture.  But you need another set of texture coordinates for each triangle when there's a darkmap (or any other than the main texture).

If you take that first room and put in another texture coordinate array to handle each texture then the offset comes out right.  I think that explains why the main menu was right and the training room was wrong.

What version of the Geometry List and room entries are your training.map, Mike?

Mine are 1 and 8 respectively. (map version 4.0)

I believe the heirarchy of the geometry is list->room->polygon->triangles.  There don't appear to be any primitives represented other than triangles. (no quads, fans, strips, etc.)

I'm changing my parser code right now to see how it handles it with logic for handling darkmap texture coordinates.  I'll report back as soon as I know if it's working.
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Ronin
Full Member
***

Rep: 0
Offline Offline

Posts: 449
664.00 credits

View Inventory
Send Money to Ronin

View Profile WWW
« Reply #19 on: January 07, 2007, 06:31:28 AM »

Okay, I just modified my code to handle dark map (texture) coordinates if there is a darkmap in the texture list for the polygon.  It processed all of the (useful) geometry in the training.map file.

It works fine on the mainmenu.map file, too.  So I think at least handling the bulk of the geometry data is correct.

I don't know what the "NONE" data is or what it's for. That might not be the word "none" but an acronym that RSE created.  I don't have a clue what a NONE acronym could mean, though.

I'm going to study the data in the first part to see if I can figure out exactly what it is and what it's for.  I can't write new map files if I don't know what to write.  I'm sure that would cause a CTD pretty quickly.

I'm also going to explore the other chunk types, too.  Lights, fog, portals and much of the rest I'm pretty sure we can figure out.  Some of the other stuff like Occluders and PlanningLevels might prove more difficult. :hmm:

Anyway, once I'm pretty sure I have the data figured out.  I'm going to write some code to try to display the maps and add the other stuff as I go.  Once I know how to display it, I can be pretty sure I understand the data and should be able to write an exporter for MilkShape (or some inexpensive 3D editor) to build map files.

I got sidetracked earlier today and figured out the format for the GRAW bundle files.  It's not very complicated at all.  I'm going to write my own handler for that, soon.  Nemon's works.  But you can't tell it not to overwrite files. You can't select more than one file.  You can't scale the tree view window.  It doesn't really give you any information about the data in the bundle (how many files,  the size of the file(s) that's selected, etc.)  And it doesn't tell you what's going on while it's extracting.  I also am not a fan of .NET.   I'm also going to put up the source code with it so others can build on it if they want.  But that little project is on the back burner.

TTFN Cheesy
Logged

Just an average schmo.
    * Intel Core 2 Duo 2.4Ghz, 3GB DDR2-667 RAM
    * GeForce 8800GTX PCI-Express 16, 768MB
No XBox 360 :sad:
Pages: [1] 2 3   Go Up
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.4 | SMF © 2006-2007, Simple Machines LLC
SMF Theme © Gaia
Valid XHTML 1.0! Valid CSS!