Benvenuto Visitatore(Log In|Registrati)

3 Pagine V  < 1 2 3 >  
Reply to this topicStart new topic
> Game formats converter, Requests and game format research
Nyerguds
messaggio22 Sep 2017, 12:25
Messaggio #26



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




Hahaa! I can save it! Any image that can be divided into smaller blocks can be compressed with this system. And I figured out a pretty fast optimisation system, too. And the resulting matrix table was exactly the same as the one in the original file blush.gif

Now I just need to add the save options for normal BMP, which needs frames instead of one single image.



Ah, crap. I just realised that I have no way to import a single frame, and many of these formats use single frames. I'll need to make a frame import function in the menu specifically, or tweak all the code to allow saving from one image...

I might make some options in the program. I already had auto-zoom in mind. I might add an option to disable auto-frame-detection in there; it's been bothering me sometimes. If I make it a menu option that shouldn't be a problem blush.gif

[edit]

The OFF chunk is clearly not what I thought. It seems to be a bunch of 32-bit integers. Maybe data offsets in the chunk. But it seems the VGA/BIN format doesn't have them, so maybe they're just extra data needed for the SCN and VQT types which I don't know how to open.

Messaggio modificato da Nyerguds il 22 Sep 2017, 14:01


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
Nyerguds
messaggio22 Sep 2017, 14:34
Messaggio #27



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




Released v2.0 v1.2 biggrin2a.gif



Messaggio modificato da Nyerguds il 24 May 2018, 08:55


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio22 Sep 2017, 16:44
Messaggio #28



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




CITAZIONE (Nyerguds @ 22 Sep 2017, 15:34) *
Released v2.0 biggrin2a.gif



I am currently looking into the scummvm code (DGDS engine) to see if I can find something about that pesky VQT. Found nothing so far, unfortunately. Early stage engin and documentation, though.


--------------------
 
Nyerguds
messaggio22 Sep 2017, 17:00
Messaggio #29



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




The research continues... it seems Dynamix frames can only have a width divisible by 8. In fact it might apply to all their image formats. I'll release a new version soon to address that; currently it crashes on uneven frame widths.

Also, I finally dumped the RotD files, and found they use the same tiled matrix BMP image format for their scrolling street backgrounds. There appears to be some text on there, so it looks like my efforts of reconstructing the Matrix format weren't even for nothing biggrin2a.gif


("bum-away activated"... lol)

You see the wall is suspiciously patterned; the blocks are 8x13, and the whole image is built to be optimised by this system. The full image is 2880 blocks, but as you see, with all duplicates removed, the final image only contains 606 (000->605) unique frames.

Unfortunately, the actual matrix block is not compressed so that does take some space, but being an index of mostly consecutive numbers, there is very little repeating data in there, so I guess compressing it wasn't deemed very useful since it wouldn't compress well anyway. The image data itself is compressed after this optimisation, though.

Messaggio modificato da Nyerguds il 22 Sep 2017, 17:17


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio22 Sep 2017, 17:05
Messaggio #30



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




Found something about framed images, could it be of any help?

CITAZIONE
/**
* The animation map is a list of files loaded from an ADS
* RES file that contains filenames and an index. Since the index
* is incremental, theres no need to make this a hashmap.
*/


CODICE
Animation::Animation() {
    _script = 0;
}

Animation::~Animation() {
    if (_script)
        delete _script;

    _animationMap.clear();
}

bool Animation::init(Resource *res) {
    bool ret = false;

    Resource *ver = res->getSubResource("VER:");
    Resource *ads = res->getSubResource("ADS:");
    Resource *adsres = ads->getSubResource("RES:");
    Resource *scr    = ads->getSubResource("SCR:");

    if (ver == NULL || adsres == NULL || scr == NULL)
        return ret;

    _version = ver->to_s();
    delete ver;

    scr->seek(0);
    if (scr->readByte() != 0x02)
        error("[%s] Script Data Corruption", getName());

    uint32 size = scr->readUint32LE();

    assert(size != (uint32)scr->size());
    //Common::SeekableReadStream *decomp = decompLZW(scr, size);
    delete scr;
    //_script = new Resource(decomp, false);

    adsres->seek(0);
    uint32 resCount = adsres->readUint16LE();
    for (uint32 i = 0; i < resCount; i++) {
        adsres->skip(2); // skip over the index
        _animationMap.push_back(adsres->to_s(false));
    }

    delete adsres;
    delete ads;

    debugC(kDebugResources, "[%s] Version[%s] ResourceItems[%d]", getName(), _version.c_str(), resCount);
    for (uint32 i = 0; i < _animationMap.size(); i++) {
        debugC(kDebugResources, "Resource %2d: %s", i + 1, _animationMap[i].c_str());
    }

    return ret;
}


--------------------
 
Nyerguds
messaggio22 Sep 2017, 18:58
Messaggio #31



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




Looks to me like that loads animation scripts, not the actual images...


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
Nyerguds
messaggio23 Sep 2017, 20:26
Messaggio #32



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




v1.2.1 released:
-Fixed crash when saving images with uneven width in Dynamix VGA/BIN format.
-Added tighter checks on Dynamix image input; they can only handle widths divisible by 8.
-Exporting to frames now always takes png as default format.


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio23 Sep 2017, 21:04
Messaggio #33



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




Ahhh! You made it. YOU MADE IT. This is a major update. And I personally appreciate some of the new features (secret ones included, obviously!)! ;-)
CITAZIONE
-Type detection is restricted to the types contained in the chosen type in the Open File menu.
-Added very efficient automatic filter system to determine what types a file can be saved as.
-Changed form size to be able to display a 640x480 image without needing a resize.

Ok, those are mostly cosmetic, but I like all the technical ones too. :P


--------------------
 
Nyerguds
messaggio24 Sep 2017, 10:47
Messaggio #34



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




The format selections one is amazing. It made my life so much easier, really. No more listing exactly what kind of formats can use what other kinds of formats as input or output... it just defines it in general terms (mostly 1-bit, 4-bit, 8-bit and hi-color images) and it works for absolutely everything, completely automatically, just by comparing those flags.

Type restriction is a nice one in my opinion. If you got a .bmp file and you open it, and you specifically select to open as type "image file" (the normal supported image types), it will not try to load it as any of the other types that have .bmp extensions. So you can be sure you're opening exactly what you want to open. And if you don't want that... well just don't select a specific type blush.gif



I'm also considering an auto-zoom feature, which will automatically set the initial zoom on an image to a factor that makes the image largest while still fitting inside the frame, but which is restricted to a maximum you can set in the settings (like, 10x or something). But then I'll need to add a settings dialog and a settings file, too, heh.

Messaggio modificato da Nyerguds il 24 Sep 2017, 11:18


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio24 Sep 2017, 13:30
Messaggio #35



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




Asked Encode.ru about VQT.
https://encode.ru/threads/2837-New-challeng...54490#post54490

They tossed a conjecture but definitely cyrillic to me:
CITAZIONE
I suppose it could be unary coding of log(runlen) + run len.

Maarteeeeeen? Dicaaaaaappaaaaaaa?


--------------------
 
Nyerguds
messaggio24 Sep 2017, 15:43
Messaggio #36



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




No idea what that means...

I tried to replace that bfall.bmp file with an equivalent version using the internal VGA/BIN format, and it seems to work, except that I somehow again forgot to enable that "is container" bit in my converter. Derp. I'll fix it :P


Look at that lovely cyberpunk pink hair!

[edit]

v1.2.2 released blush.gif

Also check out the addons folder:
http://nyerguds.arsaneus-design.com/projec.../release/addon/
This contains the now-usable Elvira items palette blush.gif

Messaggio modificato da Nyerguds il 17 Feb 2019, 11:02


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
Nyerguds
messaggio24 Sep 2017, 16:15
Messaggio #37



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




Seems the sprite is mirrored in that scene, hah.

Maybe this can help them; this is the sprite's original uncompressed data, reconstructed using your screenshot. When I saved this as Dynamix .bmp (with the VGA/BIN internals) it gave a 100% match with the original unreplaced graphics.

CODICE
0000000000000000000000000000FA000000000000000000 .......................
00000000000000000000000000FA9C000000000000000000 .............œ.........
00000000000000000000000000E3D2000000000000000000 ......................
00000000000000000000000000D2E3000000000000000000 ......................
00DA00000000000000000000009CDA00E4DA0000FA870000 ............œ...‡..
00DAE50000000000796D790000DBDA008793E600E329D300 ......ymy...‡“.).
00DAE5BC72727272F0797379F0DA13DB93D3DB00F021E500 .rrrrysy_“.!.
00DAE5DADBDBDBDADA13E379DAD979E5DADA73996EDBDB00 ._yys™n.
0000000000000000DBDB13DAD99CE5DAE279E3DBDBE50000 ........_œy..
F0050000F8BC75CAF0D9DAD92FE5DAE2D6F0000000000000 _..u/......
DAF0BCCAF4D9DA089CF4DA79E5DADA207CDB000000000000 _œy_|......
00DADADADA0000DADA20BCDADA202F79DB00000000000000 ... _/y.......
0000000000000000DADAE5DA7979DBDB0000000000000000 ........yy........
000000000000000000DADADBDADB00000000000000000000 ...................


Messaggio modificato da Nyerguds il 24 Sep 2017, 16:23
Files allegati
File Allegato  bfall.rar ( 1.24k ) Numero di download: 4
 


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio24 Sep 2017, 16:43
Messaggio #38



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




Thanks Maarten, posted to the forum.


--------------------
 
TheRuler
messaggio24 Sep 2017, 17:57
Messaggio #39



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




Shelwien replied:

CITAZIONE
What I mean is,

1) the "raw" file looks like bitcode, visually. But I doubt that its huffman.

2) from repeated patterns in "raw" (which I listed in txt files) we can try finding clues about coding method.
For example, I see lots of "000...01" patterns in there, which could mean unary coding.
https://en.wikipedia.org/wiki/Unary_coding
Like, it could be RLE with unary coding of length - might make sense with small sprites like that.

If that unpacked picture is precise, we can also try finding numbers from it (like "FA") in the "raw".
At least ones that would be encoded by literals with LZ should be there.

3) From the "VQT" name, it could be also some vector quantization
https://en.wikipedia.org/wiki/Vector...ta_compression
But in that case we'd not be able to decode it without actual algorithm.


no FA in original VQT, but I've found something that is definitely in that data you've reconstructed.
this chunk of four bytes from bfall.bmp:
D29CE3DA


--------------------
 
Nyerguds
messaggio24 Sep 2017, 18:08
Messaggio #40



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




Um, I don't find that anywhere in the reconstructed data... though all of those bytes do seem to be on the image, yes.

Messaggio modificato da Nyerguds il 24 Sep 2017, 18:10


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio24 Sep 2017, 18:23
Messaggio #41



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




CITAZIONE (Nyerguds @ 24 Sep 2017, 19:08) *
though all of those bytes do seem to be on the image, yes.

I meant this. They are inverted, between those zeroes at the start.


--------------------
 
TheRuler
messaggio24 Sep 2017, 18:50
Messaggio #42



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




He replied:
CITAZIONE
That's certainly interesting. Doesn't it mean that picture has to be rotated though?
Also, I guess it means that bitcode is 7-0, not 0-7.

Still, you can look for FA as "11111010" in binary code, because compression methods
frequently use codes with variable number of bits.
Its possible that your D2 block just got accidentally aligned.

Seem to be right since you too discovered the rotation!
Convert all to binary and then back would be the solution?


--------------------
 
TheRuler
messaggio24 Sep 2017, 23:22
Messaggio #43



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




More.


--------------------
 
Nyerguds
messaggio25 Sep 2017, 05:32
Messaggio #44



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




Yes, I did look into the exe too. The "BMP:VQT:" string (at dseg:158C) is indeed the lookup of the VQT chunk inside the BMP chunk; they treat it like directories and subdirectories. A bit after that in the code it also looks up "BMP:OFF:", so after that it should add the offset to the VQT chunk data pointer and get the actual data to decompress in there. So somewhere after it should be the function to actually decompress it.

He's wrong, though; there's another one. "SCR:VQT:", a bit above the "BMP:VQT:" string, at dseg:1570. It's referenced at seg007:005D. Since SCR format only contains one image there's no offsets for this one, so it should go to the decompression code right away. But I really suck at reading logic from 16-bit ASM, and clearly his disassembler is better than mine; IDA doesn't even manage to link these references to the data segment, so I suspect I'm missing a lot more information than that.

I'm not sure what he means by "actual processing is done by some overlay" though.

Messaggio modificato da Nyerguds il 25 Sep 2017, 08:08


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio25 Sep 2017, 10:55
Messaggio #45



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




CITAZIONE (Nyerguds @ 25 Sep 2017, 06:32) *
I'm not sure what he means by "actual processing is done by some overlay" though.

I am quite sure he refers to an external overlay file. Most like Centurion, with centurion.exe and OVL0.ovl.
Have to check the presence of such file.


--------------------
 
Nyerguds
messaggio25 Sep 2017, 11:33
Messaggio #46



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




Still, getting the image data unpacked should still be the job of the actual exe file, especially since we have 2 methods (VQT and BIN/VGA) with identical output.


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio25 Sep 2017, 14:13
Messaggio #47



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




This overlay has "VGA:" tag!
Files allegati
File Allegato  SDE.ZIP ( 3.66k ) Numero di download: 2
 


--------------------
 
Nyerguds
messaggio25 Sep 2017, 15:10
Messaggio #48



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




All right, let's see. The VGA chunk uses compression format, but it's set as uncompressed. Interesting. Haven't actually encountered files with a compression header before that don't actually have compression set.

So 0x10 is the compression type (0), and following it is an Int32 with the uncompressed size, which is simply the size of the all the rest of the file, 2150 bytes long, which starts at 0x15. I have no clue about the format, but there does seem to be some kind of index about 0x100 bytes into the file.

That's all I got for now. So, what are these things supposed to do, exactly?

Messaggio modificato da Nyerguds il 25 Sep 2017, 15:22


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 
TheRuler
messaggio25 Sep 2017, 15:53
Messaggio #49



Gruppo icone

Gruppo: SMod
Messaggi: 11.333
Iscritto il: 20 January 06
Da: Bologna
Utente Nr.: 2.653
Entropologo part-time

BGE AG: Indy3
Arcade: Silpheed
Free Roaming: Shadow of the Colossus
RTS: Praetorians

Playing Magari...
SO Windows7

OGI Supporter




Since he didn't find any evidence on the exe, I think he thinks this overlay contains the decompression algorithm.


--------------------
 
Nyerguds
messaggio25 Sep 2017, 17:29
Messaggio #50



Gruppo icone

Gruppo: OldGamer
Messaggi: 257
Iscritto il: 27 June 14
Utente Nr.: 20.463
BGE Command & Conquer: Tiberian Dawn
SO Windows7




I think he's referring to this:
https://en.wikipedia.org/wiki/Overlay_(programming)

DOS programs have this notorious habit of "mutating" in working memory; rewriting their own code in memory to save on used RAM. I know the people who reverse-engineered Dune II (the OpenDune project) actually had to write the debugger-equivalent of a keylogger just to catch all the instructions actually being sent to the CPU, because disassembling the exe itself wasn't sufficient for actually getting all the game code.

I'm not sure that file would be related... I mean it seems related to VGA graphics handling maybe, but nothing seems to indicate it being related to the VQT sections.

Messaggio modificato da Nyerguds il 25 Sep 2017, 17:31


--------------------

Command & Conquer Gold v1.06 - Upgrading C&C95, byte by byte...
 

3 Pagine V  < 1 2 3 >
Reply to this topicStart new topic
1 utenti stanno leggendo questa discussione (1 visitatori e 0 utenti anonimi)
0 utenti:

 

Modalità di visualizzazione: Normale · Passa a: Lineare · Passa a: Outline


Versione Lo-Fi Oggi è il: 13th November 2019 - 10:08