[PC-DOS] HEROQUEST by Gremlin (1991) REMASTERED, - Modding, Bugfixing e Traduzione ITA/ESP devlog Opzioni

[TheRuler]

opcodes aren't that simple... there's a different code for moving data into each of the different registers (ax, bx, cx, dx, sp, si, etc; the small memory spots on the CPU itself that can be used as temporary storage) and for moving data into pointer locations, and the commands are also different depending on the size of the data to move into that (byte, word, dword, though I don't think dword exists here since it's 16 bit).

Thanks Maarten, that's interesting as hell.
My reminiscences of Z80 studies in high schools are kicking in and I'm wondering why I left that path. I was always amazed by the fact that changing a single byte in a program can change the behaviour or break it. Experimenting without knowledge is fun but hard (not pointless), but seeing the result when you know what you're doing is rewarding.

If I understand correctly the diagram you posted I can write the whole 64bit string using RAX and then I can read or write a single byte. If I want to read the "CD" value I use ah, right? but what if I need to read the position occupied by "67"? rbx or bl?

[Nyerguds]

No, the diagram just shows the way a single register can be split up in smaller parts. "rax" is the 64-bit name for the "a" register. In 64 bit, that's 8 bytes long, but its 32-bit part (so, the lower 4 bytes) can still be accessed in assembly by using "eax", even if there's a larger full value in there. Similarly, its 16-bit part (so, the lowest 2 bytes) can be accessed through "ax".

None of that is really relevant here though, since this is a 16-bit program, so all you have is the 16-bit part, so, in case of the "a" register, that's "ax". But if you look beyond that, things get a bit more interesting: for ax, you can do more than just access its lowest 8-bit part: you can access both the low and high byte of "ax", by using "al" and "ah".

So if you ever encounter "al" or "ah" in the code, you know these are the two bytes that make up the "ax" register.

So yea, that's mainly the cool part of that diagram, but besides that, it also shows the list of all usable registers. Note that *P and *I ones are used for special purposes like stack and variable management; in fact, sp/esp is the stack pointer, which is typically lowered at the start of a function and reset back to its original value when leaving a function; the space between those two points is the memory allocated for that function's local variables. And between these areas on the stack used by each function's variables it saves the address to jump back to when the function ends. The stack works top-down; starts at the end, and gets subtracted until no more is left, so the computer simply knows that "sp=0" means "out of memory". That's what's called "stack overflow"; no more memory available for calling deeper functions.

Messaggio modificato da Nyerguds il 28 Sep 2023, 19:15

[TheRuler]

A little wrap-up of the case I am currently studying:

Upon defeating Bellthor the table turns into the stairs and no text is displayed.
The goal is to make this text, that explains the situation and is actually present in the code, to appear: THE TABLE SLIDES ASIDE REVEALING THE GATE TO THE WITCH LORDS DOMAIN.

I SHOULD be pretty close to the solution, since I've found the address of the function mentioned in my previous post that resets the tiles where the table is and draws the stairs and then returns where it was called from: 0xE457 of quest.exe
 Screenshot_2023_10_02_162043.png ( 4.2k ) Numero di download: 0


I then I tried to point to a different address of the same function to see how it would behaves (red arrow original starting point, blue arrow new one)
 trick.png ( 14.89k ) Numero di download: 5

Regarding the space there is a lot of "unused" space in the exe, especially after its uncompression, but I don't know if it's copied in ram and if it is actually usable or not.
Maybe messing with it could produce unwanted behaviours. Have to play but anyway I think I can squeeze some code into the function that resets the tiles:
trick.png

If my guesses are correct we can draw the stairs in the first 4 blocks, basically shifting up the 4 mov (38,39,3a,3b) overwriting the two mov (0,0), and this should delete the table,
freeing up 10 bytes.
If we need more space we could reduce the stair to one or two tiles only along the wall and I think it would be still convincing, It's a stair concealed underneath a table btw.
In this way we should have 20 bytes free to store the new code.
I still didn't figure out what the last mov actually sets.

So 06 is the code for mov, C6 is byte and C7 is word?

No, it's not that simple. "C6 06 XX XX YY" is simply the code for "mov byte ptr [XXXX], YY", and "C7 06 XX XX YY YY" is the code for "mov word ptr [XXXX], YYYY". You can't really predict these; you just have to look them all up. There might be an actual logic in it, but if there is, it's bit-wise, so not that easy to see in the bytes.

Basically, every single instruction has its own code, and they are built in such a way that the CPU can see from the first byte that more bytes need to be read to get the full command. This system is called "expanding opcodes". If you're familiar with UTF-8, it uses a similar system to encode characters either in a single byte, or in multiple bytes to encode larger unicode values.

Note that in these instructions you do need to pack together only consecutive bytes; there's a 5932h-5934h range and a 594Ch-594Eh range; each writes three bytes, but you can only compact one of each into a word.

So

mov byte ptr [5932h], 0
mov byte ptr [5933h], 38h
mov byte ptr [5934h], 3Ah


mov byte ptr [594Ch], 0
mov byte ptr [594Dh], 39h
mov byte ptr [594Eh], 3Bh


mov byte ptr [3F97h], 1


can become

mov byte ptr [5932h], 0
mov word ptr [5933h], 3A38h

mov byte ptr [594Ch], 0
mov word ptr [594Dh], 3B39h

mov byte ptr [3F97h], 1


...which should save you 8 bytes in total.

Note that I'm not entirely sure about the values; the two bytes might need to be reversed.

Basically bypassing the "table deletion" and the first two stair tiles and draw only the last two stair tiles. And here's the behaviour:
 quest_014.png ( 14.06k ) Numero di download: 0


Then I completely disabled the function replacing its address B82B with 9090 that simply skips to the next operation (90 is the opcode for "no operation"), and this happened:
 quest_015.png ( 13.37k ) Numero di download: 0

The function has been disabled as expected.

Now, since I understood where the code is, I can relocate the function that resets the tiles to another place expanding it with the code able to display the scroll with the text.
Next step: understand that code.

And this is where the scheme maarten provided could come in handy.

... all you have is the 16-bit part, so, in case of the "a" register, that's "ax". But if you look beyond that, things get a bit more interesting: for ax, you can do more than just access its lowest 8-bit part: you can access both the low and high byte of "ax", by using "al" and "ah".

Understand! That narrows the cases in fact.


EDIT:
At the end those 30 bytes (5x6) can be reduced to (6x3) 18! Leaving us with 12 bytes for the code and without changing the appearance of the quest:
 unnamed.png ( 5.86k ) Numero di download: 1

[Nyerguds]

Byte 0x90 is such a handy thing

[Nyerguds]

Then I completely disabled the function replacing its address B82B with 9090 that simply skips to the next operation (90 is the opcode for "no operation"), and this happened:

Ohhh, no, you're wrong there. That's actually a really good way to get crashes.

90 is only "no operation" in program instructions. What you are editing there is not program instructions, it's an address stored as data. If there was a full "call address" or "jump to address" instruction there, and you replaced that entire thing with 90 bytes, then yes, it would work. But it's not; it's only the address. So all you did there was tell the piece of code that tries to access that address to instead jump to address 9090h. Which actually exists. In fact it jumps into the middle of the bytes that make up a full instruction, so that has a very high chance of causing a mess.

Since all the pieces it jumps to have an "end the function and return" C3 byte at their end, the simplest safe way to fully disable that code is to simply give it the address of a C3 byte, so it just executes the return statement.

Though looking at the place where that data is stored, there are in fact places there which seem to be addresses set to 0000, so maybe that's a safe way too.

Messaggio modificato da Nyerguds il 3 Oct 2023, 12:37

[TheRuler]

Ohhh, no, you're wrong there. That's actually a really good way to get crashes.

DANG! You're right. I was just lucky that 9090 was an "apparently" harmful address.


I am obstinate. A new step ahead on the subject "MISSING TEXT WHEN DEFEATING BELLTHOR - ORIGINAL CONTENT RESTORATION":


I've found a way to make A text appear (I wrote A because it's not the text I want) when you defeat Bellthor (but can be any other action) by simply hijack the pointer to the function that make the stairs appear, to another function in the same area of the file (I've searched for C3s that are the RETURN opcode).
 Screenshot_2023_10_05_230226.png ( 156.04k ) Numero di download: 2

In this screenshot every C3 is the end of a function that does something. If I point to right after a C3, from the action I've Hijacked, the game executes that function.

The simplest TEXT I could find was "YOU FIND NOTHING" used during treasure searching that makes the text appear and that's it.
 quest_018.png ( 8.18k ) Numero di download: 0

great, and the code is also 7 bytes so it can fits almost everywhere!
It's basically a MOV and a CALL:
 Screenshot_2023_10_05_231225.png ( 3.47k ) Numero di download: 0

 but the "YOU FIND NOTHING" strangely has 2 instances...
 Screenshot_2023_10_05_231311.png ( 3.48k ) Numero di download: 1

Same instruction (mov bx, 4Ch - call 88EDh) but differently written in the file. SHOOT!
Now I see what you meant Nyerguds with "unpredictable" and I've fallen directly into the valley of despair (dunning-kruger effect). I really know nothing.

Now I also see how they managed to move furniture around and make chests disappear. Manipulating the code in ram, and that makes things much more difficult, considering that we overwrote some code using the "trick" of the length of QUESTxx.BIN for the random events.

but wait, nothing is lost, I've discovered an interesting thing indeed!
The addresses change when loaded into ram but they maintain the same layout as written in the file, so the positions are all relative!
The math is tedious but not that hard, it's a matter of simply finding the right value (you dont' say?)
 Screenshot_2023_10_05_233140.png ( 295.77k ) Numero di download: 3

I took the BB 4C 00 mov instruction as reference, the difference between the first one which calls the address 1C48 and the second one which calls address 19B6 is 292, the exact amount of bytes which separates them! The only caveat (figures) is that the memory page seems to be reversed as compared to the file, since the higher value (1C48) is on top and the lowest (19B6) on bottom.

SO if I manage to calculate exactly the memory address WHERE I want to put my instruction I can finally be able to make a text appear whenever I want!

EUREKA (mostly)

[raul]

Wowwwwwwwwwwww
Congratulazioni...

Per favore, dimmi cosa stai modificando nel codice così posso farlo anch'io.
Non è solo una traduzione, stai già migliorando il gioco.

[TheRuler]

DONE!

 quest_020.png ( 7.68k ) Numero di download: 0

Original content restored!

Since it's a code change in the common exe, it works straight forward also in all the other languages:
 quest_000.png ( 7.56k ) Numero di download: 0


this is the change to be made in QUEST.EXE (offset 0x2FD8) remember to work always with file uncompressed:

old code:
C606325900C6064C5900C606335938C6064D5939C60634593AC6064E593BC606973F01C3

new code:
BB3C02E82F5DC7063359383AC7064D59393BC606973F01C606325900C6064C5900C300C3

I am learning a lot while working on this project! a not so marketable 16bit dos assembly, but I couldn't be happier.
Thanks Nyerguds for the patience, comments and corrections!

Raul, remember to correct those bytes in QUEST.EXE changed because of the editing while still compressed (uncompress both the original and your spanish and compare them to spot the differences, there are a few near the "change configuration" dos prompts).
And report any other content to restore if you happen to remember any!

If you want to test the protection screen in UNCOMPRESSED QUEST.EXE modify offset 0x0541 from 90 90 90 to E8 DC FE

[raul]

Quando finirò questo esame da incubo, avrò molto lavoro da fare su questa traduzione. Aspetterò la fine dell'esame per apportare queste modifiche con pazienza.

La verità è che sono più che soddisfatto di questo progetto, che ha superato tutte le mie aspettative.

[Nyerguds]

Note that depending on the specific uncompress program, offsets in the resulting exe can differ slightly.

I always use the DOS program UNP for unpacking exes.

[TheRuler]

Note that depending on the specific uncompress program, offsets in the resulting exe can differ slightly.
I always use the DOS program UNP for unpacking exes.

That's the only one I've ever used and am aware of.



Slight modification to Bellthor's quest (QUEST19.BIN), there are two chest oddly palced and in a way that you can find yourself trapped if pass through rock is used with only one step left. Same thing for the rock placed in the very next room.
It's very unlikely to happen but getting stuck in this game means PC RESET or alt-f4.

 quest_025.png ( 14.8k ) Numero di download: 0


better arrangement:
 quest_026.png ( 15.77k ) Numero di download: 0

 quest_027.png ( 16.27k ) Numero di download: 0

QUEST19.BIN (Bellthor)
Modificare il byte all'offset 0x5E9, da 5C a 5D.
Modificare il byte all'offset 0x5EB, da 11 a 0D.
Modificare il byte all'offset 0x5B4, da 13 a 12.

[Nyerguds]

Nevermind the translation, you're just making a fan-patch for the game, lol.

[TheRuler]

Nevermind the translation, you're just making a fan-patch for the game, lol.

One thing that always bugged me while using the map was that Barbarian and Wizard icon colors didn't reflect the actual game sprites. They always seemed swapped to me.

 Screenshot_2023_10_11_223615.png ( 60.17k ) Numero di download: 2

So, I've searched in the code for a way to simply swap the pointers, but then I remembered that I could manipulate the graphic instead!
So went to the DK's RAW_VIEW and found the heads in MAPS.VGA, that should be 8bpp (Nyerguds correct me if I am wong).

 Screenshot_2023_10_11_223340.png ( 30.99k ) Numero di download: 4



 Screenshot_2023_10_11_224628.png ( 9.14k ) Numero di download: 0

original Barbarian

 Screenshot_2023_10_11_224646.png ( 8.51k ) Numero di download: 0

original Wizard

Then used my trusty hex workshop to directly change MAPS.VGA file (after decompression) byte by byte with a visual aid:
 Screenshot_2023_10_11_225441.png ( 8.53k ) Numero di download: 0


and that did the trick! I think it turned out pretty good:
 Screenshot_2023_10_11_233407.png ( 5.18k ) Numero di download: 0



For the records:

Original barbarian:
008D8888888D00008D88202223888D008D88222323888D0089883D253D888900888B2220228B8800
888D243D248D88008B90272427908B008D3E0000003E8D00
New barbarian:
002E2D2D2D2E00002E2D2C2C2C2D2B002D2D2323232D2D002D2C3D273D2D2C002D2C2323232D2C00
2C2C2329232C2D002D2C2523252E2D002D2E0000002E2C00

Original Wizard:
003E3D3D3D3E00003E3D3D3D3D3D3E003D3D2223233D3D003D3D2F292F3D3D003D3D2220223D3D00
3D3D3E3E3E3D3D003D003E3E3E003D003D00000000003D00
New Wizard (Original Barbarian):
008D8888888D00008D88202223888D008D88222323888D0089883D253D888900888B2220228B8800
888D243D248D88008B90272427908B008D3E0000003E8D00

[raul]

Incredibile, avete imparato a giocare sempre di più.

[TheRuler]

Incredibile, avete imparato a giocare sempre di più.

E' davvero divertente capire come funzionano i byte!


il mio perfezionismo coi pixel (leggasi OCD) mi ha portato a ritoccare leggermente anche le altre teste per renderle più simili ai personaggi, quindi l'elfo ora ha finalmente gli occhi e un mento, il nano è meno "testa tonda" ha l'elmo e barba più somigliante. Infine ho messo più gradiente nei capelli del barbaro. L'unica testa che non ho toccato è quella del mago (ex-barbaro).

originali:
 Screenshot_2023_10_12_074401.png ( 4.41k ) Numero di download: 0


ritoccati:
 Screenshot_2023_10_12_222802.png ( 5.05k ) Numero di download: 0


e centrati:
 Screenshot_2023_11_01_152154.png ( 3.42k ) Numero di download: 0


Le nuove teste degli eroi (copia e incolla 100h bytes dall'offset 0xD180 di MAPS.VGA decompresso):

00002D2D2D2C2B00002B2D2B2B2C2D2B002C2C2623262C2D002D2C3D273D2C2C002D2D2323232D2C
002C2E2529252E2D002D2F2623262F2D002C2F0000002F2C0000383C3C3C380000383D3A3A363D380
03D3E3E3E3E3E3D003D3A3D253D3A3D003D3A2220223A3D00363E351E353E380000361E361E360000
0036331E33360000003A3634363A00003A36363636363A00383622232336380036363D263D3636003
43623202336360038382527253836003890292829903800383E0000003E3800008D8888888D00008D
88202223888D008D88222323888D0089883D253D888900888B2220228B8800888D243D248D88008B9
0272427908B008D3E0000003E8D

[raul]

Ho un sacco di compiti da fare quando torno a lavorare su questa traduzione.

Ci sono altri testi che non sono stati attivati?

[Nyerguds]

That's some really nice editing work! And yes, one byte per pixel is 8-bit.

Can't that raw_view program load palettes though? That would make things a lot simpler to visualise. You should be able to get the palette out of any DOSBox screenshot. Engie File Converter can export the palette of any image to 8-bit or 6-bit palette. Most such visualiser apps need palettes with 8-bit colour components, which is rather annoying, since the built-in VGA palettes in the data itself generally have 6-bit colour components, since that's what VGA itself actually used.

Messaggio modificato da Nyerguds il 12 Oct 2023, 15:05

[TheRuler]

Thanks!!

Ci sono altri testi che non sono stati attivati?

Li sto cercando, piano piano. Ovviamente tutti i massi cadenti specifici per eroe, ma proprio quelli non saprei come usarli. Sono spazio sprecato, tanto vale usare quello spazio per allargare testi troppo stretti o crearne di nuovi.

Can't that raw_view program load palettes though? That would make things a lot simpler to visualise.

It can't. It's pretty old and I actually never asked the author (DK a member of this forum) about that feature, but it would be nice. I well know that engie can load palette, and it would be a neat feature to open raw generic files as well like raw_viewer and adjust geometry on the fly.
I've streamlined the manual process and now I can make any changes and see the result in one minute, but sure edit the image directly would be pretty awesome.

One thing is the game still require the file to be compressed and even if the fake_encoder works flawlessly, it creates files a lot bigger than the original pnes, 64K compared to 13K, good compresson indeed! I've asked the author again but I don't think we will receive a reply soon, his country, Ukraine is still in critical condition atm.

[TheRuler]

but I don't think we will receive a reply soon

aaaaand he replied!

Raul, when you have time test the new encoder v2 by shelwein, IT WORKS!
 Screenshot_2023_10_13_091716.png ( 71.79k ) Numero di download: 0


Nyerguds, he provided also the code, with his permission and if you're interested obviously, you could implement it in your engie and support HQ image file format too!

[Nyerguds]

Oh, that's neat. It's c++ though... gonna take a bit to figure out. Everything works a bit different in c#, since you normally don't use raw memory pointers there.

Not sure why he would make the encoder without RLE tokens, though, if the format supports it; RLE is generally a lot simpler than lz.

Messaggio modificato da Nyerguds il 13 Oct 2023, 11:09

[TheRuler]

Oh, that's neat. It's c++ though... gonna take a bit to figure out. Everything works a bit different in c#, since you normally don't use raw memory pointers there.

That would be AWESOME.
Making a tool that can read and write HQ graphics will sure lead to mods like proper monsters/boss without reuse of sprites, at least colour/poses variations!
The format is very basich, seems like a tar, they threw in line all the images with different W/H, like in maps.vga: the first is the 320x200 map background and then all 8x8 hero and monster heads (there is something in between that I didn't recognize) and then the furnitures and floor tiles.
How'd you tell what is what automatically?

Not sure why he would make the encoder without RLE tokens, though, if the format supports it; RLE is generally a lot simpler than lz.

Don't know, maybe he works faster with that approach or simply he just edited a code that was just laying around on his pc. They eat bread and LZ for breakfast on encode.su, you know.

[raul]

Raul, when you have time test the new encoder v2 by shelwein, IT WORKS!

Mio Dio, tra 12 giorni finisco il famoso esame!!!!

Nyerguds, he provided also the code, with his permission and if you're interested obviously, you could implement it in your engie and support HQ image file format too!

Che engie ha il Nyerguds?

Messaggio modificato da raul il 13 Oct 2023, 18:53

[TheRuler]

Mio Dio, tra 12 giorni finisco il famoso esame!!!!

In bocca al lupo!

Che engie ha il Nyerguds?

https://moddingwiki.shikadi.net/wiki/Engie_File_Converter

[TheRuler]

Dang. The file MAPS.VGA where heads I've just changed are stored behaves perfectly but in one case.
In quest07, I've found two monsters "invisible" (one is Karlen, easy to reach, just cast "pass through rock" in the starting room on you and go left).
Even if I decode the original one and encode (v2) it back (14KB instead os 13KB) without editing anything it produces the glitch anyways. The monsters are in SPRITES.VGA, not in MAPS.VGA!
If I use the fake encoder it produces a larger file (67KB) and this is the result:
 quest_029.png ( 16.07k ) Numero di download: 0

Changing the memory space occupied by the questxx.bin doesn't produce any benefit but I think It's a memory allocation issue. We overwrite something.
I've also tried trimming down to the original size of the file encoded with v2. No change.
The strange thing is that with v2 encoder the monster is invisible but when he attacks become visible for a couple of frames. Nyerguds, could it be related to the lack of RLE?
Damn, the encoding corrects the animation issue in the intro but breaks this.

EDIT:
I tried to encode other files, here how it behaves with SPRITES.VGA:
 quest_030.png ( 17.62k ) Numero di download: 0

I am a bit ashamed but asked the author. Hope he will came up with a solution or at least a workaround.

[TheRuler]

SCREENSHOT GALORE:
Quest 6 (legacy of the orc warlord) is a strange map and differs in various part from all the other platforms (checked only ZX,C64,Amiga).

The starting room has a pointless secret door, nowhere to be found in any of the others games.
 quest_038.png ( 13.84k ) Numero di download: 0

 Screenshot_2023_10_14_133832.png ( 727.75k ) Numero di download: 1
 Screenshot_2023_10_14_132717.png ( 920.88k ) Numero di download: 0


This is the outside (very tactical):
 quest_039.png ( 16.95k ) Numero di download: 0


I really like more the amiga's layout with the walls behind the monsters:
 Screenshot_2023_10_14_162945.png ( 573.47k ) Numero di download: 0



this is the room where you find the stolen equipment:
 quest_035.png ( 15.64k ) Numero di download: 0


changed to:
 quest_036.png ( 16.71k ) Numero di download: 0


another pointless secret door:
 quest_033.png ( 16.94k ) Numero di download: 0
 quest_034.png ( 9.11k ) Numero di download: 0


changed to the correct shortcut moving the rocks two spaces to the right between the doors.


the last room have the dumbest door layout:
 quest_040.png ( 16.48k ) Numero di download: 0


reverted back to the amiga one:
 Screenshot_2023_10_14_133132.png ( 802.15k ) Numero di download: 0

Where if you have enough movement you can pass by the monster unnoticed