Tuesday 26 June 2012

A real saviour

one thing that is rather common among all adventure games, is not only the ability, but also (without naming names) the NEED to save early and save often, certain games could even make good use of a version control system for that particular thing...

Oh well, puns and jokes aside, last week I added savegame-support in the WME-port, initially this worked fine for everything BUT J.U.L.I.A. and after looking for the problem for a good day, I noticed that the savegame-code for the theora-wrapper had a special case for savegame-versions below a certain threshold, thus reading fewer bytes than was written. (And well, I had reset the savegame-version system to 1.0.0 at the time, since I wasn't aiming for savegame-compatibility anyhow). Fixing that allowed for working savegames in all the games I have tested so far (which would be Dirty Split and J.U.L.I.A.).

I also made some headway with the renaming of functions the past few days, now a good chunk of the functions follow the "funcName()"-convention, instead of the "FuncName()"-convention. A few still remain, and the arguments and variables still stay mostly unchanged.

I also finally got around to compiling the port on a PPC Mac, only to realize how much worked there: the mouse cursor. Yup, some Big-Endian-love is needed to get this train rolling on such CPUs. The main problem at the moment lies in how the scripts are read: *(uint32*)char_buffer... I'm thinking I'll replace most of that code with MemoryStream-code, to make the endian-handling a tad easier (and possibly avoid having to read the entire file in one big chunk at game-start.

The persistence system used in WME originally made a note of the saveable classes by having a static instance of CSysClass per class, this was a bit too global for my tastes, and I have changed that to a function that manually registers all the classes, this way it can be cleaned up properly on exit.

Along with these changes, I also added in load-from-launcher support, and RTL-support, right now the thumbnails are a bit off for load-from-launcher, but the same problem exists for ingame thumbnails too, I guess I'm using a bit too much of the render-surface for the thumbnail, either that, or the downscaling works a bit quirkily.

I'm hoping to have atleast the scripts rolling on PPC this week.

Sorry for the lack of exciting pictures this time, but atleast this means both your game progress and bandwidth can be saved now. 



Thursday 14 June 2012

10 days later

I left the last blogpost with an image of a video from J.U.L.I.A. being played inside ScummVM, but that image still didn't have the clearing of the screen included, so to start of this post, here is how it currently looks:
The J.U.L.I.A.-demo intro

That improvement did however exist at the time I posted that last blogpost, so let's get on with what has improved since then:

TTF-Rendering looks better

Last time I posted a screenshot of the TTF-rendering that looked like this:
How fonts looked last week
Notice that the text is impossible to read, as well as being quite small. There was also an issue with the size of each line making the "descending" parts of characters cut off on the lowest lines. Thanks to a bit of help from LordHoto, I managed to find out that I had been stupid, and used the wrong variables for creating the drawing-surface for the fonts, I had used the font height that the scripts ask for, while in reality I should have used the font height that the TTF-loader in ScummVM returns (which is the height from the lowest point to the highest point, instead of the height from the baseline to the highest point).

While looking through the code to find out why this happened, I took this screenshot, which shows red lines at the points I was asking the TTF-font to render each line:

Why using the wrong height looks wrong.
The fonts were also drawn about too small, as WME used a dpi that was 4/3 of what ScummVM uses in it's TTF-code (96 vs 72), simply multiplying the font-size specified by the scripts made the font a lot better, although, there were still some issues relating to the "layering" that WME does to get the shadowed-fonts it has. As it turns out, this was done by drawing the same line multiple times with different colour-values, at differing offsets. The current draw-code did something rather silly when it got offsets (well, it simply changed the left and top parts of the rect that specifies the draw-area, without making sure the width stayed the same, making it an odd skew-and-scale-operation). Fixing that as well provided the following result:
Font-drawing after fixing offsets and sizing.
Making sure we even bother to use the text alignment the scripts ask for, as well as mostly the right colours, provides quite an improvement for Dirty Split too, compare last posts font-rendering:
Last week's rendering
To this weeks version of the exact same scene:
This weeks rendering
It's not perfect, as the text should be a bit lighter, but it's getting there atleast.

String-concatenation works again:

... and I didn't even know I broke it, well, turns out I must have replaced some variable-names in the II_ADD-script-function, and forgot to make sure everything was changed correctly, which made concatenation of strings always return NULL instead of what it was supposed to return, thus making things like SetImage completely broken, putting that right again, provided one of the biggest leaps of functionality I've seen in a while:
What difference one single variable-name can have.
 Notice that the images on the buttons are in place now in the above picture.

Entity-videos now work too

I forgot that I had to add a few bits and pieces to AdEntity too, to get the videos working for non-fullscreen scenes like this one:
Waking up all alone in the middle of nowhere, literally.
Those little videos also use alpha-masks to allow for non-rectangular videos, in this case, the face in the upper right corner has such a mask to hide the parts that would otherwise fit in the rectangular box it's being played in:
Alpha-masked video
Note though that the current solution for applying alpha-masks is both ugly, and non-endian safe, and was mostly put in to make sure things looked right, I will be cleaning this up in the coming weeks.

As a final comparison: Look at this picture from last week:
Last week's "Enter manual override"
Now compare that to:
This week's version of the same scene
In the upper picture you can see the rectangular box that the videos are being played in, that is hidden by the alpha masking now. Also note the difference in the menu-buttons, and the alignment and presentation of the text. You'd almost think that the demo was completable now...

And it is:

After fixing all of the above, and getting rid of a few hundred megabytes of memory leaks, the game still hung when going down to the planet, so I started looking, and it turns out that the string-splitter function had a bug that caused it to never move on from the first token, but simply continued to push that token to it's result-list, thus eating up a few gigabytes in a matter of seconds. Replacing the STL-string-like solution with ScummVM's Common::StringTokenizer, fixed the issue, and allowed me to complete the planet-side mission, thus letting me complete the last two puzzles.
The last puzzle in the J.U.L.I.A.-demo running in ScummVM

 In the tradition of showing something that DOESN'T work as a closing-point, here's a picture of how J.U.L.I.A. looks when closing the game during the tutorial...


Monday 4 June 2012

A show and tell in 10 steps

If the last post was worth a few thousand words, on account of it's images, then this one might be worth quite a lot more than that, as I've had quite some progress in the visual parts of the Wintermute-engine, without further ado, ten improvements from the last post:

1: Alpha-blending now works properly

As you might remember from my last post, alpha blending was sort of non-existent in the code at that point. I did do specific skipping of completely transparent pixels, but that didn't prove to pretty:
Previous version, without alpha-blending for anything but full transparency
_sev commented on my last blogpost:
"I recommend to take a look at the code of RenderedImage::blit() in engines/sword25/gfx/image/renderedimage.cpp" 
Which I did, and although I haven't completed the refactoring into common-code that he asked for yet, I did add the code to my own branch, which gives the following result:
New version, with alpha transparency
 Notice that the trees look a lot better, along with the bars on the wall. The hair is also less jaggy, although that is not that visible in this small screenshot. A little downside about the current solution is that it is quite CPU-intensive. But that's a problem for another day, for now it atleast works properly.

2: Bitmap-fonts are drawn correctly now

Previously, the bitmap-fonts were drawn with some rather weird offsets, making the characters have odd spacing. The transparency in those images were also quite off:
Previous version, with broken bitmap-fonts
Now, these bitmap-fonts are BMPs, which have rather quirky alpha-channels, the original WMELite-code had this comment hanging around:
// convert 32-bit BMPs to 24-bit or they appear totally transparent (does any app actually write alpha in BMP properly?)
Which explained that part of the problem, I clearly had to replace the existing alpha-channel with the colour key specified by the game.

As for the weird spacing, the engine uses BSurface::IsTransparentAtLite to detect the width of the characters. This is also the reason why the engine explicitly requires the characters to be left-aligned.
Now IsTransparentAtLite was one of the functions that were stubbed when I removed the SDL2-requirement, so adding in new code to check for transparency solved that problem:

New version, with working bitmap-fonts.

3: Sprite scaling works

If you look at that last picture, do you notice anything else that's different from the previous ones? Yeah, Baxter is a bit smaller, and for a good reason, the code I borrowed from the Sword 2.5-engine included scaling before drawing, which solves quite a few issues. Previously if you tried to walk up to the door, Baxter would walk in-place, and not seem to go anywhere, simply because he couldn't "walk into the screen" in any visible fashion (he would have had to get smaller to show it). But now, he's scaled accordingly. That screenshot is actually quite close to this screenshot from the original WME Lite-engine, that you might remember from my last post:
The original WME Lite-engine

Yes, there are a few minor differencies, for one, there's some definite breakage in the gradient that makes up the road he's standing on, but the important bit here is, that this is the very first scene in the game, without any movement done, if you compare this to the first two images in this post, notice how he was way to big to begin with? Well, that's solved now.

4: Sprite mirroring works

One of the features that were missing from WME Lite, when compared to the full Wintermute-engine, was sprite mirroring/rotation, since the Sword 2.5 code I borrowed also contained support for vertical/horizontal flips, it was rather straight-forward to add that little feature back in. Now for testing this, I had to find a game that used it, which in this case was the game Rosemary. Walking left in this game was done by simply mirroring the walk-right sprites, which made it look rather odd without sprite mirroring enabled:
Walking left in Rosemary, without sprite-mirroring
 Now, simply hooking the right data back up to the blit-code from Sword 2.5, made this look way better:
Walking left in Rosemary, WITH sprite-mirroring

5: Fixed some bugs that crept in while refactoring code

Yes, bad things might happen when handling large code-bases, back before the last blog post, I changed a lot of char* into const char* to silence some warnings, and ease the hunting of a few segfaults, this had the interesting side effect of breaking a few virtual-functions, making inheriting classes no longer override the proper functions (C++ can be tricky there, as const char* and char* are quite different types that can be cast only one way). This led to scripts like playRandomMouseOver in Rosemary not running, as the super-class version got run instead of the correct one (the reasoning behind the bug was simple; the function-name was stored in a char* and the super class declared a function that accepted char*  as parameter, while the subclasses accepted const char*, thus the wrong one was chosen).

I also had a few issues with another silly bug: After adding in handling of key-presses, every key seemed to trigger the exit-game dialogue. After hunting that one for quite some time, I realized that I'd made a silly mistake in the event-handling in PlatformSDL.cpp,  right after the case for case Common::EVENT_KEYDOWN: was case Common::EVENT_QUIT: and... there was no break; between them...

6: Fixed speaking to characters (and in doing so, most of the sound)

Previously, if you tried to talk to any character in Dirty Split, the conversation would be cut short, this was because the engine needs to know the length of the text, to know how long it should wait before issuing new lines of text, and unlocking the objects that are speaking. When I did the sound-system before the last blog-post, I only added in the necessary bits for getting something to play, but I didn't bother doing all the various other functions. Specifically GetLength() proved to be quite important for this little tidbit, as it was previously just stubbed to 0. Explaining the rather short conversations.

What remains to be done in the sound-system, is (among other smaller things) supporting loops and seeking in the audio.

7: Added detection for a few more games

I changed the detection-scheme to use the Advanced Detector so that I could easily flag the various games for how compatible they were, as well as detect them based on more detail than the original detection solution. Now, adding every single game to this list will be a never-ending job, as new games are made all the time, but at least the known good and known bad games should be added over time. Particularly the known bad ones should be important to add here (such as all the 3D-games), so that the users will know why their particular game won't work. Another thing planned for this solution, is to have a simple fallback that registers unknown games by simply finding a data.dcp file that doesn't match any known ones. This should allow playing any new or in-development titles without having to first put the hashes into ScummVM.

The currently added games are:
I'll happily add more, if anyone has any good suggestions, but these at least let me test a few of the functions in the engines rather thorougly.

8: TrueType-font support

I have started working on allowing the games to load the TTF-fonts they ask for, which is rather necessary to for instance get subtitles in the conversations in Dirty Split. WME does support TTF-fonts, and defaults to using Arial, this does pose a problem for a ScummVM-port though, as while Arial can be loaded easily on Mac OS X or Windows, from the system-font-directories, various other platforms either lack Arial, or even a system font-concept at all. I'm looking at finding a way to use GNU FreeFont as a fallback alternative for these platforms.

There is also the ever-existing question of what do to if FreeType2 isn't available when building, I guess there will have to be some sort of fallback to one of the GUI-fonts in ScummVM, as a fallback in that case.

Anyhow, TTF-support is still a bit lacking in the WME-port at the moment, as I haven't entirely worked out the details for getting the proper sizes for the fonts, but that's work that will be solved in the coming weeks.

For the curious, this is how TTF-font-rendering looks in WME Lite:
TTF-font in WME Lite
And this is how it currently looks in ScummVM:
TTF-font in ScummVM

For comparison's sake, both of these use Arial at the moment, but long term, if users want to have Arial or something similar used with their games, they'll have to put a copy of the font file in their game folder.

One of the things that are handled fine at this point though, is text-wrapping:
Text-wrapping in J.U.L.I.A.
Yes, it's ugly at the moment, and the picture above even uses a few cheats to get the lower parts of the characters to get drawn (think the "cellar-part" of a g for instance), but it is what it is, a work in progress. 

9: Screen refreshing now works

The original engine cleared the screen to black before every frame, that was another part of the SDL-code that I removed while getting it compiling in ScummVM. At certain points this would mean that parts of the screen that were supposed to be black, just kept what was there in the last frame that updated it. Which isn't really pretty:
J.U.L.I.A.-demo without clearing
Especially the particle engine in the above shot gives use problems. Anyway, making the entire screen draw a color between frames made things quite a lot prettier:

J.U.L.I.A.-demo with clearing
The downside of this though, is that the blitting functions used here aren't exactly free CPU-wise, so having to redraw everything every frame doesn't exactly reduce the CPU-usage. But atleast J.U.L.I.A. looks right now.

10: Theora-video-support

Now here comes the reason why I was talking about images in the opening of this post, because, if one image is worth a thousand words, then full motion video must be worth rather vastly larger sums than that again.

I mentioned J.U.L.I.A. in #9, and that's a game that really needs video support to shine. I got bored of hunting a bug at some point during the past few days, and decided to do something else for a while, which meant adding back Theora-video-support to the engine.

This was yet another field where the Sword 2.5-engine came to the rescue, as it already had a ScummVM-friendly implementation of libtheora, ready for use as a VideoDecoder, which meant that all I had to do was download the original WME sources (after contacting Mnemonic to make sure I could use them under GPLv2, instead of GPLv3), and dig out the removed pieces of code from there. In this case, that meant the code for functions like PlayTheora in AdEntity/AdGame, as well as the Vid-files from Base/ to see what the interface had been originally.

Connecting up the Sword 2.5 wrapper for libtheora to the WME-theora handler wasn't much work, and soon I had video's playing in all my games. A current limitation though, is that seeking is not yet implemented for Theora.

I'll close of this rather lengthy blogpost, with an image of a video playing in J.U.L.I.A.:
The intro video from the J.U.L.I.A.-demo
(note that this image was taken before fixing the screen-clearing issue)