Table of Contents

Protection bashing ...

4 April 2003

Intro

Lots of interesting things have happened since the last update. For example, in the space of a few weeks, support was added for practically most of the Commodore Amiga protection formats ever used. As of this WIP, there is only two generic protection schemes that seem to remain. (Remember, protection should not be confused with disk format, though custom disk formats do have the side effect of making copying more difficult.)

  1. The generic Atari track. This is actually used as both data format on many games like Last Ninja Remix, and as protection data as on Dungeon Master.
  2. The other is another cell density based protection track, used on Defenders of the Earth, Dragon’s Breath, Super Off-Road, New York Warriors, Risk the World Conquest Game and retail versions of most Novagen games. See Figure 1 for density graph.

Fig 1: Cell density based protection track

Disk Formats / Protection Tracks

New

Removed

Analyser and Library Changes

New

Modified

Fixed a problem where a block without any kind of synchronisation could not be saved since gap information became invalid. This was only possible with blocks that contained certain data, and were not used before.

Found a much more exact way of calculating gap size, as the current method has a very rare problem on some disks (e.g. on Dogs of War track 0.0), reporting zero gap. If any of the saved tracks’ size exceeds a given threshold (0.2%) a warning is given. This way this gap sizing problem can be auto-detected. These gap size calculations are now very much advanced. All the weird exceptions that couldn’t be handled by the previous method are taken care of.

Manually Selected Formats

Added an editor option to mark formats as only manually selectable. Such formats are completely ignored by the automatic analysation, but can be selected through either importing or directly. This option is most useful to exclude formats from automatic analysation that contain only generated data or only differ from other formats having additional generated data, since generated data always scores a full match when compared with itself.

Bit Shifting

The IPF decoder (library) generates bitshifting into the decoded stream automatically at the most likely position it would occur if the data would be written onto a real disk for absolutely faithful reproduction of the real media properties. This helps emulation of certain protections, without any kludges or extra information required to distinguish between data meant for mastering and data meant for actual use (emulation).

Flakey Bit Support

Support for protections that explicitly check flakey (random state) bits. This consists of a new FData type, which means flakey data will be inserted at the exact location as marked in the analyser script.

When the decoder library encounters such data under emulation, it duplicates the track and generates four revolutions of data instead of the usual one, where each flakey data is different in the alternate track revolution. This makes the protection checks more likely to pass. Flakey bit protection is very unreliable by its nature... being flakey, and by now it may no longer even work on real hardware.

The mastering software will use special patterns on spot that generates a similar effect on the real media.

Fatal Heritage uses this protection, so in order to play it you will need to download the new user library.

Psygnosis Recording Format

Some time was spent with unsupported Psygnosis titles with unknown recording format like Obitus, Nitro, Armour-Geddon and Killing Game Show.

It is actually very impressive indeed. The games use MFM recording (see MFM resources on the net for more info), but instead of the usual encoding resulting in 2 bits recorded for each bit, the encoding manages the theoretical minimum storage required using an average of 1.5 bits (one and a half bits) encoded size per bit.

The process is very computationally intensive and has to be encoded / decoded on a bit by bit basis instead of the usual hardware/software implementations. It can’t be even sped up by good programming! The meaning of every bit is based on the previous bit state, i.e. it can only be decoded sequentially bit by bit, starting from the very first bit. There is no way to look at the data at any point, it’s plain impossible. But storage space is about 30% more than normal. First we thought it is just some fake decoder to disguise the encoding like on Perihelion, but this one is real. Unfortunately the integrity information can only be accessed by decoding the entire bit sequence (there is no duplicator checksum). This decoding requires a new Psygnosis transport layer in the analyser (see previous WIPs), and a new decoder module in the library. This is quite a lot of work, so we may not continue with it right now.

This game is also interesting since it is definitely duplicated on a Trace, but we believe it was not scripted (see “Freeform” scripting in previous WIPs) as we are pretty sure this was impossible. The data was probably written as a raw bit stream and thus without verification - this is not surprising given Psygnosis duplication track record.

Note: This encoding seems to be used only on 4 titles so far. Probably due to a lot of trouble with the duplication. No big surprise there.

XMark, XData

New data types, XData and XMark. They allow a descriptor to insert values into the data stream as if they were read directly by the transfer layer. XData is for data values, XMark is for mark (raw) values to be inserted.

Transparent stream emulation layer is implemented in order to keep these external (hence the ‘X’) values compatible with all parts of the analyser. Since both types are represent a small stream chunk, the values must be set in stream order. As the data types are working on the stream level, all existing and future analyser functions work on them properly.

The new types are most useful on formats where data is likely to be damaged or already unreadable, usually protection formats where very small chunks of real data (such as encryption key or id) is surrounded with flakey (random) data bits. It is also useful to ensure proper data is read each time for such protections making the original loaders much more reliable.

Formats that already benefit are the early Probe games, like Super Hang-On, the encrypted levels of Flimbo’s Quest (from level 2), the very rare original version of Katakis, and the recently added alternate (not density based) version of Copylock. Games using this Copylock version that no longer work from their original disks, now work again!

New EXT Commands

The following features are useful for certain protections where the protection data continues from one track to another (!), such as the protection used on early Rainbow Arts titles, at least Garrison.

Garrison requires very precise timing to work, however timing can’t be measured by sampling the disk since one of the protection tracks is mostly unformatted giving a different cell timing by each attempt to read. The game itself uses an adjacent track to synchronize the cell reading and reach the desired track position, then quickly switches to the mostly unformatted track containing the second part of the protection.

Original game disks using this protection are not likely to work at all.

Set Track Size

Sets track size explicitly, either to an absolute value (in bits) or by using the last valid track size (i.e. formatted tracks only) modified by any amount of bits. The new track size is checked to be large enough to hold all necessary data for the format.

Set Track Start

Set data writing position within a track explicitly, either to an absolute value (in bits) counting from the disk index pulse or by using the start position from the last valid track (i.e. formatted tracks only) modified by any amount of bits. The new start position is aligned properly for track data wrapping around in either direction.

Dumping Tool Updated

Incredibly, the tool that does all the disk dumping magic (CT) had not changed since it first public version in September 2001, though we later added some scripting around it to avoid having to use the command line.

The reason for the update is because unfortunately there is certain hardware that breaks the delicate timing we need to properly dump floppy disks. This is because some hardware generates periodic interrupts and they can be difficult or even impossible to disable.

The hardware essentially seems to be peripheral cards (Serial, USB, SCSI, Ethernet (not PCMCIA) adaptors) and PPC expansions.

To cope with this, we had added new dumping modes to the tool that are less effected by these interrupts and allows us to actually use the dumps for preservation.

Changes include: