Protection bashing ...

4 April 2003

Lots of interesting things have happened since the last update. For example, in the space of a few weeks, support was added for practically most of the Commodore Amiga protection formats ever used. As of this WIP, there is only two generic protection schemes that seem to remain. (Remember, protection should not be confused with disk format, though custom disk formats do have the side effect of making copying more difficult.)

1. The generic Atari track. This is actually used as both data format on many games like Last Ninja Remix, and as protection data as on Dungeon Master.
2. The other is another cell density based protection track, used on Defenders of the Earth, Dragon’s Breath, Super Off-Road, New York Warriors, Risk the World Conquest Game and retail versions of most Novagen games. See Figure 1 for density graph.

• Protection data track used on Crack
• CyberDos by Richard Aplin, on Double Dragon 2
• Dragon’s Lair. Unfortunately there is no EDC on the program code, only the data tracks have a checksum.
• Epic (original release) protection tracks
• New format on Eye of the Horus
• Flakey bit protection on the original European releases of Barbarian (Palace), Armageddon Man, Buggy Boy, Three Stooges, Wizball, and an alternate version on ECO (Ocean). Likely to be other games too.
• Support for the flakey bit protection on Fatal Heritage.
• Added a descriptor containing generated data that causes most Factor 5 games to reset their highscore table and produce a version 100% identical to the original. Unfortunately Great Giana Sisters and AdWare games by Factor 5 use a different highscore logic that can’t be reset.
• Protection track (and variants) used on most Gremlin and many US Gold titles, possibly other software houses as well. Some examples: Axel’s Magic Hammer, Dark Fusion, Federation of Free Traders, Switchblade 2, Italy 1990, Shadow Dancer, Turbo Outrun, Chip’s Challenge, Ghouls’n Ghosts, and much more as usual.
• Alternate Gremlin protection track used on Gauntlet 2.
• Protection track used on Infogrames and Ere Informatique / Exxos, Cobra Soft games supported. Examples: Action Service, Captain Blood, Drakkhen, Full Metal Planete, Hostages, Kult, Light Corridor, North & South, Operation Jupiter, Purple Saturn Day, Warlock’s Quest... and possibly much more.
• Anco protection used on Kick Off games, Kick Off 2, 3 and all of their add-ons. (original Kick Off is Copylock, and hence is already supported). The protection might have been used on other games as well. The Anco protection format relies on bit shifting, i.e. Bitshifting that happens on the real media. Supporting this enables mastering or using the data directly without any additional information.
• P.O.W (Actionware) protection
• Millennium, slight variants on Rome Ad’92, James Pond 3
• Protec protection track support, 2 variants. Titles include such classics as Blood Money, Hacker 2, Silkworm, Speedball, Xenon, Menace and Golden Axe.
• Alternate Protec version used on James Pond 2.
• Another Protec version used on some Virgin games, like Deluxe Monopoly, Computer Scrabble Deluxe and Cluedo Master Detective.
• Support for newer Rainbow Arts format used on Garrison data disks. The protection track (which uses new EXT commands) is also supported.
• Protection tracks supported on early Rainbow Art titles, at least Bad Cat, In 80 days around the World, Jinks, and the same protection on Spaceport by Linel. The protection is similar to the one used on Garrison, however it is even more sinister: data is expected to continue within a 16 bit (!) threshold on the adjacent track.
• Protected boot data on certain Psygnosis titles at least on Obitus, Nitro, Armour-Geddon and Killing Game Show. Also, some time was spent on Psygnosis's unknown recording format on these games.
• Slightly different RATT format on Knightmare.
• An old protection format by Rob Northen. Found on RAC Lombard Rally, possibly used on other titles as well.
• Rob Northen, support for later versions of the above protection, on Nebulus and Zynaps, where a key value is used with it.
• Rob Northen, support for alternate version of the same (or clone) protection on Cybernoid and Garfield: Big Fat Hairy Deal.
• Not cell density based Copylock, (see XMark, XData for more info) two variants with slightly different block layout. Known titles: Antheads, Austerlitz, Bloodwych Data Disk, Castle Master, Castle Master & The Crypt, Chase HQ, Cloud Kingdoms, F29 Retaliator, First Contact, Galaxy Force, Gravity, Interphase, Operation Thunderbolt, Pipemania, Rainbow Islands (alt), Resolution 101, Snoopy, Stormlord, Stunt Car Racer, Theme Park Mystery, Tower of Babel...
• TSL, a different format on Pinball Fantasies.

• Rome Ad’92: It is Millennium

• Manually Selected Formats
• Bit Shifting
• Flakey Bit Support
• New datatypes XMark, XData
• New EXT Commands

Fixed a problem where a block without any kind of synchronisation could not be saved since gap information became invalid. This was only possible with blocks that contained certain data, and were not used before.

Found a much more exact way of calculating gap size, as the current method has a very rare problem on some disks (e.g. on Dogs of War track 0.0), reporting zero gap. If any of the saved tracks’ size exceeds a given threshold (0.2%) a warning is given. This way this gap sizing problem can be auto-detected. These gap size calculations are now very much advanced. All the weird exceptions that couldn’t be handled by the previous method are taken care of.

Added an editor option to mark formats as only manually selectable. Such formats are completely ignored by the automatic analysation, but can be selected through either importing or directly. This option is most useful to exclude formats from automatic analysation that contain only generated data or only differ from other formats having additional generated data, since generated data always scores a full match when compared with itself.

The IPF decoder (library) generates bitshifting into the decoded stream automatically at the most likely position it would occur if the data would be written onto a real disk for absolutely faithful reproduction of the real media properties. This helps emulation of certain protections, without any kludges or extra information required to distinguish between data meant for mastering and data meant for actual use (emulation).

Support for protections that explicitly check flakey (random state) bits. This consists of a new FData type, which means flakey data will be inserted at the exact location as marked in the analyser script.

When the decoder library encounters such data under emulation, it duplicates the track and generates four revolutions of data instead of the usual one, where each flakey data is different in the alternate track revolution. This makes the protection checks more likely to pass. Flakey bit protection is very unreliable by its nature... being flakey, and by now it may no longer even work on real hardware.

The mastering software will use special patterns on spot that generates a similar effect on the real media.

Fatal Heritage uses this protection, so in order to play it you will need to download the new user library.

Some time was spent with unsupported Psygnosis titles with unknown recording format like Obitus, Nitro, Armour-Geddon and Killing Game Show.

It is actually very impressive indeed. The games use MFM recording (see MFM resources on the net for more info), but instead of the usual encoding resulting in 2 bits recorded for each bit, the encoding manages the theoretical minimum storage required using an average of 1.5 bits (one and a half bits) encoded size per bit.

The process is very computationally intensive and has to be encoded / decoded on a bit by bit basis instead of the usual hardware/software implementations. It can’t be even sped up by good programming! The meaning of every bit is based on the previous bit state, i.e. it can only be decoded sequentially bit by bit, starting from the very first bit. There is no way to look at the data at any point, it’s plain impossible. But storage space is about 30% more than normal. First we thought it is just some fake decoder to disguise the encoding like on Perihelion, but this one is real. Unfortunately the integrity information can only be accessed by decoding the entire bit sequence (there is no duplicator checksum). This decoding requires a new Psygnosis transport layer in the analyser (see previous WIPs), and a new decoder module in the library. This is quite a lot of work, so we may not continue with it right now.

This game is also interesting since it is definitely duplicated on a Trace, but we believe it was not scripted (see “Freeform” scripting in previous WIPs) as we are pretty sure this was impossible. The data was probably written as a raw bit stream and thus without verification - this is not surprising given Psygnosis duplication track record.

Note: This encoding seems to be used only on 4 titles so far. Probably due to a lot of trouble with the duplication. No big surprise there.

New data types, XData and XMark. They allow a descriptor to insert values into the data stream as if they were read directly by the transfer layer. XData is for data values, XMark is for mark (raw) values to be inserted.

Transparent stream emulation layer is implemented in order to keep these external (hence the ‘X’) values compatible with all parts of the analyser. Since both types are represent a small stream chunk, the values must be set in stream order. As the data types are working on the stream level, all existing and future analyser functions work on them properly.

The new types are most useful on formats where data is likely to be damaged or already unreadable, usually protection formats where very small chunks of real data (such as encryption key or id) is surrounded with flakey (random) data bits. It is also useful to ensure proper data is read each time for such protections making the original loaders much more reliable.

Formats that already benefit are the early Probe games, like Super Hang-On, the encrypted levels of Flimbo’s Quest (from level 2), the very rare original version of Katakis, and the recently added alternate (not density based) version of Copylock. Games using this Copylock version that no longer work from their original disks, now work again!

The following features are useful for certain protections where the protection data continues from one track to another (!), such as the protection used on early Rainbow Arts titles, at least Garrison.

Garrison requires very precise timing to work, however timing can’t be measured by sampling the disk since one of the protection tracks is mostly unformatted giving a different cell timing by each attempt to read. The game itself uses an adjacent track to synchronize the cell reading and reach the desired track position, then quickly switches to the mostly unformatted track containing the second part of the protection.

Original game disks using this protection are not likely to work at all.

Sets track size explicitly, either to an absolute value (in bits) or by using the last valid track size (i.e. formatted tracks only) modified by any amount of bits. The new track size is checked to be large enough to hold all necessary data for the format.

Set data writing position within a track explicitly, either to an absolute value (in bits) counting from the disk index pulse or by using the start position from the last valid track (i.e. formatted tracks only) modified by any amount of bits. The new start position is aligned properly for track data wrapping around in either direction.

Incredibly, the tool that does all the disk dumping magic (CT) had not changed since it first public version in September 2001, though we later added some scripting around it to avoid having to use the command line.

The reason for the update is because unfortunately there is certain hardware that breaks the delicate timing we need to properly dump floppy disks. This is because some hardware generates periodic interrupts and they can be difficult or even impossible to disable.

The hardware essentially seems to be peripheral cards (Serial, USB, SCSI, Ethernet (not PCMCIA) adaptors) and PPC expansions.

To cope with this, we had added new dumping modes to the tool that are less effected by these interrupts and allows us to actually use the dumps for preservation.

Changes include:

• Dumping process is now slightly faster than before
• Recognition of kickstart ROMs dumped with CT and submitted since last update
• ROM file type defaults to .IPF
• Slightly higher compression threshold
• Slightly slower head stepping rate
• Different dumping modes selectable with the new option: “-h[mode]”, where [mode] is one of the following:
  • 0: Original interrupt driven method (default)
  • 1: Interrupt free dumping mode
  • 2: Alternate interrupt free dumping mode