Hardware MFM Format Generator

27 September 2004

New Commodore Amiga disk formats or copy protections.

• Alternate Visionary Design format on C64 Emulator (by ReadySoft)
• Virtual Worlds: Flakey bit protection

This is bit is rather heavy going. However, reading though it will be rewarded.

A new data process type has now been implemented that allows on the fly generation of valid scripts which are fed into the analyser. What does that mean? Well, we have talked about “describing” formats in the past, this is where we reverse engineer a disk format and implement that “description” using our Freeform-like disk description language. What we are talking about here is doing that automatically - generating formats that must comply to a certain set of rules, like the limitations of the controller hardware. The Commodore Amiga has no limitations whatsoever apart from the tolerance of the PLL, so automatic script generation is impossible for such a controller.

The analyser “thinks” the generated scripts are a genuine hand written descriptor. It can do all the processing without worrying about special cases that would otherwise arise all the time. Anything can be generated now - even a generated script that generates another one for whatever bizarre reason - and it all works without ever leaving the analysation process, so data that is already known such as filters etc., can be re-used and is fast to process.

Of course, we cannot just create something out of thin air, the logic must be able to “understand” how the data recorded on the disk can be described algorithmically, that is, what are blocks, headers, gaps, what happens if anything is different from normal, how data can be encapsulated into other data, and so on. Should it ever encounter something that it can’t describe properly it must of course recognize that possibility as well and tell the user.

Now, if the script generation succeeds, the actual script (that contained the data type that invoked the script generator) is discarded, and the generated one is processed as a new available format. To avoid excess descriptor generation, complex matching algorithms ensure that only modified parts of data/block/format descriptors are generated and any existing generated descriptors are recycled whenever possible.

The generated descriptors can be seen and edited in the usual way - this is helpful for debugging as well as for fun, but they cannot be saved, and are not saved by design. As usual they can be cloned, so in essence they are real scripts, and are helpful for scripting formats that can only be partially generated.

Generated descriptors are marked with “!”. Purely generated formats are treated as child formats of ones that created them in the first place (the parents). Child formats cannot be selected for a track since they cannot be saved. However their parents are, and when one of those is selected, it automatically generates the child it was designed to create.

When a parent format is selected for viewing, the child will be auto-selected in the format selection dialog. Tracks analysed as child formats are listed the usual way, so each data element and its value can be seen. All in all, child descriptors behave exactly the same as their parents, apart from saving and selection, since the parent script is responsible for their on-the fly generation.

This has been the (very) hard part of generated script handling; making it seamlessly integrated into the existing technology without breaking anything. It is now it is possible to generate scripting logic automagically for various types of “hardware MFM” floppy controller. That is, floppy controllers completely implemented in hardware, like the one in a PC, Atari ST, etc.

To start with, some demo logic was made active. This just creates a format that has 3 sync bytes. By analysing an Atari ST disk, the analyser will sooner or later find out what it is - then the various descriptors can be seen, complete with data listing etc. This is created “on its own”.

Format caching and auto-priorities work. That is, once it finds out about a track, subsequent tracks will first be tested with the selected format generator script, which in turn will generate a new script if needed, etc. Even though child formats may exist, they are never considered as viable formats by the analyser, they are only accessed through their parents. This is very useful, think if a track that has 9 blocks, but another 10 with exactly the same format otherwise, the 9 one won’t be selected by mistake since the generator logic creates a new script with 10 blocks that is only specific to the track being processed.