IPF Library Changes

2010-05-21

Implemented the “fast forward” function required for backward stream decoding. This finally makes it possible to decode all 3 streams required to fully decode/encode a logical block, i.e. data stream, forward gap stream, backward gap stream.

Added the ability to reset the encoder or continue an existing stream when encoding data. This is especially useful for MFM encoding where the previous bit state modifies how the next data should be encoded.

In practice encoding a complete logical block would involve:

  1. Define the start position and buffer size that needs to be filled with the block (all bit based and bit precise as discussed previously). These parameters can be taken from the block descriptors stored in the IPF file, but can be modified on the fly any time, e.g. to modify the data for writing, changing the track size etc - the decoded data would still read as the original, say written with a drive that had different speed, or if the gap size chosen for mastering was different and so on. This makes the format completely flexible and data is fully encoded according to the requested parameters without ever changing the meaning of the data if desired. I.e. only change the track gap size if the desired track length is different from the original disk preserved in IPF, etc.
  2. Reset the encoder and process the data stream.
  3. Calculate the remaining space to be filled (this will be the sector or track gap area), and decide the size to be filled with each gap stream.
  4. Continue encoding with forward gap stream.
  5. Finish encoding with backward gap stream.
  6. You may have noticed that the very first data may be incorrectly encoded at this point if it is encoded as MFM, since it may be modified according to

the cell written right before this data. However at this point the correct value of the previous cell is unknown. Therefore, once all logical blocks have been encoded, there will be a simple fixup phase to all blocks starting with MFM encoded data to correct the affected cells. Luckily, at this point all necessary data is already available and can be pre-cached making the fixup phase almost free.

If you’ve followed the WIPs carefully, missing functionality is (3) and (6). (3) is a bit tedious as the desired stream sizes have to be carefully selected, but all data needed for this decision is already available. (6) is trivial code.

Still to do:

  • Block encoding function as described above; practically code for (3) and (6) everything else works and tested already.
  • Migrate/rewrite library functions to use the new core
  • Generate weak bits when needed, currently areas containing weak bits are just marked and the track buffer is set to a default value that would cause weak bits on real media when written.
  • Re-add the various existing timing models, like Copylock.

Also, eventually add functionality needed to compare IPF images, but that is not top priority. That does exist already, but it won’t work with any new IPF files, since the format has become much more flexible and encoding agnostic, so it needs to be rewritten. As long as we do not release alternate versions it’s not needed, therefore not a blocking issue for releases.