The Importance of Digital Preservation

An article about the importance digital preservation. Touching on subjects such as games, copyright life, media life, industry enforcement, and the need to preserve our digital work.

I am deeply concerned about a certain aspect of the new digital world, and the works of human kind that flourish around it. They may one day be lost by the same technology that gave them life...

It seems like only a little while ago, I would be travelling back home from school to further Karadoc along the halls of Castle Wulf, solving puzzles and avoiding minions of the evil necromancer Dianos in the 1990 game “Cadaver” by the Bitmap Brothers. I loved that game. It could be argued that it was little new as far as the genre was concerned - avoid the nasties while navigating a huge map of isometric rooms - but the implementation was second to none. The feeling of adventure, of what lies beyond the next door kept me going... and going. I still like to play it now and again, and I have not yet completed it, probably because I do not want to cheat - Dianos will have beaten me and Karadoc deserves nothing less.

The Bitmap Brothers are known for their unique graphical style.
This particular screen uses only 17 (!) colours through clever use of dithering.

But this is not a article about gaming directly. Though the games I played in my youth certainly are in the forefront of my mind as I write it. What I am actually concerned about, is what happens when my Amiga finally gives up the ghost, or my Cadaver floppy disk dies the death that floppy manufacturers always predicted it would.

Buy another copy? Sure. There are certainly ones available on auction sites. But what happens when all of them die? Floppy disks only reliably last for 10-15 years or so, though in practice most last a lot longer. My Cadaver disk is 13 years old and counting. But what happens when I want to show my children the kind of games I used to play in my youth, let alone my grandchildren?

The fact is that digital media just does not last very long. EEPROM chips as used in arcade machines to store game data are subject to damage by ultraviolet light and electrostatic charge, cassette tapes used by many 8-bit machines are subject to wear, and damage from magnetic sources, floppy disks are incredibly unreliable, to the point that it is sometimes very surprising to see that many still work, though of course many again, do not. They are subject to damage by wear and tear, dirt, liquids, and even from environmental effects over a long period of tim. It is becoming widely known that the early CD’s do not last very long, but even current CD’s are not safe; initial estimates for a 100 year lifespan have recently been cast in doubt. That is referring to commercially pressed CD’s, CDR media is far far less. Most people seem to agree on only a few years, which of course if less than floppy disks.

Keeping these items in conditions safe from the various damaging factors is going to help, but only in the short term, and certainly not my lifetime. What if I actually want to use my old games in twenty years time? Tough luck really.

Well, no. Not quite. We’ll get on to that later.

It gets worse though, many types of media have copy protection, the floppy disk cornered the market there, but even cassette tapes employed quite novel ways to try to detect if a copy was being used. Today, CD’s employ a variety of techniques to hinder efforts to copy them. Digitally preserving these types of media is certainly not a easy task and requires intensive research by very dedicated individuals.

Back in the 1980’s digital distribution was far harder than today. It was the age of swapping by mail and direct-dial bulletin boards (BBS). With that also came the cracking scene and widespread software piracy, removing (or at least attempting to remove) protection on a disk to allow it to be illegally copied and distributed. However, with the race to release a game first, the groups of people involved often produced sloppy bug-ridden versions of the games. Combine that with the ego-driven additions of “intros” 1) proclaiming exclusivity, means that most of the only preservable copies of games remaining today are those that either plainly do not work, only work up to a point, or are littered with cracker graffiti. Some cases even replacing the names of the people who originally developed the game!

Not exactly suitable for preservation, though these are the most common versions found today that are not held in prison by their media. So if a Commodore Amiga games company wants to make their games available for download, they would likely have to use a cracked copy because they had no choice. This has already happened. Thankfully this no longer needs to be the case.

No. I don’t thank the crackers for the software that is left. Perhaps there would be more around, and better taken care of without all the piracy from the late 80’s and early 90’s. Perhaps. The point is that this software is damaged goods, and useless for preservation purposes.

Forgetting cracked versions for a moment, one quite devastating point about the lifetime of these types of media is they don’t outlast copyright. So, even when the time comes when we, or our children can legally play software that have fallen into the public domain, they actually won’t be able to, because all the media that the games resided on died years ago.

Copy protection serves a purpose; games companies could never hope to win the battle against the people illegally distributing their games, and they knew this. One famous misinterpretation of actions by the games industry by software pirates (and one that still exists today) occurred with the release of Robocop 3. This protection was supposed to be un-crackable. It used a hardware “dongle”, and the game would not work without it being plugged in. The reality was that it was not a strong protection, and a pirated version was released very quickly. Software pirates jeered at the publisher’s efforts, and rumours were floated around that the people responsible for the protection had been sacked, and left in disgrace.

But they pirates completely failed to understand the motives of the publisher in this regard. They knew perfectly well it would be cracked. Once a piece of software has been mastered, it is locked in stone and therefore the crackers had a relatively easy job to work round any protection that came their way. Nowadays, in the days of Internet updates, copy protection has become more of a moving target, but it is still in its infancy in this regard. The copy protection on Robocop 3 was not a technological one, it was a psychological one. The media frenzy surrounding the copy protection worked, and in the critical weeks surrounding the games release the perception was that if you wanted to play this game, you would have to buy it. People did. The ploy was successful and the software pirates fell for it, hook, line and sinker. This two-pronged attack on software piracy was not repeated, and this furthered the perception that the copy protection was a dismal failure. In actual fact it was not repeated because it would not have worked a second time.

So, from a publisher’s point of view, the battle over software piracy was one fought in the playground, not with the professional pirates. 2) Casual copying had far worse effects on revenue than dodgy market traders and these dedicated pirates. So in most cases developing strong copy protection was costly and pointless when a simple weak protection would do.

There are laws specifically designed to prevent copy protection being circumvented. Most notorious is the DMCA in the US and soon the EUCD in Europe. Both have exemption clauses for both research and for obsoleteness. But it remains to be seen how the EUCD will be implemented by European member states. Although many of the proposals in these statures are driven by large entertainment organisations and associated lobby groups, and are quite draconian in some ways, they are actually fairly software preservation friendly, certainly not anything like as bad as many people believe. However, the exemptions in the DMCA expire in 2006, and need to be renewed. If this does not happen, then it is indeed a terrible death nail to software preservation projects based in the US.

It would certainly be an ironic twist of fate if the laws designed to protect intellectual property were also the cause of its destruction. For that kind of future nobody wins, but it will be too late to correct those mistakes.

Not a very pleasant picture is it?

So what is digital preservation? You may argue that there are plenty of preserved classics that have been made available for download on the Internet. But I would have to disagree with you almost completely. For one, I don’t count games as preserved if they have been modified in any way by a third party, not only that, I do not count games as preserved if they cannot be proven to be unmodified. This is easily most of the stuff available for many platforms.

Audio tapes are not so bad; it is pretty obvious when they have been modified since covering the write protection hole and pressing record on your tape recording will stop the game working. Of course, it is possible, but it is far more difficult for a casual user. In any case, projects are around then ensure original tapes are preserved. So here I am talking more about floppy disks, where the problem is more common that most people think, and made all the more worse since they were distributed write-enabled!

There are some software companies that do care about their history and either allow their games to be distributed. There are also many that do not, probably due to the fact that anybody that had anything to do with a particular game may have left the company, or the company no longer exists.

It is still quite difficult for the companies that do want to provide their back catalogue for download. Since if the software was copy protected, it is not a trivial matter of getting the data off the disk and into a file that can be used as the disk was. Certainly, very few companies would be willing to put games that had been tainted by the testosterone filled egotism of crackers. It would not be so bad if they had a working unprotected master (if one ever existed) or the source code, but in many cases these things have been irreversibly lost.

It’s not all doom and gloom!

We need to figure out how to really make a concerted effort into stopping all these great pieces of human creation being lost forever. Time for that in a moment, but what can companies developing software do now to stop this kind of situation from re-occurring for their current games?

Well, there are some extreme ideas, and some barely workable ones. I don’t pretend to know the answers but a workable solution needs to be hammered out. People should remember all these treasures of human creation have a time limit and act appropriately; we must succeed in this.

Perhaps games companies should consider defining a policy for long term storage of the works they produce. Only then can the current trend of loss be stopped. A more controversial solution would be to ask/beg/force software companies to submit retail editions of creative software to a digital library or physical museum. This actually happens in the UK for books. One copy of every book published has to be sent, by law, to the British Library. In the software world, it is relatively far less time before something is considered “retro” and “classic”, and games generally have far less appeal to the masses over time that books do.

Of course this would be hard to enforce in a world where the concept is so alien. It can probably be only workable under some kind of voluntary scheme, but it does need widespread industry support, and not just acceptance. Although is probably true to say that the PR opportunities of a games company doing this are limit, but it should be more about pride in your back catalogue!

Of course, there are quite a few games that are now legally distributable. This is truly encouraging, and by digitally preserving these copies (those that are original anyway), means we can take them into the future with us.

A sad fact is, that some may not care about the games they produced years ago, particularly those that own the rights but were not involved with development. It is all about the money. I don’t have anything in particular against commercialism, in many respects in fosters innovation, arguably innovation (on the scale needed for commercial games at the time of production) needs a driving force, money may as well be it.

It may be that nothing will change, and it will be up to those individuals with the passion for these games to make sure they stick around. This may be okay with some people, but it means that the control does not lie with the makers or producers of the games anymore and thus will not necessarily be presented entirely in the way they would wish.

One things is for sure, the technology now exists to digitally preserve magnetic media games no matter what system it is for, no matter what copy protection has been applied. You can find out with absolute surety that a disk has no errors and has never been modified since it was originally mastered. This technology is available for free, and there is no excuse not to take advantage of it.

The battle now only needs to be waged on only one front, the social one. This battle has been waged for some time now, but there are some people who are dedicated by actually doing something to improve the situation rather than wait until there is nothing left....

SPS, The Software Preservation Society (http://www.softpres.org) preserves both physical and digital copies of magnetic disk based games for many systems. The technology is generic and can be utilised to preserve any magnetic disk no matter the system, or copy protection. SPS was founded in 2001 by an Amiga games developer who was tired of seeing pirate games, and the foresight to do something about it before they became the only copies left available.