SnTT - SCT, What if you lost the Single Copy Templates

For those who do not use it or ever heard of it, R6 produced the Single Copy Template (SCT) option.

This allowed one to conserve disk space by not loading the whole mail template, or any template for that matter, into the mail file or application.

To enable SCT you can follow the steps in the admin guide as posted here in IBM Technote # 7002574.

Be aware of these details:
Restrictions for Single Copy Template

* To use this feature the templates and the databases linked to them must have an On Disk Structure (ODS) version of 43.
* Once a template has been designated as a Single Copy Template, it cannot be deleted through Notes, and it cannot be deleted while the server is running. Deletions are restricted to ensure that there are no unresolved pointers to the template.
* A Single Copy Template cannot have its design replaced.
* If you change the name of a Single Copy Template the databases that use it will lose their association to it.
* You must run Compact to remove the design elements after running the Convert task (preferably against the Domino \Data directory containing the files changed).
* If you "copy" a new Single Copy Template to the server by creating a New Replica versus a New Copy, the Single Copy Object template database property is de-selected; therefore, it defeats the Load Convert and Compact attempts (leaving you bewildered as to why these processes don't work, and the size of the database never shrinks as it should).

Well guess what happened. Yes, the SCT template was written over. And the backups, being incremental did not include it.

We started getting all kinds of errors that basically said unable to open database because it was missing the single copy database template.

Users could not open their email, email was in some cases stuck in the mail.box files and other scary messages appeared in the server console about attachments being invalid or corrupted attachments.

Go find documentation on how to resolve this problem. Not much, if anything is out there.

But customer pays for IBM support and IBM Support came to the rescue.

With a new mail template matched to the version we enabled SCT again on the template and hard booted the server, this was a key part.

We than ran the convert command.

At the server console run (substitute your template name for mail85):
load convert mail\username.nsf * mail85.ntf

If that does not work, as it did not for some people, you then do a Replace Design of the template, even if it shows the correct name, to the new template and include the folders too.

This will now allow your mail inbox to receive emails. prior to this, and you should verify it with some people, new emails and all other emails "lost" in the interim will be seen under the All Documents view. We tested new emails and found they did not go to the inbox until after this step.

But what about the "lost email" how do you get it back to your inbox?

You need to run an agent against EACH MAIL FILE, IBM was working on a batch utility but I am sure some developer out there could figure one out). The code for the agent is found here at IBM Technote # 1104973.

Go to Code-Agents in Designer and follow the technote. If when you paste the code into the Agent and get any errors, it usually is a break line that is misplaced.

Once run against the database, and it can take quite some time in larger files, the "lost email" returns to the inbox. We did find a small number of messages also returned to the inbox that were orphans, meaning emails without folders or a folder was deleted. These would show up as replication conflicts or similar error and by opening and saving them anew they would revert to normal mails.

Until you get the convert and replace done, iNotes also does not work.

So a lot of effort went into resolving something that should not be so difficult to start. One should be able to replace the design on any file no matter if SCT has been used.

And what did we change on site?
Backups are now 100% full backups and need much more time for a backup window.
SCT database is copied to a USB as well as backed up.
Backups of everything should be done before any work. In this case, there was not enough disk space to back up the Directory. Resolved with an external hard drive since.