The New Math of Microsoft Office Compatibility

A recent article by Chris Jackson , Microsoft Office: The Mathematics of Office Compatibility on Technet Magazine made some arguments about the costs associated with a Microsoft office data migration that, in our experience, appeared counter intuitive.

In the article Mr. Jackson provides the reader with figures for the cost of testing and fixing problem files during an Office migration. While we agree with some of these numbers, we struggle with others.

In the article he states, “Assuming you’re tracking document failure, you discover you have a 5 percent failure rate for Office documents in this particular migration. Assuming a user’s time is worth $250 an hour, this failure will cost you $125 times 5 percent, or $6.25. If the amortized cost of your proactive testing process exceeds $6.25, even by just a penny, you shouldn’t bother proactively testing the document at all.”

Typically, we see a lower critical failure rate, 2 percent than the 5 percent noted. At issue is, which 2 percent of the files are going to fail? It’s usually an organization’s most business critical files that are at risk for failure. Further, are you willing to wait for them to fail in production? Mr. Jackson indicates that reactive fixing is what you should plan for based on cost assumptions for proactive testing. It’s important to point out this fact – you are going to have to test those files at some point.

Regardless of when you deal with problematic files let’s look at the costs referenced in the article.

We question if the hourly rate is accurate since, even if it’s a weighted cost, the employee would be earning in the range of $250K (I’ve attached my resume at the end of this document for any prospective employers). Typically we advise a lower rate for cost estimates, but let’s go with the $250 an hour.

Further, we agree with the time it takes to manually test a file. Based on Mr. Jackson’s $125 cost, we assume that he allows 30 minutes per file.

The problem is how Mr. Jackson applies the 5 percent factor, to the per file cost – perhaps it’s new math. Or, it may be a way to amortize that cost across the organization’s file estate. Either way, the cost to repair a given file by his estimations has to be $125 not $6.25 per file, unless you apply that $6.25 to all the files in the organization. This observation is not made in the article.

Typically, our rule of thumb used to factor how many files are likely to be in an organization is this: simply take the number of Office seats and multiply by 1,000 (in the article he uses 500 files per user). Using the article numbers, for an organization of 50,000 office users, this would result in 25 million files, or a total cost of $156 million (25 million * 6.25). We feel it’s simpler to just apply the true cost per file ($125 as above) multiplied by the number of likely problematic files, which, based on 5 percent would be 1.25 million files in a 50,000 user organization. From our experience, it would likely be closer to 500,000 files.

Each organization’s cost will be different but, it’s likely that the cost for having the end-user test files is probably lower than $250 per hour. And, it’s worth reiterating – there’s no avoiding that at some point they are going to have to test the files.

The article suggests that testing be done in live production.  Here we differ in dramatically. Failure, while in production, is never an option for any of our clients.

Full disclosure, as a vendor of a solution targeting this area, we obviously have a horse in this race. We believe that manually scoping and testing (let alone repairing) the 5 percent of your files manually is a hugely daunting task.

We note above that the failure rate we typically see is in the 2 percent range.  This significantly impacts the time needed on the project and the cost of testing. For risk-conscious organizations, we have a prudent solution that allows for the quick identification and remediation of problematic files that takes a fraction of the time indicated in the article. We have seen that using our solution, the total number of files that need end-user review is reduced dramatically, along with the duration of the Office migration project.

We have also seen an increase in organizations that have determined that the end user should be the key person that identifies files of highest importance. We agree that they are most likely to know their critical files and can easily identify what needs to be tested proactively. Mr. Jackson notes this, and again we concur that for some, not all, organizations this is a rational approach. But, the fact remains that the files must be tested and remediated, if required.

As we stated earlier, manual remediation is a very cost prohibitive approach. The ROI, that Mr. Jackson argues for does not make sense. We advise our clients to take a different approach using tools to alleviate a significant portion of the costs, and only repair what needs to be repaired. Leaving this to chance, and rolling out the latest version of Office without some consideration is a very risky approach indeed.