The Satisfaction Of A Job Done Well

There's a satisfaction you get from knowing that you've done good work, that you simply can't recreate in any other way.

I had a former employer -- and I use this term loosely, I worked there for less than three weeks in between two jobs, and I was there for such a short period of time that it's not even resumé-worthy -- call me out of the blue this past Thursday. They were down. Hard. Their server was over at a data-recovery place, their RAID array (with the backups) was there too. The hot-spare machine wasn't actually configured right and didn't have data, it was pretty much every organization's nightmare scenario:

The data that is our business is missing, and if we don't get it back soon, we might as well just shut the doors and go home.

The question was, could I come down this weekend, stay in the city, and rebuild their servers while the data-recovery people tried to recover their data. I've become fairly good friends with one of the principals of this company after I left, and frankly I couldn't say "no" to him. His very lifeblood was on the line.

The initial "Data Recovery" we got from their recovery guy sucked. Not nearly enough recovered from the corruption. As I continued building the systems, their programmer started to investigate -- what data could he reconstruct from the remnants, reports, the various system detritus that was left around after batch jobs were done. The results didn't look good. Some of the patient data would be around, but there were also going to be crucial holes, things which could be reconstituted only by paying people who weren't supposed to get paid, and having the error reported to them.

Their data recovery place calls me in my hotel room last night, asking me "how to mount the XFS RAID-5 array". My heart sinks. The guy they've got can't even get this far? What the hell?

The programmer and I confer for a few, and decide we're going to yank the rug out from under the data recovery place. We get the RAID array back, and I start trying to talk to it. To make a long blog entry short, a minor I/O card replacement, and a rebuilt partition table later, and I've done in about two hours of labor what the data recovery people couldn't do in three days.

Now, make no mistake, the Wedding Fund has definitely gotten a nice contribution from this weekend's labor, and D is to be commended for running me down a spare set of clothes as my stay got extended from one night to two on short notice. But the look of sheer joy in the owners of the company as they realized that they were not going to be closing their doors after all... that is a pretty decent reward all unto itself...