(Updated) How Hard Can It Be To Install Some Memory?

| | Comments (4)

So, as I may have mentioned before, The Pit has a nice fat IBM RS/6000 7026-6H1 unit... 6 CPU, 2GB RAM... not too huge (I mean, I've had co-workers who were able to say they worked on a Cray with serial number "1"), but nothing to sneeze at.

We decided to upgrade it from 2GB of RAM to 4GB of RAM. We call our VAR, we order up some refurb memory (because it's about $4K less than new, give or take), and it shows up.

We schedule a down-time window, we call up IBM service, tell them "Hey, we bought some memory and want you to come out and install it in our box!", they ask "do you have an MES number?" I say "Nope, never heard of it." A slight pause, then "OK, no big deal, blah blah blah..."

Fast-forward to Wednesday night. I drive 1.5 hours to Secaucus (brought George along for company on the road, what the hell?) and wait patiently for IBM. Our field engineer shows up (Tom.. it's almost always either Tom or Fred who gets our tickets). We go to the machine, I power it down, I hand him the memory and basically expect him to go nuts.

He starts looking the memory over. Asks for an MES number. Said "never heard of it, what's that?"

He says "That's what you need to be able to install hardware in your machine. It's what puts it on your service agreement."

There's a panic that sets in. We need this upgrade. We had the AIX professionals come in and hands down, "lack of available memory" was at the top of their list of things to rectify. So I start calling our PurchasingGuy, then I call NewGuyCIO (who I'll have to rename at some point as he's not much of a NewGuy anymore), because I can't get ahold of PurchasingGuy on his cel, basically asking "WTF do I do?"

At one point in the conversation with the engineer, he mentions "well, usually you just buy a one-off service agreement to have us install the memory, and we call it a day." I offer him my credit card right then. Do this, make it get done. It's not that simple, says he, it takes contracts and fax machines, and all sorts of time-delaying bureaucracy.

In the end, the engineer won't touch the memory without the MES #, we don't have one, and we reschedule for Thursday (tonight), hoping that "we can straighten out the paperwork" before 9pm the following evening.

We call Vendor first thing in the morning, saying "where's our goddamn MES #?" They ask us what crack we're on, because you don't get an MES # for refurb hardware, you have to pay IBM to do the install, as it's not covered under the normal support contract.

Nice of someone to tell us that when I called and didn't give them an MES #, eh?

So I call IBM. I explain what happened. They tell me "The engineer should have just checked off a box that said to bill you, they can do that," and is baffled that Tom didn't just do that, especially when I offered him payment for services rendered.

They open a different ticket, which will end up assigned to Fred, which tries to make it really clear what we want to do. We want them to come out, install our refurb memory in our under-contract box, and we want to pay them to do it.

Fred calls me, asks "Did Tom set up a contract this morning for you?" I tell him that I have no idea. He resolves to go off and get in touch with Tom and find out what the hell the status is.

Time ticks. We get impatient. It's now 2pm. Businesses start closing soon, and we need paperwork resolution fairly soon. NewGuyCIO tells me "F-IBM, order some new memory, damn the cost, and return the refurb."

Sounds like a plan to me.

I call Vendor, saying "I want to send this refurb back to you and order some new instead, so I'll have an MES#"

"Errrrr, nocandosir," says he, "We don't deal in new hardware, and we don't really easily take returns on $4,000 pieces of hardware either,"

[insert the sound of me hitting my head on the wall, not at all wanting to hear what NewGuyCIO would say if I had to pass that piece of news back up the chain]

I explain entire situation to VendorRep, including all the bureaucratic discrepancies as to how this should work. He is very nice and very sympathetic, saying things like "We've never had this happen before." He vows to call IBMSvcMgr, and get to the bottom of the situation.

A number of conversations ensue then, between VendorRep, myself, and IBMSvcMgr, wherein he learns all about my crappy day, and how I've pondered the cost-benefit-analysis in my head of filling the RS/6000 full of buckshot and replacing it with a large cluster of cheap commodity servers that I don't have to ask anyone's permission to install memory in.

IBMSvcMgr indicates that bureaucracy probably won't get us a contract today to do the service. VendorRep steps up to the plate and takes it for the team, saying "We have a nationwide support contract #, you do their work, charge us, and we'll worry about billing ThePitOfDespair."

[Insert Heavy Sigh Of Relief Here]

Tom called a couple hours ago to confirm our 9pm appointment tonight. I asked him, "Do you have everything you need in your hands to permit you to install the memory?"

"No," he says. I gulp for a second, but he quickly follows with, "I need the memory."

The memory that is tucked safely in my briefcase next to me as I type this, waiting for him to get here.

How hard could it be?

Updated: I just got home... I'll be back soon it looks like. We got 2 1024MB DIMMS... except on a 6H1 you have to add memory in quads, not pairs.

So Tom came out, looked at the 2 DIMMs in the static bag and said "I dunno... I don't think you can do that..." ... and he was right.

Suck. Ass.

And yes, Z, this is the same machine that's running on degraded power. A cow-orker asked me "What would you do if that other power supply was to die right now?" and only half-jokingly said "Tender my resignation, then run and hide." ;-) He says to me, "What would that be like, if that happened?" I said "There's this huge huge river of feces, and we're in the middle, and there's no paddles in sight." ... all joking aside, the answer I came up with is "I'd tell AT&T to bring the power in right now, and if that meant my mouth was writing a check for five-digits for emergency-anal-rapeage-service-charge, then we'd cut a big fat check and take the pain for it.

4 Comments

Is this the same box that has the wrong power supply in it?

Heh.

IBM's fun, huh?

4K for memory? Used memory? And you can't do it yourself (for tech. or legal reasons?)

Man, sometimes I really love Dell. I might upgrade some RAM tonight just for the hell of it.

Great story, Derek.

Quads ... better you than me!

Well, just got my IBM pSeries 7043 for home
AIX testing from eBay. (AIX rocks.)

Hope I'm luckier with IBM hw than you.

$4k less for 2Gb of refurb RAM?! Damn, gotta love proprietary. Meanwhile Fry's has 1GB PC2700 DIMMs for $149 this week.