According to my wife, it isn’t hard to shop. She loves it, which thrills me, because I hate it. That is, unless we are talking about shopping at a hardware or a winemaking supplies store. I could hang around a good hardware store for hours looking at stuff, especially tools. I want all of them.

But in the IT business, it seems awful. Oh sure, at a high level it should be fun to build or refurbish a data center. Given that every vendor does their very best to feature what they have and downplay what they don’t, it can be hard to get a clear picture of your alternatives.

Additionally, if vendors lie about their products, well, it can confuse and confound reaching a valid conclusion to the point where you feel you are pulling the molars from a Rottweiler without the benefit of an anesthetic (for you or the dog). After all, wasn’t it just $2.4B dollars ago that the HP EVA was a terrific product? Yes it was, right up until they replaced the EVA for something just a smidge better. But from now on, you can count on them telling the truth, I am sure of it.

I know many customers who can tell bullshit from fact from their vendors. It takes work to stay on your game with so many new real and claimed features tossed into the mix. But some folks, especially the gearheads like me, enjoy the exploration. In fact, some of them set traps for vendors to lure them into telling a whopper or two for sport.

My last blog described a tactic that I find shady: claiming that you can offer less than another guy with the same performance or recover vast amounts of wasted space due to sloppy provisioning. While you can make the argument, it is a pretty poor one because thin provisioning doesn’t add performance, nor does thin provisioning save you 50% capacity. Sure, I can spot someone 5% if they delete a lot and end up recovering space. But 50% beggars belief.

Another one we run into is cache size. Here the bullshit flies like snow in Alaska. My favorite one is this: “We don’t need cache because we have sub-LUN auto-tiering.” No, you don’t need cache because you’re stupid. You’d have to be to make such a ridiculous statement.

Don’t take my word for it. Try it yourself: take most of the memory out of your laptop, but leave enough to allow your applications to work. Watch it slow to a crawl. Next, replace the HDD with an SSD. Better right? Sure, a bit better, but nowhere near as good as semiconductor RAM memory attached to the processor. Comparing the difference between a 2.5X performance improvement in access time (from SATA to FC for example) to a 10,000X improvement from FC to RAM is just about the dumbest thing anyone can put forward with or without arms a wavin’.

It’s too bad there’s no Consumer Reports for IT that sorts through the arguments with technical prowess rather than a marketing ear. Unfortunately in our industry, all “evaluations” are done by people paid by the vendors. And I mean all of them, so if you are an analyst you don’t need to get all pissed off at this. It is ubiquitous.

Now I know a few will mention SNIA or similar groups, but they are also biased because the bigger companies put employees into those organizations full time. Additionally, in the case of SNIA, the name says it all: Storage Networking Industry Association. In other words, some of the folks in the organizations are affiliated in some way or another with the vendors themselves.

I think SNIA is great by the way. My only real concern is keeping on the benchmark treadmill, and as I have said before in this regard I agree with Chuck Hollis over at EMC. If you don’t stay current (as we I admit we have not) you run the risk of some goofus pointing at two year old results and saying you don’t perform well, arguing that their new benchmark 4 months later shows you a laggard.

So what were they when we beat them? Or another goofus saying you should have performed better because you have more cache than someone else, despite the benchmark being designed to defeat cache. So I guess we’ll just have to feed the monkey and run another benchmark. In the end it’s better to use a real application in the real environment to make performance assessments. This avoids making extrapolations that can easily go awry.

All this means that it is hard to shop. Really hard, because there is no substitute for knowing what you’re doing and staying informed. Being a good shopper for technology involves study, discipline, and good ears.

Since I don’t want to study clothes, towels, or linens, I will leave that to someone who is good at it and likes it. I will stick to hardware, winemaking, fireworks chemicals, and storage. Oh, and for those who resist shopping for soft goods, try Zappos or Amazon for shoes. You can get what you want, and if you don’t like them, you send them back all from the privacy of your own home. It’s a lot less arduous than going to the mall or a hardware store with your spouse to “have fun shopping.”