In all my many years as a Gartner analyst, I could put my hand on my heart and honestly swear that the Magic Quadrant assessment process was wholly impartial. Every vendor, client or not, had the same chance of being accurately – or, for that matter, inaccurately – depicted.
I’m not sure I could say the same today.
Readers of this blog will know there have been a couple of occasions in the past when we’ve felt we had to point out that the playing field was tilted, whether deliberately or inadvertently. And it’s happening again. It’s not going to make me popular in all circles, but I have to call it how I see it.
This is what’s worrying me. And I think it should be worrying Gartner, too.
When what you know depends on who you know
The latest tilt to the playing field occurs because an increasing number of analysts are choosing to publish research notes about the criteria they are using for assessment in upcoming Magic Quadrants. Examples already this year include Configure, Price and Quote Application Suites (31 March), Digital Commerce (18 March), CRM Lead Management (29 January), and IT Service Support Management Tools (22 January).
The problem, of course, is that these research notes are accessible only to those with Gartner subscriptions. Vendors that are not Gartner customers don’t get to see them, as they are not made public.
In terms of honing and polishing their presentations to the analysts, this puts vendors that are the Gartner paying clients at an obvious advantage. But it actually works to the disadvantage of the wider community, including Gartner customers, by tilting the playing field and distorting the Magic Quadrant’s ability to present an accurate and comprehensive overview of the market.
I talk to Gartner analysts all the time. To their credit, many recognize that there’s a problem here. But the last two I’ve spoken to about this have both come up with exactly the same response. “This research note is freely available to any company that may be eligible to be listed, whether or not it is a Gartner customer. People just have to email me and ask for it and I can send it to them.”
Well, that’s good, as far as it goes. It’s certainly commendable that this commercially published research note can be made available to any relevant firm.
But this process itself is anything but a level playing field.
If you are not a client, you need to know that you are entitled to request the research note on the latest MQ criteria. You need to know who to request the document from. And, crucially, you need to know that the document exists to be requested!
It would clearly be far simpler and fairer to simply publish these pieces as research that was accessible to all on gartner.com. If Gartner truly intended to create a level playing field, that would be the best way of showing it.
So what should I do?
If you think this problem could affect you, take my advice. There are three simple steps you should follow.
- Sign up to access gartner.com. Anyone can sign up. You may not be able to read all the research (including these key documents), but you will be able to see titles and other details of what is available. You can set up a keyword-based alert to let you know by email whenever anything relevant is published.
- If you see that one of these research notes on MQ criteria has come out, look at its details on gartner.com and make a note of the name of the analyst who’s written it.
- Email the analyst and demand your rights. If there are multiple authors, start with the first on the list. You can always reach any Gartner analyst by email, using the email@example.com address format. Explain that you are a likely candidate for that Magic Quadrant assessment and ask the analyst to email you a copy of the research note about the MQ criteria.
But what if they ignore me?
This three-step process should be enough to ensure you get to see the same published information as competitors who are Gartner clients. If you are uncertain about what’s going on or you find that the analyst is unresponsive or unhelpful, feel free to let me know. Assessment response and analyst engagement is what we do. We’ll be happy to help you escalate the matter and get that playing field leveled out again.
Are we on target? Have you experienced a situation where you were put at a disadvantage in an assessment? Have your say, and send us any practical tips you’ve discovered that we can share with our readers.
Hi Simon, great tips, thank you. I’ve also started to see some analysts post MQ guidance on the Gartner blog site as well. Obviously they’re free to all to view but you need to know they’re there.
Well done, Simon. I’ve been bleating for years that the placing on an MQ has nothing to do with if a company is a customer or not. But my views are invariably dismissed, which is irritating. The analysts are researchers at heart and, with any evaluation, they evaluate companies as per the evaluation criteria. If they only featured paying customers, any output would automatically be rubbished. I agree with you that the evaluation criteria should be made available to all.