On crowd-sourced analytics

The analytics community is a powerful but fragmented one, and that is both good and bad. Good in the sense that for all intents and purposes, every meaningful challenge in the business community has already been solved in one place or another. Bad in the sense that no one of us shows up Monday morning with any more than perhaps 30% of that collective toolset at the ready...at best.

So what do we "quant blue shirts" do when we encounter intractable business problems in our management consulting lives? We reach out to our limited network, and we cobble together the best solution we can given what's been done before, periodically innovating in the margins and typically producing incremental value add. Tight project timelines don't often allow for truly white space analytical thinking, but we get by alright. Thrillin' and billin', right?

These tight timelines, however, have always made management consultancies poor at collecting and leveraging intellectual capital. More devastatingly, there is a culture, particularly among top tier management consultants, to create but not to use intellectual capital. If rewards are given for creating the next great thing, scorn is cast on those poor plebs who, the next week, use what was discovered on another project. And so IC is cast into the abyss of some arcane, mystic IC tracker, like the warehouse from Raiders of the Lost Ark, never to be seen again, next month's IC award go to someone in the Milan office who unknowingly created from scratch a tool or approach that was solved years ago in the Tokyo office (or perhaps in the Milan office).

I would posit for the group that crowd-sourcing analytics is one answer. When an cagey problem arises, it should immediately and without scorn be thrown to the community, who can and will consume it like a swarm of locusts. The collective memory and capability of the community is as sinewy and agile as Ajax (the Homeric Greek, not the language, though perhaps both apply?). The rule of the road will be, "solve problems in your wheelhouse (yours and others') and throw the rest to the group.

I would take this a step further, however, and lay down the claim that the academic community needs to be closer to these problems. (Yes, I said it.) I think the "community" should include students. I remember, as a student, flying through Multivariable Calculus exams that today would make my head spin, and I know that my peers are nodding when it comes to other quantitative methods like econometrics or applies statistics. Even a shaky solution to a simple consumer polling challenge could be corrected with swift precisoin a freshman undergrad who happens to be on that chapter.

And why would they log on in the first place? There's no better way to understand how to apply their academic concepts than to apply them, and there is something appealing about seeing your own wizardry at work in the real world. Plus, students want jobs, and they want jobs they like. Log onto a site from time to time and interact with these blue shirts--you'll see if it's for you, if you like the work in the trenches and if you like the people, the pace, the culture.

For you skeptics, I imagine there are two major hurdles, so let's hash them out. First, everyone wants to hoarde and silo IC, and we even have issues around postings of protected intellectual property. Clearly there needs to be an understanding around client confidentiality. Our clients don't want their analytical laundry aired, and consulting firms don't want their chief value distributed. Both of these, I believe, are manageable. Don't post anything confidential, and don't post anything protected. Most challenges are abstract enough that I feel we can cross that bridge when we get there. And if the problem originates in the community, the solution is one from the community. Information wants to be free anyway, and the realists out there are getting eroded.

The second challenge is that if we blue shirts making hay are leveraging insight from the academic community as well as our peer community, there could emerge the perception that consultants are looking for free labor, getting paid for other people's insight. I don't think this is the point, though I acknowledge it is something for which to look out. The point is to create a symbiotic community. Students get closer to real world challenges and areas to apply their skills and talents, as well as a forum outside of recruiting for interacting with those people with whom they may later choose to work. There's no better way to understand the work that consultants are doing than to dig in on some of their problems and interact with the personalities. Not everyone will want to get involved, but that's true of anything, so let's put it out there and see who picks it up.

Ok, brass tacks. I see two next steps coming out of this. First, we give it a try by posting real questions to real problems on this site, and we start the dialogue today--then see what happens. At the same time, I would be keen to get everyone's perspective on this, students, professors, consultants, would-be clients...

(This effects everyone.)