Every once in a while I run into a little company that comes at an existing market as if the folks already in it are idiots — and sometimes they are right. Here’s the thing: What often happens is a company breaks out in a segment, and everyone groups around that company’s ideas and emulates them.
Few initially stand up and say, “Wait a minute — what if they’re wrong?” Often we get so excited about building the market, we don’t realize until much later that the initial attempt at solving a problem doesn’t work. With big data and analytics, the typical project failure rate, depending on how you categorize the projects, can be as high as a whopping 80 percent.
The company I met with is Pneuron, and its approach is very different and, I think, way better — at least when it comes to acquiring the data.
I’ll wrap up with my product of the week: YouMail, an interesting free voicemail service that could solve our nasty robocall problem.
The Problem With Big Data
A few years back, I attended a talk by Harper Reed, the CTO of President Obama’s reelection campaign. Most of the talk focused on how his team used analytics effectively to win, but in one part that stuck with me, he touched on why big data was stupid.
Reed argued — and I agreed — that the focus on big data caused people to do stupid things — like think of ways to aggregate and collect lots of data elements into mammoth repositories that then were too big and complex to analyze. He argued compellingly that the focus shouldn’t be on collecting massive amounts of data, because that just creates a bigger potential problem. Rather, it should be on analyzing the data you have to obtain the answers to critical questions.
I think this explains why so many big data projects fail. Too much time and money is focused on collecting massive amounts of data from systems that never were designed to talk to each other. Consequently, by the time the giant repositories are built, the data is out of date, corrupted, and damn near impossible to analyze.
But what if you didn’t collect the data in the first place?
Pneuron’s Better Idea
What if you left the data where it was, analyzed it in place, and then aggregated the analysis? In other words, rather than aggregating the data, you would aggregate the information you needed from it.
Taking that approach, you don’t create huge redundant repositories, you don’t experience the massive lag of having to move and translate data repositories, and you don’t have the potential data corruption problems. What you do have is a fraction of the cost of any given project.
If done right, you’d end up with a higher probability of being both more accurate and more timely. Your hardware costs would be dramatically lower, and because you were solving the problem in components, you’d actually be able to start getting value before the project’s conclusion.
With each additional repository, the solution would get smarter, as it would be able to answer questions not only from the new repositories, but also from those previously provided. That is basically what Pneuron does.
You’d have to be careful that bias didn’t enter the process through the intermediate analysis, but the risk should be far lower than what naturally would occur when you slammed together data elements that came from very different systems and likely very different ages.
You massively simplify the problem you are trying to solve, and you are better set up for mergers and acquisitions.
Mergers and Acquisitions
An interesting side benefit to this approach is that typically when two companies merge, getting the systems to talk to each other is a nightmare. However, if those systems don’t have to talk to each other, but talk to an intermediate translator instead, the process could be far easier and quicker — particularly if both firms already used that translator.
Imagine two merging firms that wanted to integrate their data bases quickly. If both were using Pneuron’s solution, they would be able to integrate near instantly, without mucking up how things were currently done in either firm.
Conversely, separating the companies again would be equally easy.
Wrapping Up: Lessons Learned
This is less about Pneuron than about the need to step back from time to time and check whether we are full of crap. I recall the huge client/server revolution, when we all ran around singing the praises of something that initially didn’t work, as though it did. Even in the consumer space, we had quadraphonics, 3D TVs, laser disks, and all kinds of gaming systems that folks got excited about but frankly didn’t work.
One of my own stories from a few years ago (and unfortunately I have a lot of these) took place at a Sun Microsystems presentation. The CTO gleefully painted a future when everything — hardware and software — was commoditized. In that world, Microsoft was far weaker.
I raised my hand and pointed out that in that future, Sun didn’t exist. I wasn’t asked back, but that’s the world we live in now: Microsoft is weaker, and Sun doesn’t exist. Had Sun fully understood the world it was helping to create — and assuming it didn’t want to die — it might have chosen a different path.
So I think it is good to keep our eyes out for firms like Pneuron, which point out that our current approach is stupid. Also it might be a good idea to listen from time to time to that voice from the back of the room that says the emperor is naked, because sometimes he is.
I hate robocalls. I get something like 20 a day across four phone lines — mostly from people who are selling stuff I couldn’t buy if I wanted to, because they think I live in San Jose instead of Bend, Oregon. That’s a little side-effect of taking my phone numbers with me when we moved.
I really noticed a huge bump when I moved from T-Mobile — a move I regret — to AT&T. The reason for the move, at the time, was that I was testing a lot of smartphones and they didn’t work with T-Mobile. But someone at AT&T apparently sold my number to folks with robocall machines, because right after the move I started getting those annoying calls.
YouMail is a freemium service that takes over your voicemail and will intercept robocalls for you, giving those systems a tone that tells them your line is out of service, so they automatically drop you from their lists.
It is a free service, and it works mostly with cellphone carriers, so there are some limitations if you are still on a landline. (An interesting side note to that is that according to YouMail, there are only 30 percent of us left who use landlines primarily. Most of us have switched to living on our cellphones.
The service is free for this capability and a voicemail box. YouMail does have additional services that come with extra charges. They might be handy for sales people, but after reviewing them, I think most of us would be fine with just the free service.
If you want to try it out, it is a simple service to add and to get rid of if it doesn’t suit you. There are other solutions in market — like Nomorobo.com (it mostly works with VoIP lines) — that are worth checking out.
Because I hate robocalls, any service that actually gives me freedom from the damn things always makes the short list for my product of the week. Suddenly I have that Braveheart speech ringing in my head. Freedom!!
Loading ...
The idea in fact is not new. The engineering world does this all the time. The data collected by sensors is statistically analysed and aggregated based on Time slice, length, position, item… or some combination of dimensions.
The aggregated results validated to be representative of the actual sub second data.
The aggregates then are used as the data for Analytics.
The aggregations are usually the variance, average, std. dev, range… nothing unusual.
The idea is just like human beings aggregate and form itemized granular opinions based on small subsets or observations of data and aggregate the opinions to form larger conclusions for different circumstances.