Qualcomm: Rethinking AI in a 5G Quantum World

Last week I attended a coming-out event for some of Qualcomm’s latest processors, a massive update on its artificial intelligence effort (which included an innovative Quantum element), and a connection to 5G most likely didn’t see coming. I think this is a game-changer.

The nature of the San Francisco event was to highlight a counterintuitive impact of the 5G-led revolution: an increase in network edge intelligence.

I’ll make some observations about this revolution and then close with my product of the week: Microsoft’s Vision AI Developer kit, which was on display at this event.

5G Is Here

It is expected that 5G will move aggressively into the market, with near-complete major metropolitan coverage by the end of the year. This technology is a game-changer, but it doesn’t come without issues, one of which is the massive change in network loading. 5G will shift the bottlenecks from the wireless networks to the backhaul, where the expected massive increase in traffic will force some rather impressive network upgrades.

One factor that will drive this massive traffic increase, a 10x increase over today, will be a massive jump in intelligent networked devices. To keep traffic down so the network doesn’t experience a massive bottleneck, it will be necessary to push intelligence to the network edge.

This has led cloud providers to reconsider their huge centralized computing plants and to begin to distribute this technology. It has led firms like Qualcomm to drive far greater intelligence and autonomy into the devices themselves, so they aren’t as reliant on centralized resources, which hopefully will keep network loading to more manageable levels.

Gaming’s Influence

One of the big areas that 5G already has been transforming is hosted gaming. 5G not only provides higher data throughput but vastly lower latency, making gaming from the cloud not only viable but, on paper, far more scalable and lest costly than the traditional methods.

However, to keep data from overcoming the networks, the graphics in large part still remain in the device, along weith some of the intelligence.

I think providers are anticipating an increasingly hybrid gaming experience, so that the social and interconnected aspects of gaming on 5G will be enhanced, but the network won’t become even more saturated with massive video streams. I think this also anticipates a time when the game console may be embedded in a smartphone, tablet, Roku-like set-top box, or even in the TV itself.

Mobile XR Experience

The idea of extended reality in the car, particularly as the car gains the ability to drive itself, is compelling. This is because it could enhance the driving experience significantly if wedded with extended reality (XR). With augmented reality glasses, you could find yourself traveling through imaginary realms and taking part in virtual battles, even fighting with the passengers of other cars while your car autonomously continues on its way.

The final season of Game of Thrones began on Sunday, and I can imagine a long drive looking like a trip from Kings Landing to the Wall in order to fight white walkers using remote-controlled virtual dragons. Yes, you too could become the Queen or King of Dragons!

Powering the Factory of the Future

Factories are going to go through the most massive change of any vertical over the next couple of years, largely a consequence of the disruptive influence of 3D printing.

Manufacturing and quality control equipment will be able to network directly to cloud services that then can manage them effectively across the entire logistics ecosystem.

All of this tied to machine and deep learning cloud services.

Qualcomm’s Client AI

Qualcomm shone a spotlight on the AI Engine in its Snapdragon solution, based on 10-plus years of AI research, noting that makes its solution a winner is the company’s longstanding laser focus on power containment.

This is why Qualcomm’s products are in the vast majority of AR and VR solutions today, because power efficiency increases utility and potentially lowers weight. This focus is one of the reasons Qualcomm has led to the 7nm process being driven mostly by power efficiency but with some performance advantages as well.

Qualcomm also has paved the way for development of a variety of mobile-related capabilities, ranging from extended reality to sound processing.

This has created the potential for inferencing. The market for training and inference will grow to US$17 billion by 2025, based on Qualcomm’s estimates. (I think this is really conservative given the market need.)

Qualcomm believes that the greatest volume, and thus revenue potential, is on the inference side. That likely is correct, because you need only a set number of training platforms, but once the knowledge is captured you then can spread it to huge numbers of inference products that make use of it.

Put simply, you need a lot of teachers, but the numbers of students — inference — tends to be vastly larger.

AI at Facebook

One of the presenters to take the stage at the Qualcomm event last week was Joe Spisak, product manager for AI at Facebook. If there’s a company that desperately needs AI more than Facebook, I don’t know of it.

Facebook has to get its arms around a variety of very difficult problems quickly. Otherwise, fines, litigation and customer attrition may bury it. AI can be used to make social recommendations and machine language translations, to assist with accessibility, and to provide ever-smarter bots and assistants.

In addition, machine-generated content, AR effects and VR experiences are very dependent on AI. However, I think that where Facebook truly needs to apply AI (and I know they know this) is on curating content and ensuring the security of the platform. Fortunately, AI — particularly deep learning AI — is increasingly has become able to deal with these issues at scale.

One of the interesting areas Facebook has been using AI to explore is charitable giving. It has raised more than $1 billion in charitable giving using AI. (I didn’t know this.) This aggressive use of AI has caused Facebook to hit a wall in terms of its ability to build new datacenters, and the power requirements, in particular, have made its expansion unsustainable.

This is clearly at the core of Facebook being on stage at the Qualcomm event, because Qualcomm’s solution leads in power efficiency. Facebook also praised its reliability (Qualcomm hasn’t had the security exposures Intel has had) and low latency.

Apparently, through working with Qualcomm, Facebook has created a unique modular server that allows it to maintain its pace through an easy upgrade path to the anticipated coming technology changes, eliminating constant forklift server upgrades, which are expensive and far too risky at Facebook’s massive scale.

One of the interesting technologies Facebook pointed to was PyTorch, a machine learning framework with an emphasis on eager and graph-based execution, dynamic neural networks, distributed training, hardware accelerated inference, and simplicity — because at scale, complexity is a project killer.

Following its implementation, PyTorch has become the fastest-growing platform in Facebook, and it has increased production by nearly 3x.

Full Software Stack

Given that Qualcomm has announced an accelerator, it doesn’t have to run an operating system — but it does have to be compliant with what is out there.

However, Qualcomm has provided a full software stack to speed solutions to market so that developers can gain the benefits of this technology quickly. One of the partners making use of this is Microsoft.

AI at Microsoft

Microsoft took the stage with a very Azure-centric presentation — not a surprise, given Microsoft’s huge focus on the cloud. Azure currently has the largest geographic coverage of data centers in the world, according to Microsoft.

This coverage creates a huge number of unique problems — for instance, Microsoft must comply with more than 90 distinct government entities in order to function in their respective countries. This has driven the company to rethink how it can provide its services through an approach that is both simple and consistent, to avoid problems with excess complexity at scale. It has been aggressively using AI to solve a multitude of related problems.

Microsoft spoke to the need to address customer issues, as well as the need to use existing resources to make intelligent decisions. The demonstration presented to highlight this used an in-place security camera infrastructure in a hardhat-required area to identify workers who weren’t wearing hats, so that avoidable accidents would, in fact, be avoided. Implementation costs were minimal, because it used an existing system and didn’t require the replacement of the existing security cameras.

Microsoft spoke to Windows Machine Learning (Windows ML), which is an extension of DirectX — a tool for speeding time to market, which already is familiar to many developers. Although it originated on Windows, Microsoft now runs it on Linux as well. (It is fascinating that today’s Microsoft views Linux as a peer, not really a competitor to Windows.)

The company also spoke to two Microsoft products — HoloLens 2 and the Vision AI developer’s kit — that have been making a huge impact on how AI is applied to end devices.

Microsoft introduced the idea of “responsible AI,” which addreses the fairness and reliability of the AI models. There is a huge focus on privacy and security; inclusiveness, so no one is left behind; transparency, so you know what is being done; and accountability, ensuring that the folks who use the tools understand they will be accountable for the results.

The idea is to provide a better product serving the broadest population and having serious focus on social responsibility. Microsoft clearly gets that AIs can be dangerous, and Bill Gates has been outspoken about his concerns that AI tools without these conditions could be very dangerous to the human race.

Qualcomm Cloud AI 100

At the event, Qualcomm unveiled the Qualcomm Cloud AI 100, which delivers a massive performance improvement. The company claimed a 50x performance improvement over prior generations of Snapdragon technology, with more than 350 TOPS at peak performance. As expected, it uses the leading 7nm process.

Sampling is expected in the second half of this year with production in 2020. Qualcomm has promised that this technology will improve AI performance massively without a corresponding massive increase in electrical power use.

AI on Mobile

Mobile presents some very unique challenges. Devices increasingly have become computing-intensive, but they have small form factors and limited power available. The Snapdragon 855 provided significant increases in performance over prior versions, while holding down the power requirements.

This is Qualcomm’s 4th generation of AI with support for all of the top neural network software. (It pounded on Intel’s lack of performance, claiming it had bested its competitor by 2.5x while using significantly less power for the same load levels.)

Qualcomm announced three new mobile offerings at the event. The first was the Snapdragon 665, which provides a massive improvement to low-end smartphones. Its AI engine should provide equivalent image quality to top-line phones in market today.

The second is the Snapdragon 730, which targets higher-end smartphones, enhancing things like smart cropping for pictures, face recognition, augmented reality, and a vastly improved digital assistant.

The third is the 730G, which has a huge focused improvement in graphics for gaming-focused mobile devices.

Qualcomm spoke about its waterfalling strategy: Top features migrate down from its top line 800 platform to its less expensive products from time to time.

One interesting application showcased, which should work across Qualcomm’s platforms, was the ability to take a picture of a menu in a foreign language and display a translated menu on the phone very rapidly (something that would have helped me avoid ordering a pepperoni pizza in Italy that was covered in small peppers, which I’m allergic to, or my wife ordering a fish dish that made her very sick).

Another concept is a self-improving device. Imagine a smartphone that could learn how you worked with it over time and modify itself to optimize your unique practices. This phone then could then pass the learning on to future phones, or other devices that would cross-share this information, making all of them better at working with you. This is one of the huge promises of AI, devices that learn to interface with you rather than the forcing the user to learn the changes in a new device.

This is a unique freedom promise — the idea that we could become the best us, rather than being forced to function in an ever more-defined world of devices that know and care nothing about our once-unique natures.

AI in IoT

This segment opened with a smart home door lock that would use facial recognition to allow access. It not only would eliminate house keys, but also create a record of who entered your home and when, and potentially flag people who weren’t authorized (like a change in your house cleaning staff) that might create a concern.

This could be even more powerful in a corporate structure, where people often wander into secure areas where they shouldn’t be. Such activity could be curtailed more effectively. (I once accidentally opened a door and walked right into a clean room — I was totally lost — likely contaminating the entire line because a side door was left unmarked and unsecured.)

Qualcomm provided examples in retail cameras being used to monitor shrinkage and speed check out at scale (like Amazon Go), smart displays, smart agriculture, energy efficiency, and the implementation of smart cities. Smart tech could bring massive improvements to the effective use of existing city resources. The company showcased a number of products that use Qualcomm technology — from security cameras to digital assistants.

Automotive Tech

In automotive, Qualcomm currently is the leading provider of telematics modems, allowing cars to get updated traffic; get software updates; and call home if they, or their drivers, get into trouble.

The company has been moving aggressively into automotive infotainment. Given the massive performance differences in these systems, and how the poorly performing systems really hurt automotive reviews, I’m anticipating a future when car companies more aggressively call out the solutions they are using.

Intel is in this space, and its solution defines the word “sucks.” I know this because I have it in my newest car, and the performance of the system really has been hurting the reviews (and my own experiences).

Just as with smartphones, Qualcomm is anticipating a more-defined hierarchy of technology in cars — from entry level to super-premium — that will apply both to the cockpit of the car and to the coming wave of autonomous driving solutions. (I’m thinking I may want military-grade autonomous driving technology in my car, given that many of the autonomous driving solutions out there are not very good.)

Qualcomm has a good, better, best line for cars, and that taxonomy is typically thought to be a best practice.

AI Research and the Quantum Edge

There was an interesting session on AI research. Being Qualcomm, the big focus wasn’t on intelligence alone but on power-efficient intelligence. Much of the problem with AI at the edge is that it is very compute-intensive, and compute-intensive things pull a lot of power.

Qualcomm apparently used machine learning AI to make its AI better in terms of intelligence per watt, and it got a 4x performance increase while holding down power use. It believes it can get an increase in power efficiency of up to two orders of magnitude by using Qualcomm’s unique AI execution process.

It has a project in Amsterdam on using quantum field theory with deep learning AIs. Apparently, it has developed a new type of neural network based on this approach, which is far more capable of dealing with 3D objects that are perceived by 360-degree cameras.

The company isn’t planning on developing a quantum computer — yet — but this is the application of quantum math to abstract a distorted image to remove the distortion. I think this could have far broader applications than we currently realize, as it could revolutionize security cameras and optical sensors on a large variety of hardware.

This is potentially a massive game-changer, because it would provide a real-time way to enhance image quality and improve AI recognition significantly, while inferring unseen objects — like hidden running children — to reduce significantly the chance of some common automotive accidents.

If Qualcomm can bring this to market, it would provide the firm a massive advantage in a variety of markets including defense (hidden threat defense).

Wrapping Up

As you can see, this Qualcomm event was a massive information dump, and I’ll be thinking about it for some time. It foretells a future when sub-$500 smartphones are better than today’s $1,300-plus iPhones, when cars will be both smarter and far more entertaining, when virtual dragon hunts on the freeway will be both safe and fun, and when quantum theory will give visual processing AIs the potential superpower of X-ray vision.

We’ll see most of this come to market in the next 12-18 months, and some of the big stuff by the mid-2020s. This once again points out that we really have no idea just how massively changed the world will be in 2030.

When I first looked at Microsoft’sVision AI Developer Kit, it reminded me a bit of the widely criticized Microsoft Bob product.

What Bob did well was make computers simple for those of advanced age, and it was surprisingly effective for that target demographic. It failed because some nimrod at Microsoft decided it would be the next user interface, and developers at that time weren’t fond of GUIs in general and hated Bob out of the box.

Properly targeted, Microsoft’s Vision AI Developer Kit is a game-changer, but it isn’t for those who have a firm handle on AI. It starts with a smart camera based on Qualcomm technology and then helps the user create functioning machine learning solutions that use the camera.

Call it an excellent AI primer for those who don’t code, making it ideal for education or sites where AI needs are focused on problems like security or obvious and recurring manufacturing defects.

Microsoft’s Vision AI Developer Kit

Let’s say, for instance, you wanted to build your own security camera solution that would notify you if your dog got out into your yard (a lot of breeds are escape artists and runners). This would be ideal for that. Or if your son wanted to be alerted when his brother or sister went into his room. In learning how to do this, you’d establish both some core skills and an interest in coding that could transform the child’s — or adult’s — life.

If Microsoft Bob had been targeted properly, I think it would have brought digital assistants to the market much more quickly. Properly positioned, Microsoft’s Vision AI Developer Kit could expand vastly the number of people who understand how AIs function, and encourage them to become AI programmers or techs.

Tools that help people become relevant to the coming changed world are incredibly important, and this is one of them, so Microsoft’s Vision AI Developer Kit is my product of the week.