Building Better Neural Networks

Deep neural networks (DNNs) are computing system which base their design on connectivity structure of the brain. They have uses across biomedical science, including some extremely exciting potential in medical image analysis. Given that the brain is regarded as one of the most complex biological systems in existence, it’s no surprise that DNNs are quite a challenge to design and improve. That is a challenge that companies like DarwinAI are meeting head-on. We talked to Sheldon Fernandez, DarwinAI’s CEO, to find out how their technology can make DNNs more compact and efficient.

Ruairi Mackenzie (RM): In lay terms, what exactly is happening when we optimize and improve a neural network?

Sheldon Fernandez (SF): For background, DNNs are complex entities that emulate the cognitive capabilities of the human mind. They are enormously powerful but designing them is extremely time consuming and involves considerable guesswork, even for experts.

DarwinAI's Generative Synthesis technology – the byproduct of years of scholarship from the University of Waterloo – uses traditional Machine Learning to observe a neural network then generate a number of new and highly optimized versions of the network.The result is a set of new and entirely unique networks that are not only significantly smaller than the original but infer faster without sacrificing functional accuracy. Moreover, the understanding garnered by the engine enables explainable deep learning, where the platform can illuminate how a network is reaching its conclusions.

RM: How does DarwinAI’s platform improve neural networks when used in combination with Intel technology?

SF: We use AI itself to generate new versions of a deep neural network that are considerably more compact and efficient than the original. The process is analogous to deconstructing a house, then building a new one with AI, using the original components, that is more efficient, lightweight and stronger than the original.

After the Generative Synthesis process, our new neural network is given as an input into Intel’s software, which further optimizes the model for Intel chipsets. The result is a complementary process that produces impressive results. As reported by Intel’s Solution Brief, in combination, both techniques produced image classification results 16.3X faster than the original.

RM: Why are neural networks useful for image classification?

SF: Neural networks are quite useful for imagine classification because there is a lot of labelled data that makes them extremely adept at the task.

Consider, for example, the ability of a network to identify a picture of a lion. For decades, researchers in image recognition struggled mightily with this problem. While identifying a visual pattern might be straightforward for a human, it is profoundly complex for a machine. How, for example, does one describe what a lion looks like to a computer in mathematical terms given the thousands of ways one can be portrayed in a picture? With a neural network, however, the problem becomes tractable, if still difficult. By providing the network with a million lion pictures, the ‘weights’ in the network can be incrementally adjusted until the system gets quite good at identifying lions

Image classification neural networks are used in everything from autonomous vehicles, to facial recognition systems, to navigation systems for drones and airplanes.

RM: Generative Synthesis is built on “AI building AI”. Can you explain this concept in more detail?

SF: While tremendously powerful, deep neural networks are extremely complex and difficult to construct and designing them effectively entails a lot of work that is arduous and error-prone.

A key insight from our academic team was to use AI itself in the form of traditional Machine Learning to assist humans in building and designing DNNs. In this way, an individual collaborates with AI to simplify and accelerate the development process; the AI does the arduous and monotonous ‘dirty work’ enabling an individual to focus on more creative and ‘human’ design tasks.

RM: What are the most exciting applications of AI that could be improved with DarwinAI’s platform?

SF: The most exciting applications improved by our platform are those that require AI at the ‘edge’: autonomous vehicles and aircraft, speech recognition on consumer devices, and mobile health care applications.In addition, the extent to which we reduce the size of neural networks enables use-cases that would be difficult if not impossible to facilitate without our technology. For example, deploying super-resolution networks that artificially enhance low resolution videos to HD onto devices such as TVs and phones.

Sheldon Fernandez was speaking to Ruairi J Mackenzie, Science Writer for Technology Networks