USC Viterbi professor and undergrad team up to make neural networks faster, cheaper and more powerful.
In the broad field of electrical and computing engineering, collaboration among researchers is paramount to progress—and opportunities for cooperation can emerge where least expected. That’s what happened when University of Southern California Viterbi School of Engineering Professor Mercedeh Khajavikhan teamed up with undergraduate student Haoqin Deng to devise a way to speed up neural networks and make them cheaper.
Funded by the U.S. Office of Naval Research and the Air Force Office of Scientific Research, among other bodies, the collaboration showcases what’s possible when people from different backgrounds come together to solve problems. It also highlights the value of creating opportunities for students to work alongside more seasoned academics to help guide their careers and develop their research and professional skills.
“This collaboration speaks to the value of bringing in young researchers,” Khajavikhan said in a statement. “USC Viterbi has a great culture of encouraging research among our younger students, and Haoqin’s work has really changed my perspective.”
Khajavikhan’s work had historically focused on improving communications, sensing and signal processing hardware systems using advanced photonics. Although neural networks had not previously been an area of her research, Khajavikhan has recently been using optical photonics chips to provide neural networks with a boost in speed.
A limitation of current electronics systems is that they adopt the Von-Neumann architecture, which is hindered by the data transfer rate between the processing and memory units. Conversely, optical computing requires lower energy per bit and has less latency, making it prime for optical neural networks on silicon chips (originally demonstrated by a team at MIT). Offering higher energy efficiency and computational speed than electronics, optical neural networks can replace conventional deep learning hardware.
With Deng’s experience with neural networks, the two began training the AI with the improved optical chip. In general, powerful chips are needed to train networks, with better chips leading to better training. While traditional chips have become more robust, optical photonics chips are promising because they can perform matrix-vector products more efficiently than electronic circuits. However, standard optical integrated chips are not quite up to the task. Khajavikhan’s work on an optical photonic chip is vital as the chip emits signals using a laser’s optical frequencies—which are much faster than traditional technologies. Just as fiber optics are faster than wired communications, optical photonics chips allow more rapid speeds than standard chips.
“A neural network must be trained,” said Deng. “Just like a baby learning to distinguish between objects, we have to teach a new neural network how to recognize the world around it.”
The crux of Khajavikhan and Deng’s work lies in creating a new approach to neural network training that involves active photonic platforms. The duo is using properties of III-V semiconductor compounds to produce a system that is easily tunable and trainable compared to using a passive chip with large, and energy inefficient, phase shifters for tuning.
Their new architecture is based on parity-time-symmetric couplers that improve current optical neural networks because of their use of optical gain-loss III-V semiconductors. The resulting parity-time-symmetric optical neural network ensures that the system is able to express patterns—a crucial aspect of AI. The use of parity-time-symmetric couplers also reduces energy consumption and increases training speed.
They found that the use of parity-time symmetric coupler, as a foundational component for training networks, yields increased speed and efficiency with less expense. Moreover, this approach circumvents the main obstacle of using optical neural networks, which is that they are cumbersome. Instead, Khajavikhan and Deng’s technique is ten times smaller.
What is remarkable is that the collaboration is geared to result in a completely new—and better—neural network, for which Khajavikhan gives credit to Deng for helping expand her research.
