Docket #: S18-093B
Systems and Methods for Activation Functions for Photonic Neural Networks
Stanford researchers have developed an electro-optic hardware platform for nonlinear activation functions in optical neural networks. By converting a small portion of the input optical signal into an analog electric signal, linear optical neural networks can be made to be non-linear, enabling the network to have greater capability to model complex functions (e.g. machine learning tasks) without loss of processing speed.
This scheme allows for complete nonlinear on-off contrast in transmission at relatively low optical power thresholds and eliminates the requirement of having additional optical sources between each layer of the network. Moreover, the activation function is reconfigurable via electrical bias, allowing it to be programmed or trained to synthesize a variety of nonlinear responses.
Stage of Development
-
- Learning a multi-input exclusive-OR (XOR) logic function and classification of images of handwritten numbers from the MNIST dataset
- The addition of the nonlinear activation function improves test accuracy on the MNIST task from 85% to 94%
Applications
- Artificial Neural Network (ANN) training - optimize photonic hardware platforms for artificial intelligence and machine learning with end-user applications such as speech and image recognition
Advantages
- Converts linear optical neural networks into non-linear optical neural networks
- Non-linear optical neural networks simulate how neurons in the brain respond and can support more powerful, complex machine learning tasks
- Reconfigurable via electrical bias, allowing it to be programmed or trained to synthesize a variety of nonlinear responses
- Operates at relatively low optical power thresholds
- Less components - eliminates the requirement of having additional optical sources between each of the layers of the network
- Can be readily applied to other photonic platforms
Publications
- Williamson, I.A., Hughes, T.W., Minkov, M., Bartlett, B., Pai, S. and Fan, S., 2019. Reprogrammable electro-optic nonlinear activation functions for optical neural networks. IEEE Journal of Selected Topics in Quantum Electronics, 26(1), pp.1-12.
- M.Fard, I. Williason, et al Experimental realization of arbitrary activation functions for optical neural networks Optics Express 8 Apr 2020.
Related Links
Patents
- Published Application: 20210116781
- Issued: 11,460,753 (USA)
Similar Technologies
-
Fully-automated design of grating couplers (software) S18-019Fully-automated design of grating couplers (software)
-
Efficient, Scalable Training of Artificial Neural Networks Directly on Optical Devices S18-093Efficient, Scalable Training of Artificial Neural Networks Directly on Optical Devices
-
Quantum electro-optic converter S16-292Quantum electro-optic converter