Convolutional neural networks that teach microscopes how to image

Deep learning algorithms offer a powerful means to automatically analyze the
content of medical images. However, many biological samples of interest are
primarily transparent to visible light and contain features that are difficult
to resolve with a standard optical microscope. Here, we use a convolutional
neural network (CNN) not only to classify images, but also to optimize the
physical layout of the imaging device itself. We increase the classification
accuracy of a microscope's recorded images by merging an optical model of image
formation into the pipeline of a CNN. The resulting network simultaneously
determines an ideal illumination arrangement to highlight important sample
features during image acquisition, along with a set of convolutional weights to
classify the detected images post-capture. We demonstrate our joint
optimization technique with an experimental microscope configuration that
automatically identifies malaria-infected cells with 5-10% higher accuracy than
standard and alternative microscope lighting designs.