134 — Content-Aware Image Restoration: Pushing the Limits of Fluorescence Microscopy

One common imaging modality is fluorescence microscopy, a technique that relies on the transmission of visible light to record the structure of function of a biological tissue. A pitfall of this method is the sensitivity of tissue to significant photon exposure; this technique requires a slight heating of the subject, and suffers from low spatial and temporal resolution.

This very exciting paper demonstrates that you don’t need to shower the sample with photons; with a tiny fraction of the amount of light, this process, which uses machine-learning to super-sample the results, results in images that surpass even the ideal best-case limitations of the microscope hardware.

The team does this by training a computer vision neural network on a large amount of higher-resolution data of the same or a similar sample. This is used by the Content Aware REestoration (CARE) network to increase the SNR of lower-fidelity data. Conventional methods use basic denoising that ignores the structure of the data; it is hoped that the CARE network takes advantage of these biological priors in order to enrich the dataset without sacrificing biofidelity.

I highly recommend readng this paper at least for its figures. The results are pretty astonishing, and appear to refine some details that would otherwise be lost for analysis. I do wonder how much of this is fully hallucinated rather than an actual function of the data; it appears to my eyes to be producing true, biofidelic results, and on artificially subsampled datasets, the algorithm performed very well.

I’m looking forward to trying this code out, since it’s on its way to being available for public use.