Self Supervised Learning of Electron Microscopy Images

Self Supervised Learning of Electron Microscopy Images
Generative Adversarial Training. The generator produces realistic clean images conditioned on noisy inputs. The discriminator tries to separate real images from those generated by the generator.

This research explores self-supervised learning using Generative Adversarial Networks (GANs) to enhance the analysis of electron microscopy (EM) images. Instead of relying on manually labeled data, the method leverages vast collections of unlabeled EM images to pretrain neural networks that learn rich, transferable representations of microscopic structures. Using the Pix2Pix conditional GAN architecture (Isola et al., 2017), we train a generator–discriminator pair to reconstruct clean, realistic EM images from noisy inputs, capturing underlying structural patterns and textures characteristic of nanoscale materials.

After pretraining on large unlabeled datasets such as CEM500K (Conrad et al., 2021), the pretrained models are fine-tuned on task-specific and domain-specific datasets like HRTEM Au nanoparticles and TEMImageNet. This process substantially improves performance and accelerates convergence across various downstream tasks—including semantic segmentation, denoising, noise and background removal, and super-resolution. Notably, the pretrained models achieve better accuracy and stability even with simpler architectures (e.g., U-Net, HRNet) compared to larger models trained from scratch.

semantic segmentation benefits from GAN-based SSL pretraining
GAN-based SSL pretraining improves semantic segmentation. The figure shows the EM image of gold nanoparticles, ground truth segmentations, segmentation results with randomly initialized model, and segmentation results after finetuning a pretrained model, respectively from left to right. The finetuned model matches the ground truth better compared to the trained with randomly initialized weights.

Our results show that self-supervised GAN-based pretraining can significantly reduce the dependency on annotated data and complex network architectures. It provides a framework for automated EM image interpretation, enabling more robust, data-efficient, and generalizable analysis pipelines for diverse imaging conditions and materials. This research establishes self-supervised pretraining as a powerful foundation for next-generation deep learning workflows in microscopy and materials characterization.

Contact:

Dr.-Ing. Bashir Kazimi

Tel.: +49 241/927803-38
E-mail: b.kazimi@fz-juelich.de

Last Modified: 11.11.2025