Up-to-date AI model creates virtual colors of cancer tissue to improve diagnosis

In a recent study published in Nature Machine Intelligence introduces an advanced artificial intelligence (AI) model capable of creating virtual colors of tumor tissue. The study, co-led by scientists from the Universities of Lausanne and Bern, is an crucial step forward in improving pathological analysis and cancer diagnostics.

By combining inventive computational techniques, a team of computer scientists, biologists and clinicians led by Marianna Rapsomaniki from the University of Lausanne and Marianna Kruithof-de Julio from the University of Bern has developed a novel approach to the analysis of cancer tissue. Driven by the motivation to overcome missing experimental data, a challenge that researchers often face when working with restricted numbers of patient tissues, the scientists created the “VirtualMultiplexer”: an artificial intelligence (AI) model that generates virtual images of diagnostic tissue colors.

Virtual Staining: A Up-to-date Area of ​​Cancer Research

Using generative AI, the tool creates right and detailed images of cancer tissue that mimic how it would stain for a given cell marker. Such specific stains can provide crucial information about the status of a patient’s cancer and play an crucial role in diagnosis. “The idea is that you only need one real stain of the tissue, which is done in the lab as part of routine pathology, to then simulate which cells in that tissue would stain positively for several other, more specific markers,” explains Marianna Rapsomaniki, a computer scientist and AI expert at the Biomedical Data Science Center of the University of Lausanne and the University Hospital of Lausanne and a corresponding author of the study.

The technology reduces the need for resource-intensive laboratory analyses and is intended to complement information obtained from experiments. “Our model can be very helpful when available tissue material is restricted or when experimental staining cannot be performed for other reasons,” adds Pushpak Pati, the study’s first author.

Understanding the Method: Contrasting Odd Translation

To understand the basic methodology called contrastive odd translationimagine a mobile phone app that predicts what a teenage person will look like when they are older. Based on a current photo, the app creates a virtual image that simulates the person’s future appearance. It does this by processing information from thousands of photos of other, unrelated, older people. As the algorithm learns “what an elderly person looks like”you can apply this transformation to any photo.

Similarly, VirtualMultiplexer transforms a single stain image that broadly distinguishes different areas in cancer tissue into images that show which cells in that tissue stain positively for a given marker molecule. This is achieved by training the AI ​​model on numerous images of other tissues in which those stains were experimentally made. Once it learns the logic that defines the actual stain image, VirtualMultiplexer is able to apply the same style to a given tissue image and generate a virtual version of the desired stain.

Preventing Hallucinations: Ensuring Effectiveness and Clinical Significance

The researchers used a strict validation process to ensure that the virtual images had clinical significance and were not merely AI-generated results that seemed plausible but were in fact false inventions, known as hallucinations. They tested how well the artificial images predicted clinical outcomes, such as patient survival or disease progression, against existing data from real stained tissues. The comparison confirmed that the virtual stains were not only right but also clinically useful, showing that the model is reliable and trustworthy.

The researchers then ran the VirtualMultiplexer through the so-called Turing test. Named after the founding father of state-of-the-art AI, Alan Turing, the test determines whether AI can produce output that is indistinguishable from that produced by humans. By asking expert pathologists to distinguish time-honored tinted images from AI-generated colors, the authors found that the artificial creations were perceived as nearly identical to the real images, demonstrating the effectiveness of their model.

Multi-Scale Approach: Significant Progress

One of the main breakthroughs that distinguish VirtualMultiplexer is its multiscale approach. Customary models often focus on examining tissue at the microscopic scale (cellular level) or macroscopic scale (whole tissue). The model proposed by the team from Lausanne and Bern takes into account three different scales of tumor tissue structure: its global appearance and architecture, the relationships between neighboring cells, and the detailed features of individual cells. This holistic approach allows for a more right representation of the tissue image.

Implications for cancer research and beyond

The study represents a significant advance in oncology research, complementing existing experimental data. By generating high-quality simulated stainings, VirtualMultiplexer can lend a hand experts formulate hypotheses, prioritize experiments, and deepen their understanding of cancer biology. Marianna Kruithof-de Julio, head of the Urology Research Laboratory at the University of Bern and co-author of the study, sees significant potential for future applications: “We developed our tool using tissue from people with prostate cancer. In the paper, we also show that it works similarly well in pancreatic tumors – which gives us confidence that it could be useful for many other types of disease.”

The inventive approach also has the potential to support so-called fundamental AI models in biological research. The strength of such models is their ability to learn by processing huge amounts of data in a self-supervised manner, allowing them to understand the logic behind elaborate structures and acquire the ability to perform different types of tasks.

Available data on uncommon tissues are insufficient. VirtualMultiplexer can fill these gaps by generating realistic images quickly and free of charge, and thus lend a hand future foundation models analyze and describe tissue features in many different ways. This will pave the way for up-to-date discoveries in research and diagnostics.”

Marianna Rapsomaniki, University of Lausanne