Transformer models in biomedicine

Deep neural networks (DNN) are an artificial neural network that have fundamentally revolutionized the artificial intelligence (AI). Transformer models are a type of DNN that was originally used for natural language processing tasks but has gained more interest in processing sequential data including biological sequences and structured electronic health records. Transformer models have been developed to address various scientific questions in biomedicine. 

The article published in BMC Medical Informatics and Decision Making reviews the development and application of these transformer models for analysing biomedical-related datasets such as biomedical textual data, protein sequences, medical structured-longitudinal data, and biomedical images and graphs, some of which are of direct relevance to IDERHA. The review also touches upon AI strategies that help to comprehend the predictions of transformer-based models and dives into limitations, challenges, and novel research directions. 

The authors conclude that the ability of transformers to manage diverse biomedical data types makes them promising to address biomedical research questions. However, the need for large datasets limits their use in areas with less data, like wearable device signals or clinical studies. Looking ahead, advancements in linking and integrating healthcare data across various organizations, such as IDERHA is aiming to do, are expected to broaden the use of transformers in medicine, offering exciting possibilities for the future.

Source: BMC Medical Informatics and Decision Making , 29 July 2024