AI all-rounder: ethical, trustworthy, reproducible
Artificial intelligence (AI) and machine learning (ML) methods are tools that have long been indispensable in many scientific disciplines. They allow conclusions to be drawn from data volumes that are too large for scientists to process manually, uncover previously unknown correlations in confusing piles of data, help to solve complex computational processes and much more.
In the transfer of scientific results to concrete social applications, AI should ultimately be of benefit to the public. In the recent past, AI models have become particularly powerful and have thus attracted increased attention. However, with the increasing influence of AI in science and everyday life, there are also growing concerns about the lack of transparency that often exists in AI models and the possible bias of their results.
In order to exploit the immense potential of AI responsibly, expertise is therefore required at numerous levels. At our institute, solutions are developed in various groups and interlinked in interdisciplinary projects. The INM-7 is an “AI all-rounder”.
Reproducibility
The applied result of an AI-supported analysis is only the end product in a long chain of analysis steps. It is important that a result is completely comprehensible and reproducible. The entire analysis must be transparent and reproducible, starting with the data used to train an AI model, the parameters used in individual calculation steps and the key figures used to evaluate the quality of the model on test data sets. Otherwise, users run the risk of blindly trusting the erroneous results of a non-transparent process, for example due to methodological errors or distortions in the underlying database.
Several open source software applications developed at our institute simplify reproducibility. The data management solutions developed by the“Psychoinformatics” group [LINK] lay the foundation for reproducible data analysis. With data versioning, provenance recording, automatic recalculation if required, but also metadata catalogs for data retrieval or decentralized data transport, they represent flexible tools across application areas. Data collection tools, such as the JTrack ecosystem of the 'Biomarker Development' group, expand the data basis by means of smartphone-supported data collection. And the junifer library, developed by the “Research and Infrastructural Software Engineering for Machine Learning” team, simplifies feature extraction from commonly used imaging data types and helps with automation.
Trustworthiness
However, reproducibility alone is not enough - AI-supported research must also be trustworthy and robust. This means methods and results that are bias-free and consistently tested for generalizability. In its work, the “Cognitive and Clinical Neuroscience” group investigates the fairness of AI models, among other things, in order to protect applications from bias.The “Applied Machine Learning” group and the “Research and Infrastructural Software Engineering for Machine Learning“ team take a step further and work towards error-free method application. In julearn, they provide software that simplifies complex machine learning methods and prevents typical errors such as data leakage. This makes machine learning simpler and more trustworthy.
Ethical application
The ethical use of AI is particularly important in the medical field. The “Neuroethics and Ethics in AI” group works at the intersection of ethics, medicine and data science. Among other things, it works on identifying the ethical challenges of AI applications, ensuring that AI is used within a socially accepted and responsible framework, and evaluating the extent to which AI approximates human performance and how this can be tested.
Out of the ivory tower
We don't keep software and expertise to ourselves. All the tools we develop are open source and freely available. There are also virtual office hours to exchange ideas with users and interested parties. And we actively combine our tools and expertise in various projects, such as FRAIM or ABCD-J.
Further information and other projects can also be found in the cooperations overview.