AI PHCA
AI-Based Preventive Health Care for Children and Adolescents.
AI and Ethics
Copyright: Created using AI ChatGPT
What ethical principles are appropriate for guiding the development and use of AI applications? What regulations should be put in place to ensure the responsible use of AI applications? And what fundamental philosophical and epistemological issues does the development and use of AI raise? In addition to conducting excellent research on and with AI, scientists at Forschungszentrum Jülich are also dedicated to addressing such ethical questions.
AI is used in medicine to support diagnoses and treatment decisions. But what if an AI-based diagnostic system fails to detect a dangerous brain tumor or makes a fatal treatment decision? These are real problems that need to be addressed.
How do AI and ethics fit together?
The ethics of artificial intelligence is a subfield of applied ethics that addresses the ethical issues surrounding AI systems. The role of AI systems in society and the ethical values underlying their use are particularly relevant topics at the FZJ.
Why are ethical considerations relevant in the development and use of AI?
AI offers great potential, for example, through faster and more precise analysis of complex data, the identification of patterns that are difficult for humans to detect, support for clinical and scientific decision-making processes, and the acceleration of scientific and medical research.
However, the topic is also particularly relevant from a societal perspective because AI applications can have direct impacts on their users or the subjects of their use. Medical AI applications, for example, can have consequences for patients and medical staff, and thus also touch on fundamental human values such as trust, responsibility, and autonomy. At the same time, some of the data used for AI training is particularly sensitive, especially when it comes to medical or other personally identifiable information. In the development and application of AI, ethical issues must therefore be considered at an early stage to ensure the responsible use of new technologies.
What topics are the focus?
AI can exacerbate existing social inequalities if it is used uncritically. At the same time, it can contribute to more equitable and improved care if fairness, transparency, and accountability are systematically taken into account. From an ethical perspective, therefore, several fundamental principles of AI and their societal implications are examined:
- Since AI systems depend on the data used to train them, biased or non-representative data can result in certain groups being underrepresented or disadvantaged.
- Because it is often difficult to understand how many AI systems arrive at their results, informed decisions depend largely on the transparency of the models used.
- When applying AI models, it must be clarified who bears responsibility if AI-based recommendations are flawed and potentially cause harm—it must be ensured that AI supports human expertise but does not obscure or replace accountability.
- When sensitive data is incorporated into AI models, data protection and autonomy must be safeguarded, and patients and study participants must be adequately protected and informed.
Where is AI ethics applied?
AI ethics is becoming increasingly relevant in the medical field, in part because AI is being widely adopted in medicine and the neurosciences. For example, AI is used to analyze large and complex datasets, support diagnostic and predictive procedures, model brain, behavioral, and health data, and develop personalized therapeutic and research approaches.
Beyond the sensitive fields of medicine and science, AI ethics can also play a pivotal role for businesses. This is because ethical issues can arise whenever companies implement AI—whether in internal processes, external communications, or when products and services are to be enhanced with AI capabilities. An ethical perspective ensures that issues such as transparency, traceability, and accountability structures are given due consideration during implementation and also takes into account the needs of all relevant stakeholder groups, such as employees and customers. In this way, it is possible to create a final product that is not only functional and cost-effective but also values-based and needs-oriented.
AI ethics at the Forschungszentrum Jülich
At the Forschungszentrum Jülich, AI ethics is closely integrated into various fields at the intersection of medicine and Law, among other areas. For example, an interdisciplinary research project is assessing the current state of research on the use of AI in pediatric and adolescent medicine, surveying those affected to gather their perspectives, and identifying ethical and legal issues—in addition to results published in scientific journals, this work also yields position statements, such as those issued by the Commission on Ethical Issues within the Alliance for Child and Adolescent Health (Bündnis Kinder- und Jugendgesundheit e.V.).
Researchers at Jülich are also involved in developing ethical recommendations for the use of AI in forensic medicine. The focus is on the question of under what conditions AI systems can support experts in the evaluation of findings or imaging data. Since forensic medical assessments can have far-reaching consequences for the individuals involved, reliability, transparency, and clear lines of responsibility are particularly important. The goal is to use AI in a way that complements human expertise without replacing professional judgment.
Furthermore, AI ethics is also a direct subject of research within different fields, for example neuroscience. Modern neuroscience, for example, uses machine learning and MRI scans of the brain to determine a brain’s biological age and thus potentially detect early signs of disease. While predicting brain age holds great potential for medical diagnosis and prognosis, it also raises significant ethical questions. In addition, researchers at Jülich are working on the philosophical analysis of so-called “hallucinations” in scientific AI models.
Systems such as AlphaFold can predict protein structures with impressive accuracy and have greatly advanced miomedical research. Nevertheless, even such models can produce erroneous or misleading results that are not always easy to detect. Researchers at FZJ are evaluating the conditions under which the results of scientific AI are trustworthy and developing conceptual and methodological foundations to better assess the reliability of such models and promote a critical, responsible use of AI in research.
For companies and public-sector institutions, Forschungszentrum Jülich also offers AI ethics consulting, which is integrated into its general AI consulting services. During the initial general consultation, new ideas are first assessed for their suitability and then developed into specific AI use cases. An ethical pre-check determines whether a project is likely to be uncontroversial from an ethical standpoint or should be examined more closely. Suitable projects can then be implemented jointly as part of a research collaboration.
If necessary, an in-depth ethical assessment examines the context of use, the affected groups of people and their specific protection needs, the data used, and the division of roles between humans and AI. This also includes the question of which decisions should remain in human hands and how effective human oversight can be ensured. In this way, embedded AI ethics allows for the early identification of potential risks and the incorporation of appropriate safeguards as early as the development phase.
Last Modified: 18.09.2026