Will AI replace research software engineers?

No.

Consider this: a typical human software developer ends up with 15-50 bugs per 1000 lines of code (see e.g. this book and this blog referring to this book). I’m not sure this is transferable to researchers writing code, but for the sake of argument let’s assume it is. Now, your AI agent or chatbot can produce 1k lines in seconds and even assuming these things reduce the number of bugs by a factor of 10 (as they are definitely much better in avoiding the typical syntax errors, but as everything in the internet this is debatable) gives you around 1 bug per handful of seconds. And these bugs are typically the hard ones: hidden deep in the logic, won’t be found at compile time, industry-grade unit tests won’t find them, those bugs that make you either hammer your screen with your keyboard or doubt your ability to write proper code at all.

So, instead of spending 80% of your time writing code and 20% debugging your own code, you now spend 20% prompting and 80% fixing bugs in code you did didn’t even write. How does research software engineering help you here? Well, for that 20% prompting it is vital to know what you’re asking. Are you “just” asking the bot to write “the software” for you or are you asking it to start with a clean architecture and project structure, with unit and integration tests right away (you might even dare to try Test-Driven Development), and with proper documentation? And for that 80% of debugging a well designed test suite can help both the bot and yourself to get to the bottom of things more quickly.

Some people call it the Third Golden Age of Software Engineering thanks to AI and what they mean is this: After going from assembler to higher-level languages and from spaghetti code to object orientation the next level of abstraction now is AI-assisted coding. That’s what it is: a new level of abstraction, a new way of telling the computer what we want it to do. This does not change the fact that we need to know what we are doing, but it does change the “how”.

There is one competence for anyone writing software that will always be important: managing complexity. Not necessarily code complexity (only), but in particular complexity of the actual task at hand. Especially in our everyday research context, writing research software is much more than “coding” or applying standard software engineering techniques (if at all): With our software we explore the unknown. We implement new methods with unknown results. We write data analysis scripts that reveal insights into data that we could not get without this script. We help others in their scientific journey with our tools, not knowing what they will find. Writing software with this in mind raises complexity substantially and in particular to a point where AI agents just fail to grasp that complexity. In turn, this is precisely where a human research software engineer (RSE) excels: with domain knowledge, software engineering expertise, creativity, and quite a bit of tolerance for frustration, an RSE can turn a scientific idea into research software.

And you know what? This is where AI can help us. AI is here to stay, so let’s use it for the things that stop us from spending time with more fun things. Chatbots can reflect on a proper structure of a software, they can point us to tools to use, libraries to leverage, software engineering techniques to apply. AI coding agents can help us to get going, set the stage, take care of the tedious infrastructure tasks like testing frameworks, fix bugs, suggest refactoring targets, and so on. And we can think about cool new algorithms, applications, the next paper, the next project, our scientific future. Are you ready?

Disclaimer: If you are reading this as an FZJ employee, don’t take this as a suggestion to jump on the bandwagon without further considerations. Using AI for your work at FZJ requires proper training (see, e.g., here) and has legal implications. For now, we refer to the AI Intranet Portal for more information.

This text has been written without any AI support.

Figure caption: This is what you get when asking Perplexity AI to turn this text into a signature picture. And then massage it. And some more. And correct typos. And anatomical details like arms.

Written by: Robert Speck