Published on August 26, 2026

There’s a lot of talk right now about AI in software engineering. Almost every day, new tools, success stories, and predictions emerge about how fundamentally AI will change the way we develop software. At some point, we became less interested in the latest hype and more interested in a very practical question: What actually works in a real software project when you set out to make it work?

To find out, we at BAYOOMED launched our AI Lighthouse. We deliberately chose not to develop a contrived demo project, but rather to select a task for which we had a baseline for comparison.

We had already implemented a similar solution some time earlier as software in an FDA-regulated context. That solution consisted of an app and a backend; it involved a traditional project team of about six people with various roles in architecture, development, verification, quality, and product ownership. The project took nine months to complete.

We didn't have that much time. For our experiment, we changed the parameters. This time, we wanted to implement a slightly different technical scope exclusively as a cloud solution. Above all, however, we wanted to find out how far we could go if we set as few technological constraints as possible and consistently integrated AI into the development process.

The development team consisted of just two senior software engineers. Both had extensive experience in software engineering but no proven expertise in cloud technology or browser-focused user interface technologies, which were ultimately chosen for the implementation. That, too, was part of the experiment. One of our hypotheses was that, with powerful AI tools, it might no longer be years of experience with a specific technology that is decisive, but rather general seniority in software engineering.

When selecting technologies, we therefore gave the team a free hand to a large extent. They were allowed to experiment with programming languages, frameworks, AI assistants, and even tools capable of generating large portions of an application. The only requirement was that the solution had to run in the cloud.

At the same time, there were a few non-negotiable conditions: The tools used had to be suitable for commercial use and meet our requirements for data handling. In particular, our data could not be used for further training of the models.

We also deliberately scaled back the development process for this experiment. Unit tests were part of the scope, but comprehensive documentation and verification were not – at least initially. It was clear to us from the start that we were testing AI in software engineering under simplified conditions and that we could not apply the results directly to a regulated medical device.

If we were going to explore just how far AI could take us in software engineering, we didn't want to start with coding. We also wanted to include traditional product owner tasks as part of the experiment.

Our technical foundation was the software requirements specification from the original project, which we had adapted to the new scope. We also had two UI designs and an initial prototype created with Lovable. We consolidated this material into a ChatGPT project and used it to generate a backlog – including user stories, scope definitions, acceptance criteria, and links to the respective requirements. We then transferred the result to Azure DevOps via Excel.

That worked better than we had initially expected. Of course, the results had to be reviewed and refined in a few areas during the planning phase in collaboration with our colleagues on the development team. But it provided a solid foundation for launching the project.

The first user story was deliberately not a business feature, but focused on architecture and software design. After all, we didn’t just want to know how much code AI can generate. We were interested in whether two experienced software engineers, with AI support, could use a technology that was relatively new to them in such a way that, in the end, a compelling architecture and a usable product would still emerge.

We also changed our usual project rhythm. Instead of two- or three-week sprints, we worked in one-week sprints, each with planning and review on Monday. We did away with a formal daily stand-up. With a two-person team, direct communication was already a given; any open questions could be clarified at any time via a Teams chat.

The short sprints were a deliberate choice. We assumed that an experiment like this could only be planned to a limited extent in advance. We wanted to be able to learn quickly, reprioritize regularly, and adjust our own hypotheses.

It became clear as early as the first week that the backlog generated by ChatGPT was surprisingly useful. During planning, we had to clarify a few questions and refine some details, but the user stories were already so granular that the team was largely able to do without an additional task breakdown.

At the same time, we began searching for the right toolset. Claude and GitHub Copilot became essential tools in the development process. We also tried out Lovable, but ultimately ruled it out for our specific use case because it didn’t offer enough flexibility for subsequent customization.

The first review brought a certain sense of disillusionment. The hoped-for dramatic acceleration was not immediately apparent. The cloud setup proved to be more complex than expected, and in some cases, the AI initially led the team in the wrong direction. At the same time, the team had to figure out which tool was best suited for which task. And they rightly wanted to understand what the AI was producing, rather than simply adopting the generated code.

This experience already revealed a key insight that would later play an important role in our retrospective: AI in software engineering makes it much easier to get started with new technologies, but it does not render domain expertise obsolete. Especially when you know little about a topic yourself, it becomes harder to recognize when AI is wrong or taking an unnecessarily complicated approach.

After the first week, it was therefore by no means clear whether our hypothesis would hold true. However, we had already learned that AI-assisted development is not simply about generating as much code as possible.

And in the second week, the momentum of the project began to shift significantly. Find out more in the next part, coming soon...

• AI in Medical Software Development – How we support manufacturers in the development, validation, and MDR-compliant approval of AI-powered medical devices.
• Software Engineering for Medical Devices – Our range of services for MedTech, pharma, and life sciences: from consulting to market readiness.