By Dr. Karl Häfner and Dr. Simon Birkholz
Imagine if there was an AI that could provide you with customized support for your daily, even complex work processes - as easy to use as ChatGPT.
For most technology enthusiasts, generative artificial intelligence has long since become part of their everyday lives. Users are increasingly having texts and images for their daily work generated by ChatGPT and Midjourney for their daily work. Advanced users even use AI-based tools to develop software or analyze their business figures. Increasingly, these tools are designed to integrate seamlessly into daily workflows. This saves time and sometimes reduces work on rather unpopular activities.
However, there are usually no readily available AI services for more complex tasks. Find out what really matters when developing customized AI for companies in the first part of our "AI Deep Dive" series.
AI as specialized assistants for individual requirements
For some months now, our customers have therefore been increasingly approaching us to use AI as specialized assistants for their individual requirements. For example, chatbots are being created that provide answers based on extensive technical manuals. In other applications, AI provides almost unobtrusive support by making recommendations in what was previously purely manual product documentation.
Particularly important: the AI's response must always be comprehensible and verifiable for the user.
When the user interacts with our AI implementations via the chat function, they not only receive an answer. The AI also links to the documents in which the information used for the answer was found.
Retrieval Augmented Generation (RAG): The key to customized AI applications
But how is an AI application based on your own data created? This is where the so-called Retrieval Augmented Generation-pattern has been established (RAG).
In the illustration, we have mapped the workflow of an AI application that is based on the user's own data. First, the company's own data (Your Data) is divided into so-called chunks and stored in a vector database. If a user submits a query to the AI application, a search query (search query) is sent to the vector database. The relevant entries in the database are then forwarded to a GPT model together with the user query. The GPT model then uses the relevant entries from the database to answer the user query.
IllustrationWorkflow of an AI application based on own data
Often underestimated: a comprehensive database is the be-all and end-all for AI applications
The creation of a comprehensive database is important for the generative AI technologies described here. Unfortunately, this step is often underestimated for reasons of time and cost. However, the large number of texts that companies have collected in their manuals, wikis and documentation are invaluable for generative AI. Only when these texts, which often contain a large part of the company's knowledge, are made accessible to AI can their potential actually be exploited.
Large quantities of texts are unstructured data whose usefulness is not always easy to understand. Just because I have a book on the shelf doesn't mean I know what's in it and how best to use the knowledge it contains. Even a sophisticated system that ensures that the book is in the perfect place on the shelf doesn't help me.
AI systems are no different to humans in this respect. Just as a human must first read a book, take notes if necessary and relate what they have learned to what they already know, an AI must first break down a text into digestible sections, add metadata and further context. Only then can humans and AI actually do something with the text.
For AI technologies, there is the additional challenge that typical documentation texts contain not only text, but also all kinds of illustrations, tables, headings, notes, footnotes and the like. People find it easy to relate all of these to each other. However, even the best current multimodal AI models reach their limits and this challenge will remain for the foreseeable future.
Shit in, shit out: Successful AI applications require well-prepared data
The solution to these problems lies in the pre-processing of documents.
Pipelines must be implemented that obtain the documents from their original sources, transform them, relate their elements to each other, enrich them with metadata and finally store them in a new form in a location that the AI can then access (typically a search index or a vector database).
This pre-processing has a direct influence on the quality of the actual AI application: the more specifically the pipeline is adapted to the respective documents, the higher the quality and reliability. In our experience, no other adjusting screw during the development of an AI application has a comparable influence on the end result.
That's why you have to expect that 70-80% of the effort during the implementation of an AI application (without testing and evaluation) is spent on the pre-processing pipelines.
Testing for practice: ensuring the reliability and suitability of AI solutions for everyday use
After successful data preparation and implementation of the AI application, there is another important step. The AI application must function reliably for use in day-to-day business. In particular, false statements, also known as hallucinations, must be ruled out as far as possible.
Careful preparation of the data creates an important basis for reliable application.
In any case, the application must be carefully tested before going live. Systematic testing of a solution is widespread in all areas of IT. It is therefore not surprising that proven test procedures are also needed for AI applications. However, the non-deterministic behavior of AI requires procedures that differ from the test methods used in traditional IT.
Our projects are based on three test methods.
- Firstly, the prototype is always tested by domain experts. In a test period of a few weeks, during which our customers' employees use the prototype in their daily work, it can be ensured that the application works in practice. The testers provide feedback on the quality of the responses generated, which is used by the development team for bug fixes and fine-tuning.
- Secondly, the prototype is tested using a set of ready-made questions. This covers rare cases in practice that are specific to the respective field of application.
- Thirdly, an even larger set of generic questions ensures the general performance, security and harmlessness of the answers.
Conclusion: More courage for customized AI applications
Even though we have mainly talked about the challenges in this article, we would like to encourage you to develop your own customized AI applications. Because let's be honest: if you want to stay ahead of the competition, off-the-shelf AI solutions won't help you.
Our team will be happy to assist you in designing AI tools for your individual application and, in particular, help you with the all-important data preparation.
Fancy taking the first step towards customized AI?
As a simple first step towards a customized solution, we offer an AI workshopin which we work with you to develop application scenarios and practical solutions.
You can find more information about our "AI Kick-Start" workshop here.
About the authors:
Dr. Karl Häfner studied economic geography and has been working as a data scientist in the field of natural language processing and AI since 2019. He has been supporting companies in integrating generative AI and large language models into their business processes since 2021. At the Dataciders subsidiary ixto GmbH manages he the AI Solutions team.
Dr. Simon Birkholz studied physics and started working as a data scientist in 2016. He has worked as a consultant for Mercedes-Benz, Siemens, Vattenfall and Bayer, among others. Today he is Managing Director of the Dataciders subsidiary ixto GmbH with a focus on data science and AI.
