AI integration step by step - Part 2: Implementation, model strategy and validation

In the first part of our article series on AI integration, we looked at how to thoroughly prepare AI projects: defining business goals, designing the architecture and the critical role of data quality, whether it's more traditional machine learning tasks or the use of the latest language models.

We now focus on implementation - that is, when AI integration goes from theory to working system. The designs become a tangible system: data pipelines are built, the AI model is born or selected, testing takes place and finally the intelligent component is integrated into the business processes. This is where theory becomes practice, and where a lack of careful planning can most often lead to a project stalling. It is important to see that the implementation steps may have different emphasis. It requires a different approach to develop a completely new, customised AI model than it does to integrate an existing, high-performance base model (especially large language models, LLMs) into existing systems. This article will guide you through the key steps, highlighting the specificities of both paths.

1. Building a data channel: the fuel for the AI system

Every artificial intelligence (AI) system relies on high-quality data. But it's not enough to just store the data: you need to make sure it gets to where the AI will use it in a structured, clean, correct form and version. This is particularly important when using AI in cases where fine-tuning or predictive analysis is required.

Designing the data path:

Data often has to come from several systems, be transformed and finally loaded into the AI system or a central data warehouse. ETL (Extract-Transform-Load) and ELT (Extract-Load-Transform) are proven methodologies for this - their core purpose is to ensure that data is routed and transformed correctly.

Highlighting the essence (Feature Engineering):

Especially when developing custom models, it is important to extract the most important information from the data, the "features" that the model needs to pay attention to during learning (e.g. the purchase frequency of a customer, the fluctuations in the sensor data of a machine). This helps the model to learn more efficiently.

LLMs and modern data management:

In the era of large language models (LLMs), new aspects are coming to the fore:

• Data for model training (fine-tuning): if an existing LLM is to be "fine-tuned" to the company's own language or specific tasks, it needs targeted, high-quality example data.

• Making corporate knowledge available (RAG and vector databases): For example, the creation of internal policies or product specifications is increasingly done using artificial intelligence, especially for LLMs. In order for an LLM not only to have general knowledge, but also to know and use internal company documents, product specifications and policies, special databases (so-called vector databases) and techniques (such as RAG - Retrieval-Augmented Generation) are needed. This allows AI to provide up-to-date and relevant answers in the enterprise context.

• Data Drift: The business environment and data change over time. It is important to monitor these changes ("data drift") because they can affect the accuracy of the AI model and may require updating.

2. Model strategy: create a new one, adapt it or just use it? 

Not every AI task requires building a model from scratch. Choosing the right strategy is key to success and efficiency.

Custom model development: 

• When is it needed? When the task is very specific, based on unique data, or when off-the-shelf models available on the market do not provide sufficient accuracy or customisation (e.g. specialised industrial image processing, complex forecasting systems).

• What does this mean in practice? Choosing the right type of model (e.g. for image, text, figures), "fine-tuning" the model (finding optimal settings for best performance) and tracking the model variations that are created during development.

Application of existing basic models (LLMs):

Today, this is increasingly common, especially for tasks such as text comprehension, content generation or customer service.

• Choosing the right tool: there are a number of high-performance LLMs available (e.g. GPT or Claude models, or open source alternatives). The choice depends on the task, the budget and privacy considerations.

• Use as a ready-made service (API integration): In many cases, LLM can be accessed as a service (via an API). The challenge here is to establish a secure and efficient connection, monitor and optimise the cost of use.

• Fine-tuning: It is possible to 'fine-tune' a general-purpose LLM to a company's specific terminology, communication style or a more narrowly defined task. This can be faster and more cost-effective than building from scratch.

• Prompt Engineering: the art of managing LLMs. Learning how to formulate precise, clear "requests" (prompts) to the AI to get the desired result. This is key to reliable operation.

Version tracking:

Whether you are developing your own model or customising an existing one, it is important to keep track of the different versions to know how each one performs.

3. Integration: AI turn on into the corporate machinery

The completed or selected AI component must be connected to existing business applications, databases and processes. The way this is done will greatly affect the speed, reliability and future extensibility of the system. 

Making the connections:

• Secure communication (API Gateway): acts as a "gatekeeper" to ensure that only authorised systems can communicate with the MI component and controls access.

• Flexible deployment (containerisation - Docker, Kubernetes): Technologies that allow the AI application (and related services) to be easily deployed, updated and scaled - i.e. to adapt flexibly to load growth.

• High load management (message queues - Kafka, RabbitMQ): when many tasks arrive at once (e.g. processing large amounts of data), these systems help to queue and distribute tasks efficiently, preventing system overload.

• Speed up and reduce costs (Caching): Caches the results of frequently repeated requests, allowing the system to respond faster and reducing the cost of using LLM services, for example.

Testing and validation: does it really do what we expect?

The use of artificial intelligence does not exclude traditional software testing. In fact, testing systems is a specific challenge that needs to be complemented by AI-specific validation steps. In the case of predictive models, it is particularly important to pay attention to the behaviour in unexpected situations, the possibility of prompt injection attacks, and possible inaccuracies in the report generation. 

Traditional tests

Basic software tests are needed (do the parts work, are they well connected, can they withstand the load, are they secure).

MI specific checks

Accuracy and reliability:

Does the model really give the right results (measured by statistical indicators such as F1-score, MAE, RMSE). 

Specific challenges of LLM: 

• Truthfulness: it is necessary to check that the LLM is not "hallucinating", i.e. making untruthful statements.

• Bias: Check that responses do not reflect unwanted biases or stereotypes.

• Safety: to filter out potentially harmful, offensive or dangerous content.

• Manipulability (Prompt Injection): Test whether malicious "prompts" can be used to induce the AI to perform unwanted actions.

• Edge Cases: What happens if the system receives incomplete, incorrect or unusual data?

Measure and monitor: track results and operations 

The implementation of AI does not end with the deployment of the model, it is only successful if it delivers measurable business benefits. Therefore, we need to define what we expect from it in advance:

• Faster customer service?
• Less manual work?
• Better customer experience?
• More accurate forecasts?

And during live operations, the performance and costs of the system need to be constantly monitored. This includes detecting any deterioration in accuracy, tracking changes in data and, in particular, keeping the user charges for LLM services under control. Proper logging and reporting ensures transparency and the prevention of potential errors. 

Summary: From an idea to a working system

The implementation phase of AI integration is technically challenging, but it can be done successfully with proper planning, the right model strategy and thorough testing. The goal is to create a robust, transparent system that contributes measurably to the company's business goals, whether it is a custom-built model or a modern LLM integration. 

Next: sharpening, operation and secrets of the future-proof AI system

In the third and final part of this series, we will examine:

• How is the AI solution deployed and maintained in a live operational environment?
• What typical operational challenges might arise (e.g. interoperability with legacy systems, scaling, cost-effectiveness)?
• How to ensure that the AI system remains efficient and relevant in the long term?

If you want a truly working, value-creating AI solution - and not just a pilot project - stay tuned for the next part!

[banner type="encoai" text="Want to take the first step in implementing AI?" button="Apply for AI Brunch!" link="https://encoai.com/"]