© (Copyright), International Software Architecture Qualification Board e. V. (iSAQB® e. V.) 2026

The curriculum may only be used subject to the following conditions:

  1. You wish to obtain the CPSA Certified Professional for Software Architecture Foundation Level® certificate or the CPSA Certified Professional for Software Architecture Advanced Level® certificate. For the purpose of obtaining the certificate, it shall be permitted to use these text documents and/or curricula by creating working copies for your own computer. If any other use of documents and/or curricula is intended, for instance for their dissemination to third parties, for advertising etc., please write to info@isaqb.org to enquire whether this is permitted. A separate license agreement would then have to be entered into.

  2. If you are a trainer or training provider, it shall be possible for you to use the documents and/or curricula once you have obtained a usage license. Please address any enquiries to info@isaqb.org. License agreements with comprehensive provisions for all aspects exist.

  3. If you fall neither into category 1 nor category 2, but would like to use these documents and/or curricula nonetheless, please also contact the iSAQB e. V. by writing to info@isaqb.org. You will then be informed about the possibility of acquiring relevant licenses through existing license agreements, allowing you to obtain your desired usage authorizations.

Important Notice

We stress that, as a matter of principle, this curriculum is protected by copyright. The International Software Architecture Qualification Board e. V. (iSAQB® e. V.) has exclusive entitlement to these copyrights.

The abbreviation "e. V." is part of the iSAQB’s official name and stands for "eingetragener Verein" (registered association), which describes its status as a legal entity according to German law. For the purpose of simplicity, iSAQB e. V. shall hereafter be referred to as iSAQB without the use of said abbreviation.

Learning Goals Overview

Introduction: General information about the iSAQB Advanced Level

What is taught in an Advanced Level module?

The module can be attended independently of a CPSA-F certification.

  • The iSAQB Advanced Level offers modular training in three areas of competence with flexible training paths. It takes into account individual inclinations and specializations.

  • The certification takes the form of a term paper. The assessment and oral examination are carried out by experts appointed by the iSAQB.

What are Advanced Level (CPSA-A) graduates qualified for?

CPSA-A graduates can:

  • design medium-sized to large IT systems independently and methodically

  • assume technical and content-related responsibility in IT systems of medium to high criticality

  • Conceptualize, design and document measures to achieve quality requirements and support development teams in the implementation of these measures

  • Manage and carry out architecture-related communication in medium to large development teams

Requirements for CPSA-A certification

  • Successful training and certification as a Certified Professional for Software Architecture, Foundation Level® (CPSA-F)

  • At least three years of full-time professional experience in the IT sector; collaboration on the design and development of at least two different IT systems

    • Exceptions are allowed on application (e.g., collaboration on open source projects)

  • Training and further education within the scope of iSAQB Advanced Level training courses with a minimum of 70 credit points from at least three different areas of competence

  • Successful completion of the CPSA-A certification exam



Certified Professional for Software Architecture<sup>®</sup> Advanced Level (CPSA-A)

Essentials

What does the module “SWARC4AI” convey?

The module presents participants with modern software architecture concepts for AI systems as a means of designing powerful, scalable and integrable AI solutions. At the end of the module, participants will be familiar with the key principles of software architecture for AI systems and will be able to apply them to the design and implementation of machine learning and generative AI systems. With the help of the modeling techniques and architecture tools taught, they will be able to seamlessly integrate AI components into existing software systems. The course covers both machine learning systems and generative AI and teaches how these can be combined with classic software systems. Participants learn what the architecture for such hybrid systems must look like in order to ensure scalability, maintainability and extensibility.

Contents Recommended minimum duration (min)

Introduction to concepts relevant to software architecture for artificial intelligence

120

Compliance, security, alignment

120

Design and development of AI systems

320

Data management and data processing for AI systems

90

Important quality features for the operation of AI systems

160

System architectures and platforms for generative AI systems

160

Case studies and practical projects

110

Total

1080

Duration, didactics and further details

The duration of a training course on the SWARC4AI module should be at least 3 days, but can be longer. Providers may differ in terms of duration, didactics, type and structure of the exercises and the detailed course structure. In particular, syllabus leaves the type of examples and exercises completely open. Licensed training courses on SWARC4AI contribute to admission to the final Advanced Level certification examination (following credit points):

Methodical Competence:

10 Points

Technical Competence:

20 Points

Communicative Competence:

0 Points

Prerequisites

Participants should have the following knowledge and/or experience:

  • A basic understanding of artificial intelligence, machine learning and data science

  • A basic understanding of software architecture, DevOps and the design of software systems and APIs

  • A basic knowledge of the Python programming language and its use for AI problems

  • Overview of common libraries such as scikit-learn, TensorFlow and PyTorch

Knowledge in the following areas can be helpful for understanding some concepts

  • Experience with the command line interface on Linux systems

  • Knowledge from the iSAQB CPSA Foundation level training for a general understanding of software architecture, design patterns and methodologies

Structure of the curriculum

The individual sections of the curriculum are described according to the following structure:

  • Terms/concepts: Essential core terms of this topic.

  • Teaching/practice time: Defines the minimum amount of teaching and practice time that must be spent on this topic or its practice in an accredited training course.

  • Learning goals: Describes the content to be conveyed including its core terms and principles.

This section therefore also outlines the skills to be acquired in corresponding training courses.

Supplementary Information, Terms, Translations

To the extent necessary for understanding the curriculum, we have added definitions of technical terms to the iSAQB glossary and complemented them by adding referenes to the literature.

1. Introduction to software architecture-relevant concepts for artificial intelligence

Duration: 120 min

Practice time: 0 min

1.1. Terms and Concepts

AI, Generative AI, Machine Learning, Symbolic AI, Evolutionary Algorithms, Statistical Learning, Deep Learning, LLMs, Hallucination, Bias, Jagged Technological Frontier, Data Preparation, Model Training, Model Evaluation, Feature Engineering, Data Science, Data Engineering

1.2. Learning Goals

LG 1-1: Classify artificial intelligence and machine learning, data science, deep learning, and generative AI

Participants can explain the concepts of Artificial Intelligence, Machine Learning, Data Science, Deep Learning, and Generative AI, distinguish between them, and describe how they relate to one another.

LG 1-2: Identify, classify, prioritize, and evaluate AI use cases from a business perspective

Participants can identify and describe typical AI use cases from a business perspective. They can evaluate their suitability based on business goals, data availability and quality, technical feasibility, expected benefits, risks, integration effort, and impact on the software architecture, and can establish a well-founded prioritization.

Typical use cases include image recognition and image generation, natural language processing, predictive models, personalization, anomaly detection, chatbots, recommender systems, and intelligent assistance systems.

LG 1-3: Classify the strengths, limitations, and typical risks of AI

Participants can classify the strengths and limitations of AI technologies in different application contexts and explain why the suitability of AI depends heavily on the task at hand.

They can identify typical technical and societal risks, such as

  • hallucinations,

  • bias,

  • unfairness,

  • deepfakes,

  • AI-assisted cyberattacks,

  • safety risks in critical systems,

  • social manipulation,

  • infringements of intellectual property rights.

Participants are familiar with the “Jagged Technological Frontier” in relation to the task-specific suitability of AI.

LG 1-4: Understand and explain the differences between AI systems and traditional software and decide on the appropriate type of solution

Participants understand the differences between AI systems and traditional software:

  • Data-driven (for ML) – data-centric rather than code-centric development

  • Probabilistic results (non-deterministic behavior)

  • Statistical validation

  • Experimental design: support for rapid iteration and model testing

  • Model complexity and interpretability: ML models, particularly deep learning models, can be extremely complex. Their “black-box” nature makes interpretation and explanation more difficult.

  • Debugging and testing are more complex due to non-determinism.

  • Model decay: continuous monitoring and retraining are necessary to maintain model performance.

  • AI-specific regulations at industry and EU level must be taken into account.

  • Interoperability: seamless integration into existing systems and technology stacks.

Participants can decide and explain whether and why a problem should be solved using AI or traditional software development.

LG 1-5: Know roles, their responsibilities, and how they collaborate in the context of AI

Participants know typical roles and their responsibilities in these contexts. In particular, these roles include:

Data Scientist, Data Analyst, Data Engineer, Machine Learning Engineer, MLOps Engineer, AI Architect, Data Architect, Business Intelligence (BI) Developer, Data Governance Specialist, and ML Researcher.

Participants also know how these roles can collaborate within a team (team topologies for ML teams).

LG 1-6 [OPTIONAL]: Explain the concept of the Productivity J-Curve and its significance for the introduction of AI in organizations

This phenomenon helps explain why organizations may initially experience a decline in productivity when implementing AI, followed by productivity gains as implementation progresses.

2. Compliance, security, alignment

Duration: 120 min

Practice time: 30 min

2.1. Terms and Concepts

EU AI Act, data protection, copyright, license, open (source), AI security, jailbreak, adversarial attack, data poisoning, model inversion & extraction, AI safety, AI ethics, AI alignment, model and data documentation, transparency obligation, human oversight, data governance, AI governance, AI systems by risk levels (unacceptable risk, high risk, limited risk, minimal risk)

2.2. Learning Goals

LG 2-1: Know how data protection laws affect the implementation and use of AI

Participants are familiar with data protection laws such as the GDPR and know how they affect the collection, processing, and storage of data by AI systems.

LG 2-2: Analyze the requirements of the EU AI Act and derive architecture-relevant measures for data flows, logging, technical documentation, human oversight, robustness, and deployment

Participants can explain the objectives and provisions of the EU AI Act and analyze their impact on the development and use of AI systems. Furthermore, they can analyze the requirements of the EU AI Act (Trustworthy AI) regarding their impact on the development process and the architecture of the software system and derive architecture-relevant measures. In particular, they can analyze the impact on some of the following aspects and derive corresponding measures:

  • Risk management system (risk mitigation)

  • Data quality and data governance (quality management system)

  • Creation and maintenance of comprehensive technical documentation for the AI system

  • Automatic recording/logging of events in the AI system

  • Provision of clear and comprehensible information to users

  • Implementation of human oversight measures

  • Assurance of an appropriate level of accuracy and robustness

  • Implementation of cybersecurity measures

LG 2-3: Classify AI systems according to the EU AI Act risk levels

Participants can classify AI systems according to the EU AI Act risk levels (unacceptable risk, high risk, limited risk, minimal risk) based on concrete use cases and determine the applicable regulatory requirements.

LG 2-4: Evaluate copyright, degrees of openness, and licensing terms for AI models and AI-generated content

Participants can classify copyright issues relating to training data and AI-generated content and explain their impact on the development, provision, and use of AI systems.

They can distinguish different degrees of openness of ML models, for example with regard to training data, source code, model architecture, and model parameters, and evaluate licensing terms and the resulting restrictions on an AI system.

LG 2-5: Evaluate strategies for compliance with regulatory requirements and determine appropriate evidence and documentation artifacts for models, datasets, and operational processes

Participants understand the fundamental provisions of the EU AI Act, particularly its transparency obligations, and know strategies for complying with them as well as potential challenges.

In particular, they can select documentation artifacts for models, datasets, prompts, evaluation procedures, changes, and operational processes, and explain how these support traceability, transparency, and auditability.

Participants can classify regulatory sandboxes as an instrument for the controlled testing of innovative AI systems and explain their opportunities, limitations, and the consequences of non-compliance with the EU AI Act.

LG 2-6: Know security pitfalls and types of attacks on ML models and select suitable security measures against attacks on AI systems

Participants know potential security pitfalls and typical types of attacks on ML models. These include, for example:

  • LLM jailbreaks through prompt engineering

  • Adversarial attacks

  • Data poisoning

  • Model inversion and extraction

Participants can evaluate the security risks associated with these attack types for AI systems, select appropriate security measures to protect against attacks (AI Security), and justify their selection. In particular, they can evaluate options for integrating security standards and protection mechanisms into the architecture, interfaces, and operating model in terms of their suitability and select appropriate measures.

LG 2-7: Know and apply strategies for AI risk mitigation

Participants can explain how AI risk mitigation strategies are developed and apply them to specific AI systems. In particular, they can use the following strategies as appropriate for the situation:

  • Strengthening robustness through extensive testing

  • Fault-tolerant AI systems

  • Transparent development

  • Explainable AI (XAI)

LG 2-8: Understand the fundamental issues and facets of AI safety

Participants understand the fundamental issues of AI safety and are familiar with its various facets. In particular, they know specific problems such as “AI model risks by poisoning” and bias.

Participants know how bias in data and models can be detected and reduced to ensure fairness and equal treatment in AI applications.

LG 2-9: Translate ethical guidelines and AI governance principles into organizational, development, and architectural decisions

Participants can identify ethical issues that may arise when using AI systems and apply appropriate approaches to addressing these issues in specific use cases, for example when developing AI guidelines. To this end, participants can draw on important ethical guidelines such as the "EU Ethics Guidelines for Trustworthy AI" and translate their principles into concrete organizational, development, and architectural decisions.

Optionally, participants can apply important documents and frameworks on AI Governance to translate the core principles of AI Governance and Responsible AI into concrete guidelines for the organization, e.g.:

3. Design and development of AI systems

Duration: 320 min

Practice time: 45 min

3.1. Terms and Concepts

CI/CD, MLOps, CRISP-ML(Q), AI application, AI engineering, ML model development, ML infrastructure, performance, robustness, reliability, fault tolerance, model-as-service, model-as-dependency, precompute, model-on-demand, hybrid serving, multi-agent system (MAS)

3.2. Learning Goals

LG 3-1: Understand the life cycle of a machine learning or data science project

Participants have an understanding of the life cycle of a machine learning or data science project. In particular, they are familiar with the following phases:

  • Exploratory data analysis

  • Data cleansing and preparation

  • Feature engineering

  • Model training and selection

  • POC

  • Deployment

  • Maintenance

Participants know typical process models for the software development of AI systems. These include, for example:

  • CRISP-ML(Q)

  • Team Data Science Process

  • GenAI Life Cycle

LG 3-2: Understand data types and requirements as well as typical ML problems and their requirements

Participants know different types of data as well as typical ML problems and use cases involving data from the following areas:

  • Supervised learning

  • Unsupervised learning

  • Reinforcement learning

They understand the requirements these may have, for example:

  • Availability of correct labels of different types

  • Data volume and diversity

  • Temporal or semantic dependencies in data features

  • Encoding the input data of ML models, for example:

    • One-hot encodings

    • Label encodings

    • Embeddings

In particular, participants have a good understanding of the need for validation and of the typical division of data into training, validation, and test datasets.

LG 3-3: Select and compare suitable architecture and design patterns for AI systems and justify the selection based on quality requirements

Participants can select suitable design patterns for AI systems based on quality requirements, weigh them against each other, and justify their selection. This applies to the following design patterns:

  • ML systems topology patterns

  • Pipeline architecture patterns

  • Model training patterns

  • Model serving patterns

  • Model deployment patterns

In particular, participants know transfer learning and fine-tuning as ways to adapt pretrained foundation models to existing use cases.

LG 3-4: Know metrics for measuring the performance of ML models

Participants are familiar with various metrics for measuring the performance of ML models, such as:

  • Precision, recall

  • F1 score

  • Accuracy

and know how to define evaluation criteria for performance assessment.

LG 3-5: Integrate AI components into existing systems and domain architectures using suitable integration patterns, interfaces, and decoupling mechanisms

Participants understand how ML models can be integrated into existing systems and are familiar with the corresponding interfaces and integration points. They can apply appropriate integration patterns, interfaces, and decoupling mechanisms to integrate AI components into existing systems and domain architectures.

LG 3-6: Design user interactions for AI systems that appropriately account for uncertainty, explainability, approvals, and human-in-the-loop

Participants can design user interactions and user interfaces for AI systems to enable effective interactions with the ML system and optimize the user experience. In doing so, they can develop concepts that appropriately account for uncertainty, explainability, approvals, and human-in-the-loop.

LG 3-7: Analyze and design the performance and scalability of AI systems

Participants understand the importance of performance metrics such as latency, throughput, and resource utilization in AI systems for both training and inference, and understand how these can be optimized.

They can analyze the causes of performance and scalability issues and select appropriate measures to enable AI systems to handle increasing data volumes, user numbers, and request volumes. Based on this, they can design concepts for performance optimization and scalability of AI systems.

LG 3-8: Understand robustness in AI systems and apply strategies to increase robustness

Participants have an understanding of what robustness means in AI systems and can apply strategies to increase robustness in different application contexts.

LG 3-9: Classify the reliability and availability of AI systems

Participants understand the concepts of reliability and availability and know how to build AI systems that are stable and constantly available.

LG 3-10: Specify measures for the reproducibility, auditability, and traceability of models, dataset versions, prompts, configurations, and results

Participants understand the importance of reproducible and verifiable AI results and are familiar with methods for ensuring these properties. They can evaluate, select, and determine appropriate measures to ensure the reproducibility, auditability, and traceability of models, data versions, prompts, configurations, and results based on specific requirements.

LG 3-11: Classify explainability and interpretability in AI systems

Participants understand the importance of explainability and interpretability in AI systems and know how to ensure this in order to promote trust and transparency.

LG 3-12: Explain fault tolerance in AI systems

Participants are familiar with the concepts of fault tolerance and can explain how AI systems remain functional despite errors or malfunctions.

LG 3-13: Document architectural decisions for AI systems

Participants know how to document and justify architectural decisions for AI systems, for example using Architecture Decision Records. This includes in particular decisions and trade-offs between:

  • RAG versus Fine-Tuning

  • SaaS versus Self-Hosting

  • Batch versus Real-Time

  • Centralized versus Domain-Specific AI Services.

4. Data management and data processing for AI systems

Duration: 90 min

Practice time: 0 min

4.1. Terms and Concepts

Data acquisition, labeling, data pipeline, ETL processes, data aggregation, data cleansing, transformation, augmentation, files, RDBMS, NoSQL, document databases, data products, data contracts, data architectures: Data Warehouse, Data Lake, Data Mesh

4.2. Learning Goals

LG 4-1: Acquire and label data

Participants have an overview of various methods for acquiring and labeling data and can apply appropriate data acquisition and labeling methods to specific use cases.

Participants are familiar with relevant data labeling tools such as CVAT, Label Studio, or Amazon Mechanical Turk.

LG 4-2: Evaluate the sources and suitability of publicly available datasets in terms of quality, licensing, representativeness, and risks

Participants can name common platforms for publicly accessible data and evaluate the suitability of the datasets available on these platforms for specific use cases based on criteria such as quality, licensing conditions, representativeness, and risks, and justify their evaluation.

LG 4-3: Design data pipelines and data architectures for AI systems that support quality, traceability, scalability, governance, and reusability

articipants understand the design of efficient data pipelines and architectures, data quality considerations, and storage solutions and their management. Based on this understanding, they can design data pipelines and data architectures for AI systems that address requirements for quality, traceability, scalability, governance, and reusability. In this context, participants are familiar with:

  • Architectural patterns for data engineering pipelines and ETL processes

  • How effective data management ensures data quality and security in AI applications

Participants have an overview of relevant tools for data engineering pipelines, such as Apache Spark and Flink.

LG 4-4: Know strategies for data aggregation, cleansing, transformation, enrichment, and augmentation

Participants know strategies for data aggregation, cleansing, transformation, enrichment, and augmentation.

LG 4-5: Know options for storing data

Participants know various options for storing data as well as their advantages and disadvantages. The options include the following technologies:

  • CSV files

  • Column-oriented files

  • Relational and document databases

  • Data warehouses

  • Data lakes

  • Vector databases

  • Graph databases

LG 4-6: Classify data products, data contracts, and responsibilities in data architectures for AI systems and use them to establish robust interfaces between the business domain, data platform, and AI components

Participants are familiar with the fundamental concepts of modern data architectures for AI systems and understand the role of data products, data contracts, and business and technical responsibilities in ensuring stable and scalable data flows. They can apply these concepts to specific use cases and use data products, data contracts, and the assignment of responsibilities to design robust interfaces between business domains, data platforms, and AI components.

5. Important quality features for the operation of AI systems

Duration: 160 min

Practice time: 30 min

5.1. Terms and Concepts

Quality features, scalability, performance optimization, monitoring, logging, feedback, FinOps for AI platforms

5.2. Learning Goals

LG 5-1: Know (hardware) requirements for training and inference

Participants know the different (hardware) requirements for training and inference, for example for TPUs, NPUs, LPUs, GPUs, or CPUs.

LG 5-2: Know the trade-offs of different model architectures with regard to quality characteristics

Participants can give examples of trade-offs between different model architectures with regard to quality characteristics. Particularly for scaling, efficiency, and memory footprint, participants should know the trade-offs as well as the advantages and disadvantages of important architectures such as RNNs and transformers.

LG 5-3: Adjust different quality features of an ML model

Participants are familiar with approaches for balancing and trading off different quality attributes of an ML model, such as accuracy, efficiency, and memory usage. They can apply appropriate techniques to balance these quality attributes based on specific requirements and trade-offs. In particular, participants can use the following techniques as appropriate for the situation:

  • Quantization

  • Pruning

  • Distillation

  • LoRA

LG 5-4: Understand the costs, power consumption, and sustainable use of AI (Green IT)

Participants gain an understanding of the costs, power consumption, and sustainable use of AI (Green IT). In particular, they know how to develop AI models and systems that operate efficiently by making economical use of memory, compute capacity, and storage space.

LG 5-5: Gain an overview of MLOps for automating the life cycle of a data science project

Participants know the term MLOps for automating the life cycle of a data science project and its relationship to DevOps.

Using a practical example, participants understand what an MLOps pipeline can look like and what insights it provides into parameters, metrics, and other aspects.

LG 5-6: Track model training, parameters, metrics, and results

Participants understand the tracking of model training, parameters, metrics, and results. They can apply appropriate tracking methods to systematically record and monitor training runs, parameters, metrics, and results.

LG 5-7: Distinguish model evaluation, system evaluation, and assessment of business value, and select suitable evaluation approaches for AI systems

Participants have an overview of approaches for evaluating ML models and AI systems built upon them. They can distinguish between model evaluation, system evaluation, and business value assessment, evaluate and select appropriate evaluation approaches for specific AI systems based on relevant requirements and quality criteria, and justify their selection.

LG 5-8: Know types of drift and possible causes and solutions for them

Participants are familiar with different types of drift, such as data drift or model drift, as well as possible causes and solutions.

LG 5-9: Gain an overview of CI/CD pipelines, model management, and deployment strategies for AI models

Participants understand CI/CD pipelines, model management, and deployment strategies for AI models.

Participants know a selection of different deployment options for AI models:

  • API deployment

  • Embedded deployment

  • Batch prediction

  • Streaming

  • Containerization

  • Serverless deployment

  • Cloud services

Participants know common platforms and tools for model provision and for creating POCs and understand how they work conceptually.

LG 5-10: State the advantages and disadvantages of SaaS and self-hosting and make make-or-buy decisions based on quality requirements

Participants know the advantages and disadvantages of SaaS and self-hosting and can weigh them against each other.

Participants can make make-or-buy decisions for MLOps and AI platform systems or components based on quality requirements, cost, risk, compliance, and integration effort.

Participants have an overview of well-known SaaS AI solutions, such as Azure OpenAI Services.

LG 5-11: Design monitoring and observability for AI systems, particularly for drift, quality degradation, costs, latency, security events, and governance-relevant evidence

Participants understand the need for monitoring, including AI-specific requirements such as drift tracking. Based on this understanding, they can design monitoring and observability concepts for AI systems that specifically address drift, quality degradation, costs, latency, security events, and governance-related evidence.

Participants are familiar with relevant metrics such as accuracy, precision, recall, F1-score, MAE, MSE, perplexity, latency, throughput, and resource utilization, and understand why these are relevant for monitoring.

Participants are familiar with examples of monitoring tools. These include general-purpose tools such as Prometheus & Grafana, as well as ML-specific tools such as MLflow.

LG 5-12: Understand user feedback and methods and tools for collecting it

Participants understand the benefits of user feedback for further model training. They also know various methods and tools for collecting user feedback, such as choosing between multiple answers or flagging in Gradio.

Participants know further methods for obtaining user feedback, such as RLHF, RLAIF, and DPO.

LG 5-13: [OPTIONAL] Know classes of MLOps tools and platforms and essential criteria for selecting them

Participants know well-known MLOps tools and end-to-end platforms, such as:

  • Domino Data Lab, h2o.ai, DVC, activeloop, aporia, argo, arize, bentoML, comet ML, DagsHub, Databricks MLOps Stacks, Feast, Kedro, Metaflow, MLflow, MLRun, prefect, PrimeHub, Weights & Biases, WhyLabs, zenML, KNIME, RapidMiner, NVIDIA AI Enterprise, watsonx.ai

  • Open source: MLflow, Kubeflow, ClearML

  • PaaS: AWS SageMaker, Azure ML, Google Vertex AI

6. System architectures and platforms for generative AI systems

Duration: 160 min

Practice time: 30 min

6.1. Terms and Concepts

Generative AI, LLMs, MLflow, Managed MLflow, Azure Machine Learning, Metaflow, Generative AI, LLM, (Stable) Diffusion, Vector DB, Embedding, RNN, Transformer, RAG, Agentic Workflows etc.

6.2. Learning Goals

LG 6-1: Gain an overview of AI integration levels

Participants can distinguish between different levels of AI integration and explain their respective functions and areas of application. These include the following levels:

  • Applications (e.g. coding assistants)

  • AI Engineering (e.g. prompt engineering)

  • ML Model Development (e.g. PyTorch)

  • ML Infrastructure (e.g. vector databases).

Participants are familiar with some examples of commonly used libraries, interfaces, and tools for integrating AI models.

LG 6-2: Integrate AI systems into the overall architecture of an IT landscape

Participants are familiar with approaches for integrating AI systems into IT landscapes at a strategic level. For example, they can use strategic design from DDD (particularly Context Maps) to determine and document the type and degree of integration of AI systems.

Participants practice applying suitable AI integration approaches to an existing software landscape using a case study with a fictional business domain and discuss their implementation.

LG 6-3: Develop a fundamental understanding of generative AI and how it works

Participants have a basic understanding of generative AI, such as LLMs and diffusion models.

Participants understand how LLMs work and can classify the associated terminology: token, embedding, RNN, transformer, attention.

LG 6-4: Know architecture patterns for using LLMs and evaluate their limitations, risks, and trade-offs, particularly for RAG, function calling, fine-tuning, assistance systems, and agents

Participants are familiar with common patterns for using LLMs and can evaluate their limitations, risks, and trade-offs based on specific requirements and application scenarios. This includes the following patterns:

  • RAG and retrieval strategies

  • Function calling

  • Fine-tuning

  • Assistants

  • Agents

LG 6-5: Know and understand use cases for RAG (Retrieval-Augmented Generation) and selected RAG techniques

Participants can explain typical use cases for RAG, such as "Talk to your documents/database/API".

In particular, participants can explain the basic functionality of a selection of common RAG techniques and distinguish between them, for example:

  • Regular RAG, GraphRAG, CAG, agentic RAG, and multi-agent RAG

  • Context enrichment techniques

  • Fusion retrieval

  • Intelligent reranking

  • Query transformations

  • Adaptive retrieval

  • Iterative retrieval

  • Ensemble retrieval

  • Knowledge graph integration

LG 6-6: Know types of prompt engineering

Participants know different types of prompt engineering, such as:

  • Few-shot learning

  • Chain-of-thought

  • Role-playing

as well as general best practices for prompting.

LG 6-7: Analyze and architecturally classify agentic workflows with regard to planning, tool use, reflection, state management, control mechanisms, and multi-agent collaboration

Participants understand what agentic workflows are and are familiar with the concepts of reflection, tool use, planning, and multi-agent collaboration. They can analyze agentic workflows with regard to planning, tool use, reflection, state management, control mechanisms, and multi-agent collaboration, examine the relationships between these components, and assess their implications for the architecture of AI systems.

LG 6-8: Know a selection of design patterns for generative AI systems

Participants know which design patterns exist for generative AI systems. They know a selection of the following patterns:

  • AI Query Router (Simple Router; Ranking-based Router; Learning-based Router)

  • Layered Caching Strategy Leading to Fine-Tuning

  • Multiplexing AI Agents

  • Fine-Tuning LLMs for Multiple Tasks

  • Blending Rules-Based and Generative Approaches

  • Utilizing Knowledge Graphs with LLMs

  • Swarm of Generative AI Agents

  • Modular Monolith LLM Approach with Composability

  • Memory Cognition for LLMs

  • Red and Blue Team Dual-Model Evaluation

LG 6-9: Select techniques for evaluating LLM applications and apply them to response quality, retrieval quality, hallucinations, safety, costs, and user value

Participants are familiar with several techniques for evaluating LLM applications. They can evaluate their suitability for specific evaluation objectives, select appropriate techniques, and apply them to assess response quality, retrieval quality, hallucinations, security, costs, and user value. These may include the following techniques:

  • Scoring

  • Human feedback

  • Comparative evaluation

  • Model-based evaluation

Participants know common evaluation frameworks such as LangSmith or LangFuse for dealing with indeterminacy and errors in AI systems.

LG 6-10: Gain an overview of well-known LLMs and selection criteria

Participants are familiar with common LLMs such as GPT, Claude, Gemini, Llama, Mistral, or Luminous and can explain their key differences as well as relevant criteria for selecting a suitable LLM.

LG 6-11: Understand the importance of cost management for GenAI applications

Participants understand the importance of cost management for GenAI applications.

LG 6-12: Know classes of libraries, interfaces, and tools for LLM applications as well as their application contexts and selection criteria

Participants know some examples of common libraries, interfaces, and tools related to LLM applications, such as the OpenAI API or LangChain.

LG 6-13: [OPTIONAL] Know and evaluate agentic AI architectures, key architecture components, security and governance boundaries, and typical usage patterns and limitations of agent systems

Participants are familiar with agentic AI software architectures, AI agent architectural components, and types of AI agent architectures. They can evaluate different agentic AI architectures in terms of their key architectural components, typical usage patterns, limitations, and security and governance requirements, and justify their assessment based on specific application scenarios.

LG 6-14: Architecturally classify and apply agent runtime systems for LLM and agent systems

Participants understand agent runtime systems as an architectural control and integration layer for probabilistic AI components such as LLMs and agents. They can describe harness approaches using established architecture concepts such as encapsulation, interface control, invariants, operational boundaries, observability, policy enforcement, fault isolation, controlled change, governance, and auditability, and can apply these approaches when designing production-ready AI systems.

In particular, they can derive how an agent runtime system:

  • limits the autonomy of an AI component through clear interfaces, permitted actions, and operational boundaries,

  • controls inputs, context, tool access, state, and outputs,

  • enforces system invariants and policies,

  • isolates errors and unsafe model outputs,

  • ensures observability, traceability, and auditability,

  • enables controlled changes to prompts, models, retrieval components, and tools,

  • supports the secure operation and safe evolution of AI systems.

7. Case studies and practical projects

Duration: 110 min

Practice time: 110 min

7.1. Learning Goals

LG 7-1: Use an end-to-end case study or practical project to make, document, and justify architectural decisions for an AI system and evaluate them in terms of quality, compliance, operations, and integration.

Participants can independently make, document, and justify architectural decisions for an AI system based on an end-to-end case study or practical project. They can evaluate these decisions with regard to relevant quality requirements, regulatory requirements (compliance), operational aspects, and integration into existing IT systems.

7.2. References

References

This section contains references that are cited in the curriculum.

A

B

C

D

E

F

G

H

J

K

L

M

N

O

P

R

S

T

V

W

Z