AI Model Selection for EU AI Act Compliance

Model origin risk varies materially across three categories, and the provenance of a model carries implications that the AI System Assessor must assess and document in AISDP Module 3.

Open-source models from repositories such as Hugging Face or GitHub offer accessibility and community validation but introduce several risks. Training data provenance may be unknown or poorly documented. The development process may not have included the bias testing, adversarial evaluation, or documentation records that the EU AI Act requires. Any organisation using an open-source model as a component of a high-risk system inherits these documentation gaps and must either fill them through its own due diligence or accept the resulting non-conformity risk. AISDP Module 3 must record which open-source components are incorporated, the due diligence performed on each, the licensing terms and their compatibility with the system's commercial and regulatory context, and the residual risks from provenance gaps.

Proprietary third-party models present different challenges. A commercial provider may refuse to disclose training data composition, architecture details, or fairness evaluation results, citing trade secrets. This creates an AISDP documentation gap that the organisation must address. Where vendor disclosures are insufficient, the AI System Assessor records the gaps as non-conformities, and the organisation assesses whether it can compensate through its own output-level testing and evaluation.

In-house models offer the greatest control over documentation and governance. The risk centres on process discipline: whether the development team followed documented methodology, satisfied training data governance requirements, and conducted rigorous testing. Models developed informally, during hackathons or innovation sprints, may lack the documentation infrastructure the AISDP demands.

For models downloaded from public repositories such as Hugging Face, TensorFlow Hub, or PyTorch Hub, the provenance discipline requires additional rigour. Best practice is to download the model once, compute a cryptographic hash (SHA-256), store the model and its hash in the internal model registry, and ensure that all subsequent references use the internal copy. This prevents silent changes if the repository updates the model under the same identifier. Hugging Face's transformers library supports this through the revision parameter, which pins to a specific Git commit SHA.

For all model origins, provenance discipline is essential. SLSA (Supply-chain Levels for Software Artifacts) provides a framework adapted from software supply chain security. Level 2 is the minimum practical target: automated model training pipelines producing verifiable provenance metadata linking the trained model to its training code, data, hyperparameters, and execution environment. Model signing via Sigstore provides tamper evidence, and SBOM generation using SPDX or CycloneDX enables vulnerability scanning and licence compliance checking.

Organisations that fine-tune a general-purpose AI model for use in a high-risk system cross the provider boundary defined in Article 25(1)(b), assuming the full set of provider obligations under Article 16 in most cases.

Article 25(1)(b) provides that a deployer becomes a provider if it places on the market a high-risk AI system under its own name or trademark, or makes a substantial modification. Fine-tuning a general-purpose foundation model for a specific high-risk application, such as adapting a general LLM for credit assessment or recruitment screening, changes the model's intended purpose from "general purpose" to a specific high-risk use case. This triggers provider status with the complete set of obligations: conformity assessment, AISDP preparation, CE marking, Declaration of Conformity, and EU database registration.

The Legal and Regulatory Advisor should assess fine-tuning against three criteria. First, does it change the model's intended purpose as documented by the gpai provider? If the provider describes the model as general-purpose text generation and the organisation fine-tunes it for medical triage, the intended purpose has changed. Second, does it alter the model's risk profile? Fine-tuning on domain-specific data may introduce new bias patterns, failure modes, or accuracy characteristics not present in the base model. Third, does it affect the model's compliance with the GPAI provider's own obligations under Articles 51 to 56? Fine-tuning may void safety evaluations or alignment testing conducted on the base model.

Where provider status is triggered, the AISDP must document the fine-tuning process with the same rigour applied to in-house development: fine-tuning data governance, methodology and hyperparameters, evaluation against performance and fairness thresholds, and a clear delineation between base model characteristics inherited from the GPAI provider and fine-tuned characteristics under the organisation's responsibility. The Model Selection Record documents the base model as a GPAI integration decision and the fine-tuning as a development decision, with separate risk assessments for each.

For parameter-efficient methods such as LoRA, QLoRA, prefix tuning, and adapters, the same analysis applies. The compliance boundary does not depend on the volume of parameters modified; it depends on whether the modification changes the intended purpose or risk profile. A LoRA adapter that redirects a general-purpose model toward a high-risk use case triggers the same Article 25(1)(b) analysis as full fine-tuning.

The information rights framework under Article 25(3) provides a structured mechanism for downstream providers to obtain the technical information they need from the GPAI model provider. Where the GPAI provider's disclosures are insufficient for the downstream provider to complete its own AISDP, the downstream provider may submit a structured information request covering training data composition, known limitations, evaluation results, and safety testing outcomes. The AI System Assessor documents each request, the response received, and any gaps that require compensating controls. The GPAI Code of Practice under Article 56 provides additional compliance pathways for GPAI providers; downstream organisations should verify whether their provider has committed to the Code and factor that into their risk assessment.

Training data for AI models, particularly large language models and generative systems, may include copyrighted material, and the legal landscape is evolving rapidly with litigation in multiple jurisdictions challenging the legality of training on copyrighted content without licence.

For high-risk AI systems, the AISDP must document the copyright status of training data comprehensively. This includes identifying whether the data includes copyrighted works (text, images, audio), the legal basis relied upon for processing that material (licence, consent, or the text and data mining exception under Directive (EU) 2019/790), the measures taken to identify and exclude material where rights holders have exercised opt-outs, and the procedures for responding to copyright claims.

For systems incorporating pre-trained third-party models, the organisation should obtain contractual representations regarding the copyright status of the model's training data. Where such representations are unavailable or qualified, the AI System Assessor records the risk and assesses potential regulatory and reputational impact. The risk register should distinguish between models where training data copyright is documented and verified, models where the provider asserts clean training data but cannot provide detailed evidence, and models where training data provenance is substantially unknown.

The risk register should distinguish between three tiers of copyright confidence: models where training data copyright is documented and verified through licensing agreements, models where the provider asserts clean training data but cannot provide detailed provenance evidence, and models where training data composition is substantially unknown. Each tier carries different residual risk levels that the AI System Assessor quantifies in the risk assessment. For the third tier, the risk may be sufficient to disqualify the model for high-risk applications unless the organisation can demonstrate compensating controls such as output filtering for copyrighted content patterns.

Provider data collection practices add another layer of intellectual property risk. Many cloud-based model providers collect inputs, outputs, and usage patterns from customer interactions. This data may be used to improve the provider's own models, incorporating the organisation's proprietary data and potentially personal data of affected individuals into the provider's training corpus. AISDP Module 3 must record the provider's data collection practices, the data processing agreement in place, measures to prevent personal data leakage (such as pseudonymisation before API calls), and residual risks.

Foundation models reflect the values, cultural norms, and policy perspectives embedded in their training data and alignment processes, creating compliance risks when deployed in EU contexts that the AISDP must document.

Training data geographic bias is the most common concern. Models trained predominantly on data from a particular jurisdiction may perform poorly on EU populations. A credit scoring model trained on US financial behaviour data may not generalise to European markets with different consumer protection frameworks. A natural language processing model trained on American English may handle EU-specific legal terminology, regulatory references, or cultural contexts poorly.

Alignment and safety tuning reflects the model provider's values and regulatory environment. A model aligned primarily to US free speech norms may not meet EU expectations regarding hate speech, disinformation, or content moderation. The assessment must evaluate the model's alignment against EU legal and ethical standards and document additional fine-tuning or guardrails applied to address gaps. Behavioural benchmarking using frameworks like HELM with EU-specific thresholds, testing against the Framework Decision on combating racism and xenophobia (2008/913/JHA), and EU-specific MMLU subsets provide quantified evidence.

Foreign government influence requires evaluating the model provider's ownership structure, governance, and known government relationships, particularly for sensitive domains like law enforcement, migration, or public administration. Data sovereignty risks arise when inference or training infrastructure is hosted outside the EU, exposing the organisation to foreign government access under domestic laws such as the US CLOUD Act or China's National Intelligence Law. The cybersecurity module documents these risks.

For non-LLM systems, geographical bias testing focuses on whether performance varies across EU member state populations. A credit risk model influenced by US credit market data may assign systematically different risk scores to applicants whose profiles reflect European consumer behaviour. The evaluation dataset should be representative of the deployment population in each target member state, and performance metrics must be disaggregated by member state or region. Significant variations indicate geographical bias requiring additional training data, fine-tuning, or post-processing calibration.

The geopolitical dimension extends beyond model behaviour to infrastructure governance. Under the US CLOUD Act, US authorities can compel US-based cloud providers to disclose data stored anywhere in the world. China's National Intelligence Law requires Chinese organisations to cooperate with state intelligence work. These risks may influence infrastructure decisions such as choosing EU-based cloud regions or self-hosting inference infrastructure within the EU.

The practical output is a documented position covering: the provider's jurisdiction and governance structure, alignment testing performed against EU standards, results and gaps, additional fine-tuning or guardrails applied, infrastructure hosting arrangements and data sovereignty analysis, and residual risks. This assessment must be repeated whenever the model provider releases a new version, because alignment and safety tuning can change between versions.

Model selection for high-risk systems should be evaluated against six compliance criteria alongside traditional performance metrics, scored on a three-level scale (strong, adequate, weak) with evidence-based justification for each score.

Documentability asks whether the architecture and decision process can be described to Annex IV standard, such that a qualified reviewer could reproduce the training process from documentation alone. Linear models have strong documentability where every parameter is a named coefficient. Transformers with billions of parameters have weaker documentability; the architecture can be described but learned representations cannot be enumerated.

Testability asks whether accuracy, robustness, and fairness can be evaluated meaningfully per Article 15. Deterministic architectures such as decision trees and linear models simplify testing. Stochastic architectures like LLMs require statistical testing frameworks and the assessment must specify the methodology needed.

Auditability asks whether individual decisions can be reconstructed from logs per Article 12. Models requiring only the input and model version for output reconstruction are strongly auditable. Models depending on runtime conditions, session state, or RAG context require more sophisticated logging.

Bias detectability asks whether fairness metrics can be computed at subgroup level and proxy variable effects identified. Models producing calibrated probability scores are more amenable to fairness analysis than those producing only ranked outputs or categorical labels. SHAP and integrated gradients enable feature attribution analysis for proxy detection.

Maintainability asks whether the model can be retrained, fine-tuned, or recalibrated in response to post-market monitoring findings without triggering a substantial modification. Gradient-boosted trees and logistic regression produce stable, predictable changes when retrained; deep neural networks can exhibit large behavioural shifts from small data changes.

Determinism asks whether the same input produces the same output consistently. Stochastic models require temperature clamping, seed fixing for reproducibility in testing, and output logging for traceability. The performance cost of these controls must be assessed.

The criteria interact in practice. A model with strong determinism but weak documentability may satisfy Article 12 logging requirements but fail Annex IV documentation standards. A model with strong bias detectability but weak maintainability allows the organisation to identify fairness issues but not to remediate them without triggering a substantial modification assessment. The evaluation must consider these interactions, not just score each criterion independently.

Results predictability is a cross-cutting concern. As systems ingest new data, are retrained, or undergo algorithmic modifications, their behaviour can shift unpredictably. The AISDP must define quantitative thresholds for determining when model evolution constitutes a substantial modification under Article 3(23). Triggers might include a change in AUC-ROC exceeding a defined tolerance, any subgroup fairness metric breaching established thresholds, or changes to the model's top-five feature importance ranking. These thresholds are integrated into the CI/CD pipeline's automated quality gates documented in CI/CD pipelines.

The criteria matrix should be weighted by the system's risk profile. For employment domain systems where human oversight is paramount, explainability-adjacent criteria carry higher weight. For safety-critical systems, testability and determinism carry higher weight. Weights are documented and approved before evaluation begins to prevent post-hoc rationalisation.

Agentic AI systems that take actions in the world beyond producing predictions introduce a distinct set of model selection considerations that the AISDP must address, as Article 3(1) encompasses behaviour where outputs trigger real-world actions.

Action space constraints are the primary concern. The model architecture must support constraining the action space to the set of actions that are safe and within the system's intended purpose. An agentic system for a high-risk domain must have a bounded action space that can be documented, tested, and monitored. Open-ended architectures that can theoretically take any action require extensive guardrails whose effectiveness must be demonstrated in the AISDP. In frameworks like LangChain and LangGraph, this is implemented through tool definitions with Pydantic schemas that validate every tool call before execution.

Reversibility classification adds a second layer. Each permitted action is classified as reversible (can be undone programmatically), partially reversible (can be undone with effort, such as a database rollback), or irreversible (cannot be undone, such as a sent communication or physical actuator command). Irreversible actions require a human-in-the-loop checkpoint: the agent proposes, a human reviews and approves, and only then does execution proceed. Agentic AI compliance covers bounded autonomy and tool-use governance in detail.

Chain-of-action traceability requires the model to support logging the entire reasoning and action chain: the initial trigger, each intermediate observation and decision point, each tool call with parameters and return values, and the final outcome. This is essential for Article 12 compliance and post-incident investigation. The log is structured as JSON with a defined schema, written to append-only storage.

The boundary between permitted and unpermitted actions requires careful calibration. Too narrow, and the agent cannot accomplish its intended purpose. Too broad, and the agent has excessive agency (OWASP LLM08: Excessive Agency), creating unnecessary risk surfaces. Boundary derivation starts from the system's documented intended purpose in AISDP Module 1; every action necessary for the intended purpose is permitted, and every action that is not is excluded. The AI System Assessor documents the mapping from intended purpose to permitted actions, and this mapping is reviewed during conformity assessment.

Failure mode complexity distinguishes agentic systems from prediction systems. An agentic system can fail by taking incorrect actions, acting at the wrong time, entering infinite loops, escalating permissions, or interacting with external systems in unintended ways. Timeout and iteration limits (typically 10 to 20 tool calls per execution) must be hard-enforced at infrastructure level, not within the agent's own code where adversarial inputs could override them.

The choice of model architecture directly constrains the organisation's ability to satisfy Article 13 transparency and Article 14 human oversight requirements, and explainability properties must be assessed during selection rather than retrofitted after development.

Intrinsic explainability is provided by architectures where the decision process is transparent by construction: linear models expose coefficients, decision trees expose branching logic, rule-based systems expose rules. Post-hoc methods (SHAP, LIME, attention maps, counterfactual explanations) approximate reasoning after the fact with known limitations in fidelity and stability.

Three distinct audiences require different explanation formats. Operators need explanations enabling independent judgement; SHAP values decomposing predictions into per-feature contributions serve this well, with TreeExplainer providing exact, fast computation for tree-based models. Affected persons need plain-language explanations answering "why was this decision made"; Alibi Explain's counterfactual explanations are particularly useful, answering "what would need to be different for the outcome to change." Auditors need explanations consistent with documented system design, revealing whether the model actually behaves as described in the AISDP.

Fidelity testing is the quality assurance step most organisations skip. An explanation has high fidelity if it accurately reflects the model's actual reasoning. Testing involves systematically perturbing inputs according to explanation attributions and verifying the model's output changes as predicted. If removing the feature identified as most important does not meaningfully change the output, the explanation is unfaithful.

Computational cost matters for production systems. SHAP values for complex models can require thousands of evaluations per explanation. Practical approaches include full explanations for a random sample (sufficient for monitoring and audit), lightweight top-3 feature explanations for all predictions, and pre-computed explanations for common patterns. The AISDP must document explanation coverage, method, and computational overhead.

For LLM-based systems, feature attribution methods do not apply meaningfully to generative models. Instead, explanations rely on three approaches: chain-of-thought prompting that instructs the model to articulate reasoning step by step, retrieval source citation that identifies the documents contributing to a RAG response, and structured output formats that require the model to separate its answer from its reasoning. These approaches are inherently less precise than SHAP values; the model's stated reasoning may not accurately reflect its internal computations. The AISDP must document this limitation and describe the compensating controls, including output quality monitoring and human evaluation sampling, that provide assurance about explanation quality.

Explanation quality monitoring in production tracks whether explanations remain faithful and useful over time. Fidelity testing should be run periodically on production predictions, comparing explanations against the model's actual sensitivity to feature perturbations. Explanation pattern monitoring, tracking the distribution of top features in explanations over time, provides an early warning signal for model behaviour drift. Human evaluation sampling, presenting a random sample of explanation-output pairs to domain experts for quality rating, provides ground truth against which automated quality metrics can be calibrated.

Model selection for a high-risk AI system is a compliance-significant decision that requires a defined governance process documented in the AISDP, not discretion left to individual data scientists.

The governance process involves multiple roles. The Technical SME proposes candidate architectures, evaluates them against the six compliance criteria, and recommends a selection. The AI Governance Lead reviews the recommendation against risk assessment findings. Legal counsel reviews intellectual property and GPAI provider implications. For commercially significant choices, the Business Owner is consulted. The AI Governance Lead approves the final selection. A formal objection mechanism allows any stakeholder to challenge the choice; objections are documented, discussed, and resolved with the resolution recorded.

The weighted decision matrix provides the evaluation structure. Each of the six criteria is scored for every candidate model, with weights reflecting the system's risk profile. Scoring must be evidence-based: "documentability: strong" means a qualified reviewer could reproduce training from documentation alone. Weights are documented and approved before evaluation begins.

Model cards following the Mitchell et al. (2019) format capture the standardised documentation output: intended use, disaggregated performance results, known limitations, training data summary, and ethical considerations. Google's Model Cards Toolkit automates generation from evaluation metrics, producing structured documents for AISDP Module 3.

The Model Selection Record must cover every learned component in the system architecture, not only the primary decision model. Many high-risk systems incorporate multiple components: embedding models for retrieval or similarity, re-ranking models, classification heads for routing, and auxiliary models for monitoring or safety. Each influences outputs and carries its own risk profile. A primary decision model warrants full evaluation against all six criteria. An embedding model in a RAG pipeline warrants focused assessment covering provenance, linguistic performance, known biases, and version pinning. An auxiliary safety classifier warrants documentation of accuracy, failure modes, and consequences for the primary system's compliance profile.

The rationale document covers: functional requirements, compliance requirements from the risk assessment, candidate architectures evaluated (including traditional approaches), evaluation methodology and results, the recommended selection with trade-offs accepted, and the governance approval record.

The governance record serves two audiences. The Technical SME reviews it for technical soundness, verifying that the evaluation methodology was appropriate and the results were correctly interpreted. The Classification Reviewer and any notified body review it for evidence that the organisation made a considered, risk-aware choice and did not simply default to the most complex available model. A well-constructed rationale demonstrates that simpler approaches were evaluated and that the selected architecture's compliance costs (in documentation, testing, and explanation infrastructure) were understood and accepted.

The selection process should also include a formal challenge mechanism. If any stakeholder, whether the Classification Reviewer, Legal and Regulatory Advisor, or Technical SME, believes the selected model does not adequately satisfy the compliance criteria, they must be able to raise a formal objection. The objection is documented, discussed, and resolved; the resolution, including any changes to the selection or additional compensating controls, is recorded in the rationale document. This governance trail is part of the AISDP evidence pack and must be retrievable if a competent authority asks why a particular architecture was chosen.

Any model component visible in the architecture diagram that lacks a corresponding entry in the Model Selection Record is a documentation gap that the conformity assessment process will identify.