Training Effectiveness Agent
Training effectiveness measured, not assumed - Kirkpatrick and Phillips ROI aggregated by rule, with AI-supported transfer indicators, so the CSRD skills disclosure and the Apprenticeship Levy spend stand up to a Big-4 auditor.
Training measurement: Kirkpatrick 4-Levels + Phillips ROI 5-Levels, US WIOA + DOL Apprenticeship, UK Apprenticeship Levy and ESCO/ISCED - CSRD ESRS S1-13 skills reporting and ISO 30414.
Analyse your processA selection from over 5,000 projects in 25 years of software development
Kirkpatrick and Phillips ROI aggregated by rule, with AI limited to transfer indicators
Most of the measurement is rule-based. Effectiveness aggregation - the four Kirkpatrick levels and the Phillips ROI fifth level, the CSRD ESRS S1-13 datapoints, ISO 30414 metrics, and the Apprenticeship Levy and WIOA reporting - runs deterministically from LMS data, survey responses, business-KPI feeds, and skills-framework mappings, with no generative AI in the measurement. AI is confined to transfer indicators, skill-gap detection, and content recommendations, and the L&D Director and business sponsor validate them.
Outcome: The exposure is concrete: a Big-4 limited-assurance qualification on a missing skills-development disclosure, ESFA findings if Apprenticeship Levy funds expire unused at 24 months, WIOA performance failures with federal-funding consequences, EEOC disparate-impact claims on training selection without statistical validation, and GDPR fines up to 4 percent of group revenue for a missing lawful basis. The agent supplies the SOC-2-grade auditable chain that prevents them.
The architecture follows from that: measurement aggregation is deterministic, and AI is limited to transfer indicators, never to individual learner consequences.
From assumed training value to measured business impact - 78 percent of the measurement deterministic, the ML part limited to transfer indicators
Training-effectiveness measurement with Kirkpatrick and Phillips ROI
This agent follows the Decision Layer principle: each decision is either rule-based, AI-assisted, or explicitly assigned to a human. It is not a high-risk system under the EU AI Act 2024/1689 - it is measurement aggregation without individual learning consequences - but it carries obligations from US WIOA, the UK Apprenticeship Levy and its regulators, the EU skills frameworks, the CSRD ESRS S1-13 standard, ISO 30414, SOC 2, and GDPR.
The measurement decomposes into the four Kirkpatrick levels - reaction (post-training satisfaction and content relevance), learning (pre- and post-test scores and certification pass rates), behaviour (90-day on-the-job application from manager and peer feedback), and results (business KPIs such as revenue, quality, and retention) - and Phillips ROI extends it with a fifth, monetary benefit-cost level using isolation-of-effects methods.
The challenge is not the framework. It is the auditable chain around it: a SOC-2-grade audit-trail with xAPI lineage, the ESG limited-assurance scope, four-eye sign-off, and works-council co-determination on the aggregate measurement.
US WIOA, the UK Apprenticeship Levy, ESFA, and Ofqual
The US Workforce Innovation and Opportunity Act mandates six Joint Performance Indicators across its four titles: employment in the second and fourth quarters after exit, median earnings, credential attainment, measurable skill gains, and effectiveness in serving employers. The state workforce boards and DOL ETA aggregate the reports.
US registered apprenticeships under 29 CFR 29, with the 29 CFR 30 equal-employment rules, submit data to the RAPIDS database, complemented by the Industry Recognized Apprenticeship Programs and the Work Opportunity Tax Credit.
The UK Apprenticeship Levy charges 0.5 percent of pay bills over 3M GBP, collected through PAYE into an Apprenticeship Service Account with a 24-month fund expiry and a 10 percent government top-up. The ESFA oversees the funding rules and audit, Ofqual governs End-Point Assessment, and IfATE manages the standards.
EEOC training requirements under Title VII and the related anti-discrimination statutes, with the Uniform Guidelines four-fifths rule, require disparate-impact analysis on training selection to prevent discrimination claims.
ESCO, ISCED, and CSRD ESRS S1-13 skills reporting
The EU ESCO classification covers roughly 14,000 occupations and 13,000 skills and competences, aligned with the European Qualifications Framework. UNESCO’s ISCED 2011 provides nine education levels from early childhood through doctoral, complemented by Eurostat and OECD education statistics.
The CSRD’s ESRS S1-13 requires training-and-skills-development data - average training hours per employee, the share of employees receiving regular development reviews, and training participation across diversity dimensions - with each KPI mapped to source data through the EFRAG datapoints. Limited assurance applies from 250 employees, performed by the Big-4 firms.
ISO 30414 adds human-capital metrics for training-and-skills-development, leadership, organisational culture, and productivity, and ISO 9001’s competence, awareness, and documented-information clauses ensure the underlying records hold up.
GDPR Article 88 and works-council training codetermination
GDPR governs the training data: Article 6 lawful basis, Article 9 special categories, the Article 22 bar on automated individual decisions, and the Article 88 employment rules. The agent pseudonymises at extraction (Article 4(5)) and applies aggregation thresholds - a minimum of five employees per cohort - to prevent re-identification.
Works-council co-determination under the EU Information and Consultation Directive and national co-determination acts is mandatory before any aggregate-measurement go-live in the EU; a works-council objection blocks it. The EDPB HR-AI guidelines and the national supervisory authorities are the reference points.
A GDPR Article 35 DPIA is required before deployment, with an EU AI Act Article 27 FRIA where the ML transfer indicators would trigger classification.
Cross-reference to Skills-Career-Profile, Performance-Review-Documentation, and Learning-Path-Recommendation
Skills-Career-Profile-Agent feeds individual skills assessments into the aggregate measurement for gap analysis. Performance-Review-Documentation-Agent uses the Level 3 behaviour observations as evidence in reviews. Learning-Path-Recommendation-Agent uses the skill-gap detection and content recommendations for individual paths. Strategic-HR-Analytics-Agent folds the measurements into board reporting and ESG disclosure. Workforce-Planning-Agent uses the skills-framework mappings for headcount scenarios. Succession-Planning-Agent uses leadership-development effectiveness for board-level talent reviews, and Compensation-Benchmarking-Agent folds skills-certification effects into compensation bands. The ESG-Reporting-Agent extends the ESRS S1-13 data to full sustainability reporting, and the Audit-Compliance-Agent verifies SOC 2.
At a glance
- Classification: Compliance-Support, not EU AI Act high-risk (measurement aggregation)
- Compliance anchors: US WIOA and registered apprenticeships, EEOC disparate-impact rules, the UK Apprenticeship Levy and its regulators, the EU ESCO and ISCED frameworks, CSRD ESRS S1-13, ISO 30414 and 9001, SOC 2, and GDPR
- Measurement framework: the four Kirkpatrick levels and the Phillips ROI fifth, monetary benefit-cost level with isolation-of-effects methodology
- Approval: four-eye principle (L&D Director, business sponsor, CHRO, ESG Officer) with works-council consultation; interpretation and commentary human-only
- Penalties: ESG limited-assurance qualifications, ESFA audit findings and 24-month fund expiry, WIOA performance failures, EEOC disparate-impact claims, and GDPR fines up to 4 percent of group revenue
- Audit obligation: SOC 2, ESG limited assurance from 250 employees, the annual ESFA funding audit, WIOA performance accountability, RAPIDS apprenticeship submission, and EEOC disparate-impact verification
- Cross-Reference: Skills-Career-Profile, Performance-Review-Documentation, Learning-Path-Recommendation, Strategic-HR-Analytics, Workforce-Planning, Succession-Planning, Compensation-Benchmarking, ESG-Reporting, and Audit-Compliance agents
Decision-Maker Distribution Training-Effectiveness
| Step | Decider | Rationale |
|---|---|---|
| LMS data extraction, xAPI/SCORM, pseudonymisation | R | Deterministic source-to-target mapping, GDPR Article 4(5) |
| Kirkpatrick Level 1 reaction aggregation | R | Survey-response computation, deterministic |
| Kirkpatrick Level 2 learning pre/post tests | R | Score deltas with statistical significance, deterministic |
| Kirkpatrick Level 3 behaviour, 90-day on-the-job | R | Manager and peer feedback aggregation, deterministic |
| Kirkpatrick Level 4 and Phillips ROI Level 5 | R | Isolation-of-effects and monetary benefit, deterministic |
| Knowledge transfer, skill gap, content recommendation | A | ML transfer indicators with human validation |
| CSRD ESRS S1-13 training-and-skills-development | R | EFRAG datapoints, deterministic |
| ISO 30414 skills, leadership, culture metrics | R | Human-capital reporting categories, deterministic |
| US WIOA Joint Performance Indicators, Titles I-IV | R | Six indicators per title, deterministic |
| Registered apprenticeship 29 CFR 29 and RAPIDS | R | Apprentice progression and EEO, deterministic |
| UK Apprenticeship Levy and Service Account | R | Levy expenditure and funding rules, deterministic |
| ESCO and ISCED qualification-level mapping | R | Skills classification and alignment, deterministic |
| EEOC disparate impact, four-fifths rule | R | Statistical test across protected characteristics, deterministic |
| L&D Director and business sponsor approval | H | Four-eye SOC 2, interpretation mandatory |
Micro-Decision Table
Who decides in this agent?
14 decision steps, split by decider
Extract learning data from the LMS and reconcile it Are the learning records - xAPI/SCORM completions, assessment scores, survey responses, and enrolment data - extracted from the source LMS and learning platforms, pseudonymised under GDPR Article 4(5), and reconciled with a full audit-trail? Rules Engine
Learning data is extracted by deterministic source-to-target mapping from the LMS and xAPI/SCORM records, with reconciliation and pseudonymisation under GDPR Article 4(5), so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Aggregate Kirkpatrick Level 1 reaction surveys Are the Kirkpatrick Level 1 reaction survey responses - satisfaction, Net Promoter Score, content relevance, facilitator effectiveness, and course design - aggregated deterministically by cohort, course, business unit, and period, feeding the ISO 9001 training records and the ISO 30414 training-and-skills-development metric? Rules Engine
Kirkpatrick Level 1 reaction surveys are aggregated deterministically by cohort, course, and business unit, so the logic is rule-based (Decision-Type R), feeding the ISO 30414 and ESRS S1-13 datapoints.
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Calculate Kirkpatrick Level 2 learning from pre/post test deltas Are the Kirkpatrick Level 2 learning results - pre/post knowledge-test scores, skill assessments, competency demonstrations, and certification pass rates - calculated deterministically per cohort with statistical-significance testing, mapped to the ISO 30414 skills metric and the ESCO framework? Rules Engine
Kirkpatrick Level 2 learning is calculated deterministically from pre- and post-test score deltas with statistical-significance testing, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Aggregate Kirkpatrick Level 3 behaviour over the 90-day window Are the Kirkpatrick Level 3 behaviour signals - 90-day post-training on-the-job application, manager observations, peer feedback, self-assessment, and behaviour-change frequency - aggregated deterministically by cohort, business unit, and role family, feeding the ISO 30414 productivity metric? Rules Engine
Kirkpatrick Level 3 behaviour is aggregated deterministically over the 90-day post-training window from manager and peer feedback, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Calculate Kirkpatrick Level 4 results and the Phillips ROI Are the Kirkpatrick Level 4 business KPIs and the Phillips ROI Level 5 monetary benefit-cost ratio (revenue, cost reduction, quality, safety, customer satisfaction, retention) calculated deterministically per cohort, business unit, and programme using isolation-of-effects methodology, feeding the ESRS S1-13 and ISO 30414 datapoints? Rules Engine
Kirkpatrick Level 4 results and the Phillips ROI fifth level are calculated deterministically using isolation-of-effects methods (control group, trend line, forecasting, expert estimation), so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Generate knowledge-transfer and skill-gap indicators Are the knowledge-transfer indicators, skill-gap detection, content recommendations, and competency-progression signals generated as ML-supported indicators with confidence scores for the L&D Director dashboards, drawing on the LMS data and the learning-platform skills graphs? AI Agent Auditor
Knowledge-transfer indicators, skill-gap detection, and content recommendations are ML-supported on company-specific learning data, but the output is an indicator, not a final decision - the L&D Director and business sponsor interpret it, and there are no automated learning consequences for individuals. This sits within EU AI Act Annex III(4) and the EDPB HR-AI guidelines.
Decision Record
Challengeable: Yes - fully documented, reviewable by humans, objection via formal process.
Challengeable by: Auditor
Calculate the ESRS S1-13 training-and-skills-development datapoints Are the ESRS S1-13 training-and-skills-development datapoints - average training hours per employee, the share receiving regular performance and career-development reviews, and training participation across diversity dimensions - calculated deterministically for limited-assurance verification from 250 employees? Rules Engine
The CSRD ESRS S1-13 training-and-skills-development datapoints are calculated deterministically for limited-assurance audit, mandatory from 250 employees, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Calculate the ISO 30414 human-capital metrics Are the ISO 30414 human-capital metrics - training-and-skills-development (4.6), leadership (4.7), organisational culture (4.4), productivity (4.5), workforce composition (4.2), and costs (4.3) - calculated deterministically for stakeholder transparency reporting, aligned with the ISO 9001 competence and documentation clauses? Rules Engine
The ISO 30414 human-capital metrics, focused on training-and-skills-development and leadership, are calculated deterministically, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Calculate the US WIOA Joint Performance Indicators Are the US WIOA Joint Performance Indicators - second- and fourth-quarter post-exit employment, median earnings, credential attainment, measurable skill gains, and effectiveness in serving employers - calculated deterministically across Titles I-IV for the State Workforce Investment Boards, American Job Centers, and DOL ETA? Rules Engine
The US WIOA Joint Performance Indicators (employment, earnings, credential attainment, and measurable skill gains) are calculated deterministically across Titles I-IV for the workforce boards and DOL ETA, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Generate the US Registered Apprenticeship reporting Is the US Registered Apprenticeship reporting under DOL 29 CFR 29 and the 29 CFR 30 EEO rules - apprentice progression, completion and wage progression, and EEO compliance - generated deterministically into RAPIDS for the State Apprenticeship Agencies and DOL ETA? Rules Engine
Registered-apprenticeship reporting under DOL 29 CFR 29 and the 29 CFR 30 EEO rules - apprentice progression and completion submitted to RAPIDS - is generated deterministically, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Generate the UK Apprenticeship Levy reporting Is the UK Apprenticeship Levy reporting generated deterministically for HMRC and the ESFA - the 0.5 percent charge on a pay bill over GBP 3M, Service Account drawdown, the 24-month fund-expiry tracking, the 10 percent government top-up, and the End-Point Assessment outcomes against the Apprenticeship Standards? Rules Engine
UK Apprenticeship Levy reporting is deterministic - the 0.5 percent payroll calculation, Service Account drawdown, 24-month expiry alerts, and the 10 percent government top-up - so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Map the learning catalogue to ESCO and ISCED Is the ESCO classification (around 14,000 occupations, 13,000 skills, and 10,000 qualifications) and the UNESCO ISCED qualification levels and fields applied deterministically to the learning catalogue, skills profiles, and qualification frameworks? Rules Engine
Mapping the learning catalogue to the ESCO skills classification and ISCED qualification levels runs on a deterministic engine, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Run the EEOC disparate-impact check on training selection Is a disparate-impact analysis run on training selection - the EEOC Uniform Guidelines four-fifths rule with statistical-significance testing across protected-characteristic participation rates under Title VII, the ADA, and the ADEA - calculated deterministically for compliance verification? Rules Engine
The EEOC disparate-impact analysis on training selection applies the Uniform Guidelines four-fifths rule as a statistical test across protected characteristics, so the logic is rule-based (Decision-Type R).
Decision Record
Challengeable: Yes - rule application verifiable. Objection possible for incorrect data or wrong rule version.
Approve the effectiveness report under a four-eye principle Is the training-effectiveness report approved by the L&D Director, business sponsor, CHRO, and ESG Officer under a four-eye principle - with commentary and interpretation, works-council consultation where it applies, a SOC 2 Type II audit-trail, and the ESG limited-assurance auditor sign-off? Human
The effectiveness report needs human sign-off - the L&D Director, business sponsor, CHRO, and ESG Officer in a four-eye principle - for a SOC-2-grade audit-trail and the ESG limited-assurance review, with works-council consultation where it applies. Interpretation and commentary stay human, so the decision is mandatory-human (Decision-Type H).
Decision Record
Challengeable: Yes - via manager, works council, or formal objection process.
Decision Record and Right to Challenge
Every decision this agent makes or prepares is documented in a complete decision record. Affected employees can review, understand, and challenge every individual decision.
Does this agent fit your process?
We analyse your specific HR process and show how this agent fits into your system landscape. 30 minutes, no preparation needed.
Analyse your processGovernance Notes
Assessment
Prerequisites
- LMS data export with structured ETL + xAPI Learning Record Store LRS + SCORM 1.2/2004 + reconciliation + pseudonymisation per EU GDPR Article 4(5)
- Kirkpatrick 4-Levels measurement framework + Level 1 Reaction surveys + Level 2 Learning assessments + Level 3 Behavior observations + Level 4 Results business KPIs
- Phillips ROI 5-Levels measurement framework + Level 5 monetary benefit cost ratio + isolation-of-effects methodology
- ESCO European Skills Competences Qualifications and Occupations mapping + UNESCO ISCED qualification levels + EU EQF alignment
- ISO 30414 training-and-skills-development metric + ISO 9001 7.2 Competence + 7.3 Awareness + 7.5 Documented Information records
- ESG/CSRD ESRS S1-13 Skills Reporting training-and-skills-development datapoint + EFRAG datapoint Excel + auditor limited-assurance scope
- US WIOA Joint Performance Indicators + Title I-IV reporting + State Workforce Investment Boards + American Job Centers integration
- US DOL Office of Apprenticeship Registered Apprenticeship 29 CFR 29 + 29 CFR 30 EEO + RAPIDS submission + State Apprenticeship Agencies
- UK Apprenticeship Levy reporting engine + Apprenticeship Service Account + 24 month expiry tracking + 10 percent top-up + Apprenticeship Funding Rules
- UK Ofqual End-Point Assessment Organisations register + Apprenticeship Standards + Institute for Apprenticeships and Technical Education
- EU GDPR Article 6 lawful basis + Article 88 employment + works council training co-determination + DPIA per Article 35
- AICPA SOC 2 Type II audit framework + ISAE 3000 + ISAE 3402 + Big-4 limited assurance ESG capability
Infrastructure Contribution
What this assessment contains: 9 slides for your leadership team
Personalised with your numbers. Generated in 2 minutes directly in your browser. No upload, no login.
- 1
Title slide - Process name, decision points, automation potential
- 2
Executive summary - FTE freed, cost per transaction before/after, break-even date, cost of waiting
- 3
Current state - Transaction volume, error costs, growth scenario with FTE comparison
- 4
Solution architecture - Human - rules engine - AI agent with specific decision points
- 5
Governance - EU AI Act, works council, audit trail - with traffic light status
- 6
Risk analysis - 5 risks with likelihood, impact and mitigation
- 7
Roadmap - 3-phase plan with concrete calendar dates and Go/No-Go
- 8
Business case - 3-scenario comparison (do nothing/hire/automate) plus 3×3 sensitivity matrix
- 9
Discussion proposal - Concrete next steps with timeline and responsibilities
Includes: 3-scenario comparison
Do nothing vs. new hire vs. automation - with your salary level, your error rate and your growth plan. The one slide your CFO wants to see first.
Show calculation methodology
Hourly rate: Annual salary (your input) × 1.3 employer burden ÷ 1,720 annual work hours
Savings: Transactions × 12 × automation rate × minutes/transaction × hourly rate × economic factor
Quality ROI: Error reduction × transactions × 12 × EUR 260/error (APQC Open Standards Benchmarking)
FTE: Saved hours ÷ 1,720 annual work hours
Break-Even: Benchmark investment ÷ monthly combined savings (efficiency + quality)
New hire: Annual salary × 1.3 + EUR 12,000 recruiting per FTE
All data stays in your browser. Nothing is transmitted to any server.
Training Effectiveness Agent
Initial assessment for your leadership team
A thorough initial assessment in 2 minutes - with your numbers, your risk profile and industry benchmarks. No vendor logo, no sales pitch.
All data stays in your browser. Nothing is transmitted.
Related Pages
Agent Blueprint Available
A full blueprint for Training Effectiveness Agent is available with micro-decision decomposition, industry variants, and implementation details.
View BlueprintRelated Agents
Certification Tracking Agent
One auditable pipeline for mandatory training, expiring certificates and renewal across every regime that demands them - US OSHA, BSA/AML and HIPAA, UK health-and-safety and professional revalidation, the EU Framework Directive and ISO 45001 - so a missed renewal stops a task assignment before it becomes a violation.
Frequently Asked Questions
Does the agent make autonomous learning decisions about individual employees?
Why is this agent NOT an EU AI Act high-risk system?
How is UK Apprenticeship Levy reporting and the 24-month fund expiry handled?
How is the US WIOA performance and Registered Apprenticeship reporting handled?
How are GDPR Article 88 and works-council training co-determination handled?
How does the Phillips ROI fifth level and isolation-of-effects work without crossing into high-risk?
What cross-references to other HR agents exist?
What Happens Next?
30 minutes
Initial call
We analyse your process and identify the optimal starting point.
1 week
Discover
Mapping your decision logic. Rule sets documented, Decision Layer designed.
3-4 weeks
Build
Production agent in your infrastructure. Governance, audit trail, cert-ready from day 1.
12-18 months
Self-sufficient
Full access to source code, prompts and rule versions. No vendor lock-in.
Implement This Agent?
We assess your process landscape and show how this agent fits into your infrastructure.