New open standard aims to record how students really work with AI

Every time a student opens a chatbot to work on an assignment, a rich trail of evidence is generated: the questions asked, the answers accepted or challenged, the revisions made, the moments of genuine intellectual struggle. Yet almost none of that evidence survives into the records that universities actually keep. A student who delegates an entire AI-assisted essay to a language model and a student who interrogates every response the model produces leave exactly the same trace in a learning management system — course completion, a quiz score, minutes spent on task. A newly published open specification called the Standard Learning Provenance Taxonomy, or SLPT, wants to change that, and it arrives with an unusually candid warning about what it can and cannot prove.

SLPT, described in the journal SoftwareX by Trond Arne Undheim, M. Qaiser Malik and Nada Hashmi, is not a test, not a grading algorithm and not a surveillance tool, at least not by design. It is a data contract: a formally specified, machine-readable structure called the Learning Provenance Record, defined by a JSON Schema and accompanied by a conformance validator, a corpus of test fixtures and an implementer guide, all released under open licenses. The current release, version 1.2.2, is archived on Zenodo and hosted on GitHub, where any institution or third-party developer can validate records against the published schema without touching the authors’ commercial scoring service. The specification text itself is licensed under Creative Commons BY 4.0, while the validator and conformance corpus carry an Apache 2.0 license.

The motivation is a gap the authors document across the assessment literature. Existing AI-literacy instruments fall into two camps: validated self-report questionnaires such as MAILS and SNAIL, which measure what learners believe about their own competence, and objective knowledge tests such as AICOS, GLAT and AILIT-S, which measure what learners know about AI. Neither class captures behaviour with AI during real coursework. The authors report that a literature search spanning English, Chinese, Japanese, Korean and Arabic sources, including a 69-study Chinese systematic review, identified no peer-reviewed, psychometrically validated instrument whose score derives directly from authentic learner–AI interaction traces. SLPT is designed to fill that evidence-capture gap — while explicitly declining to claim that trace-derived scores are valid measures of anything yet.

At the heart of the specification are five judgment-centric dimensions, distilled from twelve candidates over four internal design rounds against criteria of parsimony, stability and observable evidence. Judgment Quality is treated as a candidate second-order composite whose evidence is distributed across the other four. Question Originality reads the learner’s prompts before any model output exists, drawing on research into creative prompt engineering. Source and Output Discernment captures how the learner disposes of model output — a dimension directly motivated by the Harvard Business School field experiment by Dell’Acqua and colleagues, which found a jagged technological frontier where AI boosted consultant performance inside it but increased errors outside it. Synthesis Under Ambiguity requires conflicting or low-confidence output as a stimulus condition, echoing King and Kitchener’s Reflective Judgment Model. Adaptability, defined strictly as revision behaviour under corrective feedback and never scored on elapsed time, deliberately inverts the ‘metacognitive laziness’ construct documented by Fan and colleagues.

The most technically interesting departure is the delegation-pattern annotation. Bloom’s taxonomy maps a CREATE-level query to the highest cognitive level, but a prompt asking a model to create something may reflect cognitive delegation rather than human cognition — the two are indistinguishable in standard mappings. The authors call this Bloom inversion, and they found the risk not formalized in any surveyed AI-literacy or learning-analytics framework. Version 1.0 of the specification responded with a Boolean flag and a recommended minimum 20 percent score reduction; version 1.2.2 withdraws both, on the grounds that the penalty had no empirical, consensual or theoretical basis. The replacement is an episode-level annotation that records its own evidence basis, a confidence value and an explicit not_assessable state — which, the authors note, most single episodes will carry — evaluated against an instructor-declared task frame. Crucially, the annotation never modifies dimension estimates; it travels alongside them as annotated evidence.

Privacy is engineered into the record rather than bolted on. Query text is hashed with SHA-256 before persistence, so plaintext never enters the record, and a conformance fixture asserts that any record carrying plaintext query text is rejected at the platform boundary. The specification requires deploying platforms to provide automatic PII detection, administrator alerts, configurable institutional policies and an audit log. The authors are careful about the legal framing: hashing may support a data-minimisation posture under FERPA, but does not establish FERPA compliance, which depends on the deployment as a whole. Similarly, the record structure may support logging duties under the EU AI Act, which classifies AI systems used to evaluate learning outcomes as high-risk under Annex III, but schema conformance does not establish legal compliance.

The conformance architecture is where the specification’s use restrictions become machine-checkable rather than advisory. Six cross-field rules address constraints that per-field validation cannot express: a record could declare credentialing use and an authorized status simultaneously while remaining structurally valid. Rule R1 permits summative or credentialing use to be authorized only where populated validation evidence is attached. Rule R5 gates the credential_tier field in both directions — a non-null tier requires credentialing intent, conditional or full authorization, and populated validation evidence. Rule R6 requires that where a learner’s dispute is upheld, the original record be marked corrected or invalidated, because a reassessment does not correct the original inference. The release even ships a probe script born of a subtle bug: from v1.1.0, one rule constrained a task-frame field’s type without requiring the property, so a record omitting the frame entirely validated with zero errors. Null, the authors observe, is not absence.

The conformance corpus itself is a model of reproducibility: 71 fixtures, 17 positive and 54 negative, each negative case constructed from a conformant base record by introducing exactly one targeted violation. Executed inside the archived v1.2.2 release under Python 3.12 with the schema library, all 71 cases reproduced their expected outcomes, and a machine-generated manifest connects each case to the constraint it tests. The authors are precise about what this demonstrates: the specification is implementable and conformance is machine-checkable. It does not demonstrate that recorded values measure learning. Following validity theory’s distinction between interpretation and use, the release separates four claims — schema conformance, measurement validity, educational interpretation and decision use — and demonstrates only the first. Because weighting is configured per institution and disclosed only to that institution, estimates produced under different configurations are not comparable across institutions, a prohibition that travels with the record wherever it is ported.

The governance sections confront problems that a data format alone cannot solve. Consent, the authors admit, is not freely given where the scored activity is required coursework; the specification now requires institutions either to offer an equivalent unscored route to the learning objective or to document a necessity-and-proportionality basis. Appeal is allocated rather than left to the operator: the deploying institution is the authority of last resort for disputes with academic consequences, and an appeal path in which the operator judges defects in its own scorer does not satisfy the specification. Retention is conditional on the dispute rather than the clock, so evidence cannot vanish mid-adjudication. And because learners can optimize any signal they see, the specification discourages real-time learner-visible estimates, warning that a provenance system can induce prompt inflation and displacement of work to unmonitored tools — degrading the very signal it records.

What emerges is a rare kind of publication: infrastructure that advertises its own limits. Permitted uses at this release are structural validation, interoperability research, within-learner formative feedback and ethically reviewed research; automated summative grade passback, cross-learner ranking, progression decisions, employment use and automated accusations of prohibited AI use are all unsupported, and the reference connector demonstrably rejects gradebook passback for records whose use is prohibited or unvalidated. Deployment evidence remains single-institution, drawn from a Babson College deployment, and the authors state plainly that no calibration baseline exists and that fairness testing across language background, disability and technical fluency is a precondition of any summative use. Whether SLPT becomes the common provenance layer for AI-mediated learning will depend on independent implementers, who are explicitly invited — and on whether the measurement validity the specification withholds can eventually be earned.

Subject of Research: An open data specification and JSON schema for recording and validating learner–AI interaction provenance in education

Article Title: Standard learning provenance taxonomy (SLPT): An open specification and JSON schema for learner–AI interaction records

Article References: Undheim, T. A., Malik, M. Q., & Hashmi, N. (2026). Standard learning provenance taxonomy (SLPT): An open specification and JSON schema for learner–AI interaction records. SoftwareX, 36, Article 103125. https://doi.org/10.1016/j.softx.2026.103125

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103125

Keywords: SLPT, learning provenance, AI literacy, JSON Schema, educational assessment, generative AI, higher education, Bloom's taxonomy, EU AI Act, learning analytics, data privacy, conformance testing