Point-in-time legal data: why version history matters for legal AI evals

Legal AI evals need point-in-time data because the law they test keeps changing. An act amended after the eval was written makes its gold answer wrong, and a retrieval corpus that holds only the latest text lets a model answer a question about March 2025 with an amendment published in June 2025. Point-in-time data records what a source said at each moment, so an eval can restrict both the retrieval corpus and the gold answers to what existed on a chosen date. That requires a version history with timestamps, and a version history only exists from the day someone starts recording it. Arieta Data’s history dataset is that record for EU and national legislation and CJEU case law.

What is point-in-time legal data?

Point-in-time legal data is a collection in which each document is stored as a sequence of versions, each with the time it was valid or observed, so that any query can be answered “as of” a date. In finance the same idea prevents look-ahead bias in backtests. In legal AI it prevents a model or eval from using law that did not yet exist, or no longer applied, at the date the question is about.

Two kinds of point-in-time record exist, and they are easy to confuse:

Kind	What it records	Example
In-force reconstruction	The legal text applicable on a date, as a lawyer would determine it	An EU consolidated text such as 02016R0679-20160504, the GDPR as consolidated at 4 May 2016
Observation snapshot	The text a pipeline saw at the source at a given time	Version 3 of a document, fetched on a recorded date and identified by its content hash

Arieta Data versions are observation snapshots. They say what the source published when Arieta’s pipeline fetched it, not which provisions were legally in force. For in-force text, EUR-Lex consolidated versions remain the reference, and they are for information only: the Official Journal is the authentic publication.

How does stale legal text break an eval?

Stale or undated legal text breaks evals in at least five distinct ways. Each one produces scores that look plausible and are wrong.

Failure mode	What happens	What fixes it
Future leakage	The retrieval corpus contains an amendment adopted after the question’s date, so the model “knows” the later rule	Restrict retrieval to versions observed on or before the question date
Stale gold answer	An article is amended after the eval is written; the old gold answer is now wrong for “current law” questions	Pin each item to a date and a version; re-derive gold answers from the new version when you move the date
Repealed text served as current	A RAG system retrieves a repealed act because nothing marks it as repealed	Join the corpus with typed repeal and amendment edges from a citation graph
Training contamination	A model trained on a later snapshot is evaluated on questions whose answers changed in that snapshot	Hold out a later snapshot and evaluate on items derived from it
Silent source edits	A source corrects or replaces a text without a new identifier, so two runs of the “same” eval use different text	Content hashes per version, so a change is visible as a new version

The general point is well known in ML: evaluation data has to respect the time boundary of the training data. Legal text makes it sharper because the “correct” answer depends on the date. A question about the information duties under an EU regulation has one right answer before an amending act applies and another after it.

What does EU law look like over time?

EU acts change through amending acts, corrigenda and repeals, each published in the Official Journal under its own CELEX number. The Publications Office then produces consolidated versions that merge those changes into the base act, identified by a CELEX number starting with 0 and ending in the consolidation date. Heavily amended acts such as the CLP Regulation, Regulation (EC) No 1272/2008, have a long chain of amending acts and consolidations.

Three dates describe a legal document, and an observation timestamp adds a fourth:

Date	Meaning	Arieta Data field
Date of document	Adoption or decision date	dateOfDocument
Date of effect	Entry into force or effect, where known	dateOfEffect
Date of publication	Publication in the official journal or gazette	dateOfPublication
Observation time	When this version was fetched from the source	fetchedAt (history dataset)

A point-in-time eval uses the observation time to choose which version of a text a system may see, and the legal dates to decide which version is the right answer.

How is Arieta Data’s version history built?

Arieta Data records a new version whenever the SHA-256 hash of a document’s normalised parsed text changes between fetches. Hashing parsed text, not the upstream web page, means that menu or banner changes never create versions, while every real text change does. Each version record in the history dataset carries the document id, language, version number, content hash, fetchedAt timestamp, title, article count, a short diffSummary and the full article text.

The selection rule is simple and stated in the data: version N is current from its own fetchedAt until the fetchedAt of version N+1. To reconstruct what the corpus held on a date, take, for each document and language, the latest version whose fetchedAt is on or before that date.

The limits are stated just as plainly:

1. History starts at first ingestion. A document’s history begins when it first entered the corpus. Changes made upstream before that cannot be backfilled.
2. Observation is not enactment. A version’s fetchedAt is when the change was seen, which can be later than when the source published it.
3. Parser upgrades can change hashes. When parsing improves, the current version can be re-parsed with a new content hash. The diffSummary says so when that is the cause.
4. Not a legal reconstruction. Use EU consolidated versions, or a lawyer, to determine the law in force.

Because history cannot be recreated after the fact, its value grows with time. A team that starts scraping today has no record of what sources said last year. See the snapshot glossary entry for how monthly snapshots are cut.

How do you build a point-in-time legal eval?

A point-in-time legal eval fixes a date for every item and enforces it on both the retrieval corpus and the gold answer. A workable recipe:

1. Choose an as-of date per item. Store it with the question.
2. Freeze the corpus view. For that date, select the latest observed version of each document. Index only those versions for retrieval.
3. Derive the gold answer from the frozen view. Record the document id, version number and content hash the answer depends on.
4. Add change-sensitive items. Pick articles that changed between two versions and ask the same question at a date before and after the change. A system that ignores dates fails one of the pair.
5. Hold out a later snapshot. Train or tune on an earlier snapshot and test on items derived from changes in a later one, to measure how a system handles law it has not seen.
6. Re-run on every delta. Monthly delta snapshots tell you which documents changed, so you know which items to re-check.

The citation graph and the history together give you gold sets; the full method is in How to build a legal eval set from a citation graph. For RAG systems, the same dates belong on every chunk, as described in How to chunk legal documents for RAG.

Diff supervision is a by-product: pairs of consecutive versions, with their diffSummary, are training data for amendment-aware models that explain what changed.

Key facts

• An eval built on the latest text leaks later amendments into earlier questions and leaves gold answers stale.
• In-force reconstructions (such as EU consolidated texts) and observation snapshots are different records; Arieta Data’s versions are observation snapshots.
• Arieta Data records a new version whenever the SHA-256 hash of a document’s normalised parsed text changes; version N is current from its fetchedAt until version N+1’s.
• Snapshot 2026-11 holds 15,098 versions, 14,204 of them in the EU lane; history starts at a document’s first ingestion and cannot be backfilled.
• Only the Official Journal is authentic for EU legislation; consolidated versions are for information only.

Frequently asked questions

What is the difference between a consolidated text and a version history?

A consolidated text is an in-force reconstruction of an act at a date, produced by a publisher. A version history records what a source published each time it was observed. Use consolidated texts to determine the law; use observation history to control what a system could have seen.

How do I stop future law leaking into a legal eval?

Give every item an as-of date, index only the versions observed on or before that date, and derive the gold answer from those versions. Hold out a later snapshot to test on changes the system has not seen.

Can version history be rebuilt later?

Official sites publish some historical and consolidated versions, but not a record of what they showed on a given day. An observation record exists only from the day someone starts keeping it.

Where Arieta Data fits

Arieta Data ships its version history as a separate product next to the corpus and the citation graph, all cut from the same monthly snapshot. Eval sets with point-in-time answers and held-out snapshots are built to your spec on request. Every snapshot covers EU legislation, CJEU case law and national legislation; national case law is included, delivered with personal data masked. Snapshot 2026-11 holds 15,098 versions across 419,355 documents; per-lane counts are on the coverage page and terms on the licensing page. Request access for a sample: we reply within one business day, and a custom cut is ready within 5⁠–⁠6 business days. For the legal sourcing behind the EU lane, read Is EUR-Lex data licensable for AI training?; for using relations between documents as training signal, read Citation graphs as supervision signal for legal retrieval.

Not legal advice. Arieta Data versions are observation snapshots, not legal in-force reconstructions.