WIKIPEDIA SOURCE ACQUISITION — JOB 124

Scope
This is a selected article corpus, not a download or inspection of all Wikipedia.
The English Wikipedia Vital Articles Level 4 lists provide coverage leads, not a
ranking of truth, a guarantee of encyclopedic importance, or Carrier approval.

Method
The Wikimedia Action API supplies batched current revision IDs and exact full
wikitext. Each selected revision is then rendered through action=parse&oldid=ID.
The returned revision ID must equal the retained source revision. No later live
article extract is silently substituted for the selected revision.

The oldid pins the article wikitext, not every template or other transclusion's
historical version. Rendered HTML is the exact retained capture produced from
that article revision at retrieval time. It is not asserted to reconstruct all
historical dependencies. Source-unit hashes pin the reading actually reviewed.
For the initial main acquisition, retrievedUTC records the batch query time;
HTML parsing followed sequentially afterward. Later patched acquisitions record
per-article completion time. Neither timestamp is presented as the exact time
of every upstream cache or transcluded template; the retained hashes identify
the actual captured bytes.
Aliases reuse an existing verified capture when the wikitext hash agrees;
earlier acquisition processes could fetch the same revision more than once.
The complete catalog/file audit checks that every declared capture hash still
matches. It does not silently replace a mismatched hash to make a check pass.

Requests use an identifying User-Agent, maxlag=5, one second of spacing, and
bounded exponential retry. Successful article acquisition is saved immediately
and is resumable. Failed requests, if any, remain in acquisition-failures.jsonl.

Each retained article has full wikitext, rendered HTML, raw parse response, a
readable paragraph/list extraction, its source-unit index, and attribution.
The source index describes half-open UTF-8 byte spans in reading.txt, not byte
spans in HTML or wikitext. Its text hash excludes the terminating line break.

Extraction boundaries
The readable extraction joins layout whitespace, removes citation superscripts,
edit controls and navigation, omits tables/figures, and omits named reference or
end-matter sections. Formula typography and embedded visual information are not
fully represented by plain text. Full source wikitext and HTML remain available
for checking these losses. A retained article is not automatically a fully
reviewed article. Candidate review and the selected supporting source units are
recorded separately; no inspection count should be inferred from download count.

Provenance and licensing
Attribution: Wikipedia contributors, with each article's canonical page,
permanent revision URL and contributor-history URL retained in articles.json.
Wikipedia text is available under CC BY-SA 4.0 unless otherwise stated:
https://creativecommons.org/licenses/by-sa/4.0/
https://en.wikipedia.org/wiki/Wikipedia:Reusing_Wikipedia_content

The extracted reading is modified as described above. New Wikipedia-derived
Carrier definitions are adaptations; their source attribution and applicable
CC BY-SA 4.0 notice must travel with the new subset and its exports. This does
not retroactively set the license of unrelated pre-existing Carrier records.
Article images are not downloaded; image/media and separately quoted material
may have separate rights and are not automatically covered by a text license.

Why not the full database
The dated enwiki 20261001 dump reports the multistream jobs complete on Oct 2,
2026. Its status is retained in dumpstatus-20261001.json. A current English
multistream dump is about 26.9 GB compressed and more than 100 GB expanded.
Exact-revision supported API requests provide the required selected corpus with
far less local storage. The latest directory is not treated as an immutable
snapshot; per-article revision IDs are the actual provenance pins here.
