Floor 4
Data & API
The building is data first. Every room in the Catalogue is a record with the same fields and the same provenance discipline.
Record fields
title · author · author_id · sphere · tradition · text_type · language · year (author-derived) · word_count · sections · source_repo · source_url · date_basis
5,924 works currently catalogued. No public API is available right now; contact us if you need API access for a research or institutional use case.
The pipeline — wisdomlit.py
Every text in this building enters through wisdomlit.py, an in-house ingestion pipeline that clones source repositories, matches authors against a persistent registry, and exports each work as Markdown with provenance frontmatter.
Each repository has its own structure, conventions, and encoding format — TEI XML from Perseus, plain-text org-mode from Kanripo, JSON from SuttaCentral, HTML from OpenITI. wisdomlit.py applies custom parsing rules per repository, extracting sections, book divisions, and metadata fields. This requires engaging with each source individually: understanding its schema, handling its edge cases, and testing against real data.
This is work in progress. Every repository needs attention and care — some are well-structured and parse cleanly, others carry OCR artifacts, mixed commentary, or ambiguous section boundaries. The Source Archive documents the current status of each source.
Reading with AI
Every manuscript room includes a chat panel that lets you ask questions about the text you are reading — using your own machine, your own model, with nothing sent to a remote server.
To enable it, you need a mid-range computer or above, and familiarity with LM Studio or Ollama. The site automatically detects a running local model and connects to it. Decent results start at around 12B parameter models; smaller models work but struggle with complex philological questions.
At present, chatting with Chinese texts yields the best results — modern models handle Classical Chinese (文言文) with surprising fluency. Support for Ancient Greek, Sanskrit, Arabic, and other historical languages is improving rapidly, and we anticipate substantial gains as model capabilities continue to develop. The chat is a reading aid, never an authority: it can orient and suggest, but the source text and its edition remain the reference.