Maintenance Audit Snapshot¶

中文页面

status: active

owner: market-data-platform

audience: human and agent

last_verified: 2026-09-30

source_of_truth: yes

superseded_by: n/a

This page summarizes repository responsibilities, accepted maintenance debt, and refactoring priorities based on governance scripts and repository-local inspection. Detailed current inventories are generated by the scripts referenced below; do not treat dated snapshots in this page as live metrics.

Responsibilities¶

The repository maintains the mainland-China market-data control plane:

Data contracts, path conventions, asset keys, manifests, dataset registry, and current contracts.
TuShare source access for the mainland-China market.
Guan and TuShare A-share minute construction, coverage audits, and atomic alias promotion.
Standardized catalogs, materialization, DuckDB queries, and local snapshot backups.
The unified marketdata CLI and retained archive/migration interfaces.

Hong Kong provider production modules, tick/depth modules, publication presets, transitional schedulers, and historical import wrappers were removed with RQData on 2026-07-26. Historical reproduction uses tag hk-freeze-20260613 or the private archive repository; the recovery page is retired guidance only.

Lifecycle classification¶

Scope	Classification	Maintenance policy
`contract.py`, `paths.py`, `manifest.py`, `registry.py`, `data_provider_contracts.py`	Active	Core platform boundary; retain Ruff and `ty` coverage
`providers/*`, `tushare_cli.py`	Active	Provider adapters and CLI parser; continue expanding type coverage
`cold_storage.py`	Removed	Deleted with RQData retirement on 2026-07-26
`scripts/dev/*`	Active governance	Local quality gates; changes require corresponding tests
`artifacts/`, `reports/`, pytest/Ruff caches, `*.egg-info`	Generated/cache	Not maintained source and must not be committed

Quality coverage and debt¶

After Hong Kong production directories were removed, Ruff and ty directory excludes were narrowed to non-source paths. Large-file excludes were removed from the core refactoring area; data_providers.py and data_warehouse.py now act as thin aggregation entry points over smaller implementations. artifacts.py is authoritative for artifacts-root resolution; paths.py retains path contracts and compatibility entry points. The default root order is explicit argument, DATA_PLATFORM_ROOT, then artifacts/. DATA_PLATFORM_METADATA_DB_PATH and DATA_PLATFORM_WAREHOUSE_DB_PATH independently override the catalog and warehouse databases.

The checked-in scripts/dev/quality_baseline.json was generated on 2026-09-05 and records 290/290 source files and 65,708/65,708 lines for both Ruff and ty, with zero excluded files or lines. It also records the complexity ratchet. This is a governance baseline, not a live size report. Run scripts/dev/quality_debt.py for current coverage and scripts/dev/maintainability_metrics.py for current code-shape metrics.

The maintainability baseline was also generated on 2026-09-05. At that snapshot it recorded 421 Python files, 105,016 lines, 38 functions over 100 lines, one over 250 lines, none over 500 lines, and 10 functions with at least 10 arguments. Its largest function was the 309-line _fixture in tests/test_cutover_a_share_minute.py. Read scripts/dev/maintainability_baseline.json for its complete dated inventory; do not copy old hotspot paths into new guidance.

At the 2026-09-30 audit, the current maintainability script reported 455 Python files and 109,774 lines, 38 functions over 100 lines, one over 250 lines, none over 500 lines, 10 functions with at least 10 arguments, a 2,045-line largest file, and a 309-line largest function. These live metrics are generated output, not a checked-in baseline.

Routine gates:

uv run --extra dev python scripts/dev/run_pytest_isolated.py -- -q
uv run --extra dev python -m ruff check .
uv run --extra dev python -m ruff format --check .
uv run --extra dev ty check --error-on-warning

Governance gates:

uv run --extra dev python scripts/dev/quality_debt.py --skip-ruff --complexity \
  --check-baseline --check-ratchet
uv run --extra dev python scripts/dev/maintainability_metrics.py --check-baseline
uv run --extra dev python scripts/dev/compatibility_governance.py --check
uv run --extra dev python scripts/dev/architecture_governance.py --check

The current complexity ratchet honors registered inline noqa suppressions. The checked-in quality_baseline.json records two unexempted C901 findings; increases in unexempted category counts or their total block the local push gate. A full --ignore-noqa inventory is historical context and should be regenerated before decisions.

Dated baseline changes¶

On 2026-09-27, the shared code-quality baseline was adjusted for the Knowledge v2 dataset pilot: knowledge_index.py, its development validator, and contract tests added strict parsing and synthetic coverage. Later 2026-09-28 updates added dataset-targeted constraint-reference publication and research-grade ST receipt/hash/row-consistency checks. On 2026-09-29, an ST event-date audit and raw reference receipt added repeatable commands; the recorded exit condition is to consolidate these checks into the existing ST quality pipeline and remove redundant standalone code. Until announcement-time evidence exists, revision_safe=false remains required for reconstructed data.

On 2026-09-30, bilingual documentation regression coverage added one Python file and 148 lines. It checks that README and MkDocs pages use English canonical pages, Chinese companions link back, and configured navigation targets remain English. The shared code-quality baseline increased from 454 files / 109,626 lines to 455 files / 109,774 lines. Complexity and long-function budgets did not increase. Keep this test while pages are maintained as file pairs. If the repository adopts a centralized i18n route map, fold these assertions into that contract test and recalculate the baseline.

On 2026-10-01, the ST event availability contract added 284 Python lines for conservative exchange-session availability, daily_clean.v2 propagation, and regression checks for receipts and row consistency. The shared baseline increased from 109,774 to 110,058 lines; file count, complexity, long-function, and long-line budgets did not increase. Once providers and consumers complete the versioned migration, assess consolidating duplicate checks into the existing ST quality pipeline and tighten the line baseline after removing equivalent redundant code. Reconstructed history remains revision_safe=false.

The research views now expose daily st_available_from to A-share and DailyWatch20 consumers. Both readers require a daily_clean manifest declaring schema v2 and the ST availability contract, so DuckDB cannot silently fill a missing field from an older partition with nulls. The shared baseline increased from 455 files and 110,081 lines to 456 files and 110,174 lines for the shared manifest validator and regression tests. Complexity, long-function, and long-line budgets did not increase. After consumers complete migration and published assets are confirmed to use v2, assess whether the contract check can move to a higher shared layer.

See the dated Chinese audit record for the exact historical line-count changes. Those figures describe specific commits and are not current size targets.

English operations-link regression budget — 2026-10-15¶

The English operations index adds 13 regression-test lines to ensure four topic links target English canonical pages and the A-share minute guide keeps its explicit link label. The shared python_lines baseline increases from 115,068 to the measured 115,081. File count, complexity, long-function, long-line, and exclusion limits remain unchanged. Recalculate and tighten this allowance if the assertions are consolidated into the existing documentation contract checks without losing English-target validation.

Compatibility decisions¶

The hkdata console script, hk_data_platform.* import aliases, rqdata-hk-depth, rqdata-tick, rqdata-hk-assets, and retired marketdata migration commands were removed from the active package. The former freeze-hk/hydrate-hk interfaces were removed with RQData; the linked recovery note is retained as historical guidance.

Downstream research-repository data-directory and snapshot wrappers have been removed. Use marketdata data ... and marketdata backup-data for data operations, and strategy ... for strategy operations.

Generated files and artifacts¶

.gitignore covers .venv/, test/Ruff caches, __pycache__/, bytecode, *.egg-info, artifacts/, data/, reports/, and local credentials. Local outputs such as artifacts/metadata/*.csv, artifacts/reports/*.json, src/*.egg-info, and caches do not belong in Git.

Documentation and CI¶

The root README, AGENTS.md, and user-facing docs/ are expected to reflect current behavior. Active entry points are marketdata and market_data_platform; compatibility and retirement notes live in docs/compatibility.md. Local gates and GitHub Actions jointly form the quality boundary: quality.yml checks architecture/boundaries, maintainability ratchets, Ruff, pytest, and dependency auditing; docs.yml runs strict documentation builds. The sdist includes docs/ and tests/; the wheel contains the runtime package only. The package does not currently publish py.typed.

Refactoring priorities¶

The recorded next steps are to split the daily executor and checkpoint/receipt finalizer in tushare_a_share_mins.py::mirror_minute_bars; break the monolithic minute-cutover test fixture into raw/full-day/Beijing receipt builders; continue reducing long minute-materialization and provider functions; further separate provider SDK adapters from frame-processing pipelines; and continue breaking down pandas-heavy data_warehouse_* modules while evaluating stricter per-module typing. RQData and the Hong Kong recovery control plane were retired on 2026-07-26 and do not need restoration drills.

QuantZone configuration size baseline — 2026-10-02¶

The shared size baseline now records 472 Python files and 112,768 lines after adding the private JSON loader/launcher, optional QuantZone planner/adapter/artifact modules and their tests. This update accounts for the intentional feature addition. Complexity, long-function, large-file and lint-exclusion thresholds remain unchanged. There is no extra size allowance beyond the measured snapshot. Future growth requires an explained feature change or refactoring. Remove the corresponding size allowance if these modules are retired.

Configuration source split size baseline — 2026-10-02¶

The shared size baseline accounts for the configuration-source module, its regression tests, and job-snapshot verification. Only the exact measured Python file and line totals change; all complexity, debt, long-function and large-file thresholds remain unchanged. Recalculate the size baseline if the legacy loader or split-source modules are retired or consolidated. This allowance does not authorize unrelated growth.

Temporary setuptools update procedure — 2026-10-07¶

Keep setuptools current through a scoped lockfile PR while Dependabot's uv updater cannot handle its combined build-system and transitive dependency roles. Upstream issue #14118 and proposed fix #14421 track the updater defect. The fix was still unmerged when this procedure was recorded.

Check for new releases during regular dependency maintenance and whenever a security alert affects setuptools. In a clean task worktree, run:

uv lock --upgrade-package setuptools
uv sync --locked --extra dev

Inspect the lock diff for setuptools-only changes, then run the local quality and dependency audit gates. If another package changes, explain and validate that dependency change separately. Open a PR targeting main and merge only after required checks pass. A lock update does not promote production or replace a live environment.

Leave ordinary Dependabot updates and security alerts enabled. After GitHub deploys the upstream fix, verify that a fresh setuptools update job successfully changes uv.lock before retiring this manual procedure. Upstream PR merge alone is not evidence that the hosted updater has deployed the fix. The October update replaces locked setuptools 83.0.0 with 84.0.0 without adding an ignore rule or changing the build requirement.

2026-10-19 statement template diagnostic baseline¶

The shared baseline increases python_files from 496 to 498 and python_lines from 117363 to 117565. The net 202 lines support a standalone retained-raw statement diagnostic, index/template conflict regression coverage, and the existing filter correction. The diagnostic preserves all ambiguous keys and actual observation clocks for review. The adjustment records measured file and line counts. Complexity, long-line, and large-file budgets retain their existing limits. If a common normalization audit replaces the standalone command or the command is retired, remove duplicated code and lower these budgets by the actual reduction. Run the shared research-code-quality --ratchet gate and affected regression tests before integration.

After integrating the minute history changes from current main, the combined measured inventory is 509 Python files and 119725 lines, compared with main's 507 files and 119523 lines. This repair still adds exactly two files and 202 lines. Main's existing features and documentation are retained.

Evening margin recovery size budget (2026-10-21)¶

The measured shared size budget changes from 518 Python files / 122007 lines to 519 files / 122475 lines. The 468 added lines comprise a 187-line owner helper, 20 entry-point lines and 261 regression-test lines. Complexity, long-function, long-line, oversized-file and exclusion budgets remain fixed. Reduce this allowance when the recovery logic is consolidated or retired, retaining repeated-run, calendar, stale-data, invalid-partition and existing-partition safety tests. Validate the shared size ratchet, all owner local gates and public CI together.