test: single-command runner, cross-platform automation, fixture auto-discovery
Adds a top-level test infrastructure layer addressing four needs at once:
a single command to run anything, cross-platform automation, install/e2e
sanity, and zero-config pickup of new fixtures dropped into test-cases/.
Top-level runner — run_tests.py
python run_tests.py # everything (default)
python run_tests.py --tool dedup # one tool's tests
python run_tests.py --unit # category scopes
python run_tests.py --e2e # end-to-end CLI
python run_tests.py --install # import / dependency sanity
python run_tests.py --fixtures # corpus + dropped-file sweep
python run_tests.py --coverage # term-missing report
python run_tests.py --quick # skip @pytest.mark.slow
Tools: analyze, cli, config, dedup, io, normalizers, text_clean.
Cross-platform — tox.ini
Envs for py310-py313 plus install / e2e / fixtures / coverage / lint.
Forces UTF-8 (PYTHONUTF8=1, PYTHONIOENCODING=utf-8) so identical fixture
bytes parse the same on Linux/macOS/Windows.
Shared config — pytest.ini
testpaths, python_files conventions, custom markers (slow, e2e, install,
fixture_sweep), warning filters that fail on our own DeprecationWarnings
while tolerating third-party ones.
New test layers
tests/test_install.py — required deps import; project modules import;
src.core public API surface; CLI --help exits 0; streamlit app.py
parses as valid Python; run_tests.py --help works.
tests/test_e2e.py — CLI roundtrips: cli_analyze table + JSON, cli_text_clean
--apply writes a real file with NBSP/smart-quote folded, dedup CLI
removes duplicates, run_tests.py self-tests.
tests/test_fixtures_sweep.py — parametrizes over every CSV/TSV/XLSX
inside test-cases/ (excluding text-cleaner-corpus/, which has its own
suite). Each fixture must: load through repair_bytes, run analyze()
cleanly, and survive clean_dataframe() with row/col counts unchanged
plus idempotency. Drop a CSV in, re-run — no test code changes needed.
tests/test_gap_coverage.py — closes audit gaps: clean_headers=False
toggle, repair_bytes with tab/semicolon delimiters, BOM+NUL+smart-
quote combined-fix scenario, analyze() over an XLSX path, sample_rows
larger than the DataFrame, mid-cell BOM, findings_by_tool edges, plus
a strict xfail documenting the known §4.17 numeric/phone whitespace
heuristic gap.
Test count
Before: 288 passed + 1 xfailed
After: 475 passed + 2 xfailed (the second xfail is the documented
collapse_whitespace gap on phone-shaped cells; spec §4.17 calls
for a heuristic that hasn't been implemented yet).
Functional gaps surfaced (not fixed in this commit):
- Text cleaner: collapse_whitespace runs unconditionally on every string
cell, including phone/numeric/date-shaped ones. Spec §4.17 requires a
skip heuristic. Captured as strict xfail so the gap stays visible.
- io.read_file does not run pre-parse repair; only analyze() and direct
callers of read_csv_repaired() get it. CLI tool pages and the dedup
CLI miss the safety net.
- Analyzer has no mixed_line_endings detector or near_duplicate_rows
detector; both planned but require additional plumbing.
- GUI tool pages each have their own uploader instead of picking up the
home-page upload through session_state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
143
tests/test_e2e.py
Normal file
143
tests/test_e2e.py
Normal file
@@ -0,0 +1,143 @@
|
||||
"""End-to-end smoke tests.
|
||||
|
||||
Round-trips through the CLI binaries with real fixture inputs to catch
|
||||
glue-code breakage that pure unit tests miss: argv parsing, file I/O, log
|
||||
configuration, exit codes, and the integration between the analyzer, the
|
||||
pre-parse repair, and pandas.
|
||||
|
||||
These are intentionally lightweight — one happy path per CLI plus a
|
||||
couple of failure modes. Bigger scenarios live in ``test_corpus.py`` and
|
||||
``test_fixtures_sweep.py``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.e2e
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
CORPUS_KITCHEN_SINK = (
|
||||
PROJECT_ROOT / "test-cases" / "text-cleaner-corpus" / "test_data" / "20_kitchen_sink.csv"
|
||||
)
|
||||
|
||||
|
||||
def _run(*args: str, cwd: Path | None = None, **kwargs):
|
||||
return subprocess.run(
|
||||
[sys.executable, *args],
|
||||
capture_output=True, text=True, timeout=60,
|
||||
cwd=cwd or PROJECT_ROOT,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# cli_analyze — full round-trip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAnalyzeCliE2E:
|
||||
def test_table_output_on_kitchen_sink(self):
|
||||
if not CORPUS_KITCHEN_SINK.exists():
|
||||
pytest.skip("kitchen-sink fixture missing")
|
||||
proc = _run("-m", "src.cli_analyze", str(CORPUS_KITCHEN_SINK))
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
# Rich tables wrap; assert on stable substrings.
|
||||
assert "Text Cleaner" in proc.stdout
|
||||
assert "csv_bom_stripped" in proc.stdout or "smart_quotes" in proc.stdout
|
||||
|
||||
def test_json_output_parses(self):
|
||||
if not CORPUS_KITCHEN_SINK.exists():
|
||||
pytest.skip("kitchen-sink fixture missing")
|
||||
proc = _run("-m", "src.cli_analyze", str(CORPUS_KITCHEN_SINK), "--json")
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
data = json.loads(proc.stdout)
|
||||
assert isinstance(data, list) and len(data) > 0
|
||||
for item in data:
|
||||
assert {"id", "severity", "tool", "count", "description"} <= set(item)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# cli_text_clean — full round-trip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestTextCleanCliE2E:
|
||||
def test_apply_writes_cleaned_file(self, tmp_path):
|
||||
# Build a small dirty CSV: NBSP padding + smart quotes.
|
||||
src = tmp_path / "dirty.csv"
|
||||
src.write_text(
|
||||
"id,name,note\n"
|
||||
"1, Alice ,“hello”\n"
|
||||
"2, Bob ,it’s fine\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "out.csv"
|
||||
proc = _run(
|
||||
"-m", "src.cli_text_clean", str(src),
|
||||
"--apply", "--output", str(out),
|
||||
)
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
assert out.exists(), "cleaned file was not written"
|
||||
cleaned = pd.read_csv(out, dtype=str, keep_default_na=False, encoding="utf-8-sig")
|
||||
# NBSP padding stripped
|
||||
assert cleaned.iloc[0]["name"] == "Alice"
|
||||
assert cleaned.iloc[1]["name"] == "Bob"
|
||||
# Smart quotes folded
|
||||
assert cleaned.iloc[0]["note"] == '"hello"'
|
||||
assert cleaned.iloc[1]["note"] == "it's fine"
|
||||
|
||||
def test_preview_does_not_write(self, tmp_path):
|
||||
src = tmp_path / "input.csv"
|
||||
src.write_text("id,name\n1,Alice\n", encoding="utf-8")
|
||||
# Without --apply, no output file should appear.
|
||||
proc = _run("-m", "src.cli_text_clean", str(src))
|
||||
assert proc.returncode == 0
|
||||
# Default output path next to input — must not exist.
|
||||
default_out = src.with_name(src.stem + "_cleaned.csv")
|
||||
assert not default_out.exists()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# cli (dedup) — full round-trip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestDedupCliE2E:
|
||||
def test_apply_removes_duplicates(self, tmp_path):
|
||||
src = tmp_path / "dups.csv"
|
||||
src.write_text(
|
||||
"name,email\n"
|
||||
"Alice,alice@x.com\n"
|
||||
"Alice,alice@x.com\n"
|
||||
"Bob,bob@x.com\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
out = tmp_path / "deduped.csv"
|
||||
proc = _run(
|
||||
"-m", "src.cli", str(src),
|
||||
"--apply", "--output", str(out),
|
||||
)
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
assert out.exists()
|
||||
result = pd.read_csv(out, dtype=str, keep_default_na=False, encoding="utf-8-sig")
|
||||
assert len(result) == 2 # Alice deduped, Bob unique
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# run_tests.py self-test — sanity check the runner itself works
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestRunTestsE2E:
|
||||
def test_tool_filter_runs_subset(self):
|
||||
proc = _run("run_tests.py", "--tool", "config", "-v")
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
# Check we limited the run via -k.
|
||||
assert "config" in proc.stdout.lower()
|
||||
|
||||
def test_unknown_tool_exits_2(self):
|
||||
proc = _run("run_tests.py", "--tool", "no_such_tool")
|
||||
assert proc.returncode == 2
|
||||
Reference in New Issue
Block a user