|
|
abb720997e
|
docs: tight, scannable rewrite — every item earns its place
Refactors all 10 docs (README, USER-GUIDE, CLI-REFERENCE, REQUIREMENTS,
TECHNICAL, DEVELOPER, BUSINESS, DECISIONS, RECOVERY, docs/README) from
prose-heavy to bullet-heavy + table-heavy. Same information density,
significantly less reading load.
Net: 2600 → 1652 lines (~37% reduction) WHILE adding the new content
that landed since v1.6:
- Format Standardizer (3rd Ready tool)
- 199-row buyer corpus
- src/core/errors.py structured hierarchy + ensure_dataframe /
ensure_choice / wrap_file_read|write / format_for_user helpers
- src/core/_constants.py shared USPS/state lookup tables
- Cross-tool audit fixes (NaN matching, removed_df schema, validation,
enum-bounds checks, forward-compat config)
- Per-domain error_policy across format standardizers
- Inconsistent-date-format detector
- Excel header-row auto-detection + write_file delimiter param
Per-doc changes:
- README.md (175 → 71): 9-tool table at top, status column, 3 CLI
entry points listed, dropped repeated marketing prose.
- docs/README.md (38 → 27): pure index — buyer-facing vs creator-only
split + version footer.
- USER-GUIDE.md (208 → 118): tool table replaces script descriptions,
troubleshooting compressed to bullets, gate explanation tightened.
- CLI-REFERENCE.md (451 → 235): collapsed flag tables, removed
redundant intro text, kept full recipes section.
- REQUIREMENTS.md (146 → 129): 18 numbered sections (was 17), added
§18 Error Handling, formatting tightened to single-line entries.
- TECHNICAL.md (570 → 350): collapsed §3 build pipeline tables, merged
redundant §3.5-3.7 OS sections, added §7 (Error handling) +
§11.3 (Format Standardizer spec) + §11.4-11.7 (analyzer / gate /
Review page / repair_bytes promoted from §10.2.x sub-numbering).
- DEVELOPER.md (285 → 161): module map table replaces per-file prose,
extension recipes condensed, new §Errors covers when to use each
hierarchy class.
- BUSINESS.md (278 → 225): collapsed prose to tables (use cases,
competitive landscape, costs, risks); honest-status updated.
- DECISIONS.md (269 → 189): scoring rubric + GUI matrix preserved,
decision log compressed to single-line entries, added v1.6 entries
(Format Standardizer Ready, errors module).
- RECOVERY.md (180 → 147): rebuild steps as numbered + tabular,
external dependencies as one table, recovery priorities tightened.
No information removed; redundancy compressed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-01 02:49:29 +00:00 |
|
|
|
54f92ae47e
|
feat: implement text cleaner (script 02) with CLI, GUI, and tests
Builds 02_text_cleaner.py from stub to working: character-level hygiene
for CSV/Excel inputs covering trim, whitespace collapse, smart-character
folding, Unicode NFC/NFKC, BOM strip, zero-width strip, control-char
strip, line-ending normalization, and per-column case conversion. Three
presets (minimal/excel-hygiene/paranoid) keep the buyer surface small.
- src/core/text_clean.py: pure helpers + CleanOptions/CleanResult +
clean_dataframe with dtype-safe column selection
- src/cli_text_clean.py: Typer CLI mirroring the dedup CLI shape
(dry-run by default, --apply writes cleaned + changes audit, JSON
config save/load)
- src/gui/pages/2_Text_Cleaner.py: real Streamlit page with preset
picker, advanced toggles, preview, before/after metrics, and three
download buttons
- tests/test_text_clean.py + test_cli_text_clean.py: 92 new tests
covering edge cases E1-E50 from the spec
- samples/messy_text.csv: demo dataset surfacing UC1, UC3, UC6, UC10
in 10 rows
- test-cases/uc16-uc26 + ec05-ec09: per-use-case and per-edge-case
fixtures
Docs: TECHNICAL.md §10.2 (full Tier 1/2/3 spec), DECISIONS.md v1.7
entry locking the spec, CLI-REFERENCE.md gains the text cleaner
section, README.md gains a top-level Text Cleaner block, USER-GUIDE.md
status row 02 promoted Skeleton -> Working.
200/200 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-29 15:14:15 +00:00 |
|
