giteadmin/lifeos-prod
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Michael
2026-03-03 00:44:33 +00:00
commit 5297da485f
126 changed files with 54767 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

261
project-docs/lifeos-development-status-test1.md Normal file
View File

@@ -0,0 +1,261 @@
# Life OS - Development Status & Continuation Guide (Test Infrastructure - Convo Test1)
**Last Updated:** 2026-03-01
**Current State:** Test suite deployed, introspection verified (121 routes discovered), first test run pending
**GitHub:** mdombaugh/lifeos-dev (main branch)
---
## 1. What Was Built in This Conversation
### Dynamic Introspection-Based Test Suite (DEPLOYED)
Built and deployed an automated test suite that discovers routes from the live FastAPI app at runtime. Zero hardcoded routes. When a new router is added, smoke and CRUD tests auto-expand on next run.
**Architecture (11 files in /opt/lifeos/dev/tests/):**
| File | Purpose | Lines |
|------|---------|-------|
| introspect.py | Route discovery engine: walks app.routes, extracts paths/methods/Form() fields/path params, classifies routes | 357 |
| form_factory.py | Generates valid POST form data from introspected Form() signatures + seed data UUIDs | 195 |
| registry.py | Imports app, runs introspection once, exposes route registry + PREFIX_TO_SEED mapping + resolve_path() | 79 |
| conftest.py | Fixtures only: test DB engine, per-test rollback session, httpx client, 15 seed data fixtures, all_seeds composite | 274 |
| test_smoke_dynamic.py | 3 parametrized functions expanding to ~59 tests: all GETs (no params) return 200, all GETs (with seed ID) return 200, all detail/edit GETs (fake UUID) return 404 | 100 |
| test_crud_dynamic.py | 5 parametrized functions expanding to ~62 tests: all POST create/edit/delete redirect 303, all actions non-500, create-then-verify-in-list | 161 |
| test_business_logic.py | 16 hand-written tests: timer single-run constraint, stop sets end_at, soft delete/restore visibility, search SQL injection, sidebar integrity, focus/capture workflows, edge cases | 212 |
| route_report.py | CLI tool: dumps all discovered routes with classification, form fields, seed mapping coverage | 65 |
| run_tests.sh | Test runner with aliases: smoke, crud, logic, report, fast, full, custom args | 22 |
| __init__.py | Package marker | 0 |
| pytest.ini | Config: asyncio_mode=auto, verbose output, short tracebacks | 7 |
**Key design decisions:**
- `registry.py` separated from `conftest.py` to avoid pytest auto-loading conflicts (test files import from registry, not conftest)
- Form() detection uses `__class__.__name__` check, not `issubclass()`, because FastAPI's `Form` is a function not a class
- Test DB schema cloned from live dev DB via pg_dump (not from stale SQL files)
- Seed data uses raw SQL INSERT matching actual table columns
### Introspection Verification Results
Deploy script step 4 confirmed:
```
Routes discovered: 121
GET (no params): 36
GET (with params): 23
POST create: 13
POST edit: 13
POST delete: 17
POST action: 19
Entity prefixes: 32
```
### What Was NOT Done
- **First test run not yet executed** -- introspection works, tests deployed, but `run_tests.sh` has not been run yet
- **Seed data column mismatches likely** -- seed INSERTs written from architecture docs, not actual table inspection. First run will surface these as SQL errors
- **No test for file upload routes** -- file routes skipped (has_file_upload flag) because they need multipart handling
---
## 2. Test Infrastructure Inventory
### 2.1 Database
| Component | Details |
|-----------|---------|
| Test DB | `lifeos_test` on lifeos-db container |
| Schema source | Cloned from `lifeos_dev` via `pg_dump --schema-only` |
| Tables | 48 (matches dev) |
| Isolation | Per-test transaction rollback (no data persists between tests) |
| Credentials | Same as dev: postgres:UCTOQDZiUhN8U |
### 2.2 How Introspection Works
1. `registry.py` imports `main.app` (sets DATABASE_URL to test DB first)
2. `introspect.py` walks `app.routes`, for each `APIRoute`:
- Extracts path, HTTP methods, endpoint function reference
- Parses `{id}` path parameters via regex
- Inspects endpoint function signature for `Form()` parameters (checks `default.__class__.__name__` for "FieldInfo")
- Extracts query parameters (non-Form, non-Depends, non-Request params)
- Classifies route: list / detail / create_form / edit_form / create / edit / delete / toggle / action / json / page
3. Builds `ROUTE_REGISTRY` dict keyed by kind (get_no_params, post_create, etc.) and by prefix
4. Test files parametrize from this registry at collection time
### 2.3 How Dynamic Tests Work
**Smoke (test_smoke_dynamic.py):**
```python
@pytest.mark.parametrize("path", [r.path for r in GET_NO_PARAMS])
async def test_get_no_params_returns_200(client, path):
r = await client.get(path)
assert r.status_code == 200
```
N discovered GET routes = N smoke tests. No manual updates.
**CRUD (test_crud_dynamic.py):**
- Collects all POST create/edit/delete routes from registry
- Calls `build_form_data(route.form_fields, all_seeds)` to generate valid payloads
- `form_factory.py` resolves FK fields to seed UUIDs, generates values by field name pattern
- Asserts 303 redirect for create/edit/delete, non-500 for actions
**Business Logic (test_business_logic.py):**
- Hand-written, tests behavioral contracts not discoverable via introspection
- Timer: single running constraint, stop sets end_at, /time/running returns JSON
- Soft deletes: deleted task hidden from list, restore reappears
- Search: SQL injection doesn't crash, empty query works, unicode works
- Sidebar: domain appears on every page, project hierarchy renders
- Focus/capture: add to focus, multi-line capture creates multiple items
- Edge cases: invalid UUID, timer without task_id, double delete
### 2.4 Seed Data Fixtures (15 entities)
| Fixture | Table | Dependencies | Key Fields |
|---------|-------|-------------|------------|
| seed_domain | domains | none | id, name, color |
| seed_area | areas | seed_domain | id, domain_id |
| seed_project | projects | seed_domain, seed_area | id, domain_id, area_id |
| seed_task | tasks | seed_domain, seed_project | id, domain_id, project_id, title |
| seed_contact | contacts | none | id, first_name, last_name |
| seed_note | notes | seed_domain | id, domain_id, title |
| seed_meeting | meetings | none | id, title, meeting_date |
| seed_decision | decisions | seed_domain, seed_project | id, title |
| seed_appointment | appointments | seed_domain | id, title, start_at, end_at |
| seed_weblink_folder | weblink_folders | none | id, name |
| seed_list | lists | seed_domain, seed_project | id, name |
| seed_link | links | seed_domain | id, title, url |
| seed_weblink | weblinks | seed_weblink_folder | id, title, url |
| seed_capture | capture | none | id, raw_text |
| seed_focus | daily_focus | seed_task | id, task_id |
### 2.5 PREFIX_TO_SEED Mapping
Maps route prefixes to seed fixture keys so `resolve_path()` can replace `{id}` with real UUIDs:
```
/domains -> domain /contacts -> contact
/areas -> area /meetings -> meeting
/projects -> project /decisions -> decision
/tasks -> task /appointments -> appointment
/notes -> note /weblinks -> weblink
/links -> link /weblinks/folders -> weblink_folder
/lists -> list /focus -> focus
/capture -> capture /time -> task
/files -> None (skipped) /admin/trash -> None (skipped)
```
---
## 3. File Locations on Server
```
/opt/lifeos/dev/
tests/
__init__.py
introspect.py # Route discovery engine
form_factory.py # Form data generation
registry.py # Route registry + PREFIX_TO_SEED + resolve_path
conftest.py # Fixtures (DB, client, seeds)
route_report.py # CLI route dump
test_smoke_dynamic.py # Auto-parametrized GET tests
test_crud_dynamic.py # Auto-parametrized POST tests
test_business_logic.py # Hand-written behavioral tests
run_tests.sh # Test runner
pytest.ini # pytest config
deploy-tests.sh # Deployment script (can re-run to reset)
```
---
## 4. Known Issues & Expected First-Run Failures
### 4.1 Likely Seed Data Mismatches
Seed INSERT statements were written from architecture docs, not from inspecting actual table columns. The first test run will likely produce errors like:
- `column "X" of relation "Y" does not exist` -- seed INSERT has a column the actual table doesn't have
- `null value in column "X" violates not-null constraint` -- seed INSERT is missing a required column
**Fix process:** Run tests, read the SQL errors, adjust the INSERT in conftest.py to match actual columns (query with `\d table_name`), redeploy.
### 4.2 Possible Form Field Discovery Gaps
Some routers may use patterns the introspection engine doesn't handle:
- `Annotated[str, Form()]` style (handled via `__metadata__` check, but untested against live code)
- Form fields with non-standard defaults
- Routes that accept both Form and query params
The route report (`run_tests.sh report`) will show warnings for POST create/edit routes with zero discovered Form fields. Those need investigation.
### 4.3 Route Classification Edge Cases
Some routes may be misclassified:
- Admin trash restore routes (`/admin/trash/restore/{entity}/{id}`) may not match the standard patterns
- Capture routes (`/capture/add`, `/capture/{id}/convert`, `/capture/{id}/dismiss`) use non-standard action patterns
- Focus routes (`/focus/add`, `/focus/{id}/remove`) are action routes, not standard CRUD
These will show up as action route tests (non-500 assertion) rather than typed CRUD tests.
### 4.4 Not Yet Pushed to GitHub
Test files need to be committed: `cd /opt/lifeos/dev && git add . && git commit -m "Dynamic test suite" && git push origin main`
---
## 5. How to Continue (Convo Test2)
### Immediate Next Steps
1. **Run the route report** to verify introspection output:
```bash
docker exec lifeos-dev bash /app/tests/run_tests.sh report
```
2. **Run smoke tests first** (most likely to pass):
```bash
docker exec lifeos-dev bash /app/tests/run_tests.sh smoke
```
3. **Fix seed data failures** by inspecting actual tables and adjusting conftest.py INSERTs
4. **Run CRUD tests** after seeds are fixed:
```bash
docker exec lifeos-dev bash /app/tests/run_tests.sh crud
```
5. **Run business logic tests** last:
```bash
docker exec lifeos-dev bash /app/tests/run_tests.sh logic
```
6. **Run full suite** once individual categories pass:
```bash
docker exec lifeos-dev bash /app/tests/run_tests.sh
```
### When Adding a New Entity Router
1. Add seed fixture to `conftest.py` (INSERT matching actual table columns)
2. Add entry to `PREFIX_TO_SEED` in `registry.py`
3. Run tests -- smoke and CRUD auto-expand to cover new routes
4. Add behavioral tests to `test_business_logic.py` if entity has constraints or state machines
### When Schema Changes
Re-run `deploy-tests.sh` (step 1 drops and recreates lifeos_test from current dev schema).
Or manually:
```bash
docker exec lifeos-db psql -U postgres -c "DROP DATABASE IF EXISTS lifeos_test;"
docker exec lifeos-db psql -U postgres -c "CREATE DATABASE lifeos_test;"
docker exec lifeos-db pg_dump -U postgres -d lifeos_dev --schema-only -f /tmp/s.sql
docker exec lifeos-db psql -U postgres -d lifeos_test -f /tmp/s.sql -q
```
### Test Runner Commands
```bash
docker exec lifeos-dev bash /app/tests/run_tests.sh # Full suite
docker exec lifeos-dev bash /app/tests/run_tests.sh report # Route introspection dump
docker exec lifeos-dev bash /app/tests/run_tests.sh smoke # All GET endpoints
docker exec lifeos-dev bash /app/tests/run_tests.sh crud # All POST create/edit/delete
docker exec lifeos-dev bash /app/tests/run_tests.sh logic # Business logic
docker exec lifeos-dev bash /app/tests/run_tests.sh fast # Smoke, stop on first fail
docker exec lifeos-dev bash /app/tests/run_tests.sh -k "timer" # pytest keyword filter
```
---
## 6. Application Development Remaining (Unchanged from Convo 4)
### Tier 3 Remaining (4 features)
1. **Processes / process_runs** -- Most complex. 4 tables. Template CRUD, run instantiation, step completion, task generation.
2. **Calendar view** -- Unified read-only view of appointments + meetings + tasks.
3. **Time budgets** -- Simple CRUD: domain_id + weekly_hours + effective_from.
4. **Eisenhower matrix** -- Derived 2x2 grid from task priority + due_date.
### Tier 4, UX Polish, Production Deployment
See lifeos-development-status-convo4.md sections 5, 8, 9.