# ACCESS End-to-End Data Flow Diagram v0.3 — Healthie-First

![End-to-End Data Flow](../diagrams/data_flow.svg)

---

## Phase Descriptions

> **Terminology:** "Workers" below are implemented in dev as Django management commands (`access_core/cms_integration/management/commands/*`); staging/prod will run them as SQS-triggered Fargate tasks (infra defined under `infra/`, worker task code pending). Phases marked *planned* describe the target design, not yet-built code.

### Phase A: Pre-Enrollment Intake (Healthie-First)

1. **Current (dev):** Healthie Webhook → Django webhook view (`intake/views.py`) → handler dispatch via `get_handler()` in the registry (`intake/handlers.py`; consumers self-register with `@register_handler` from `intake/consumers.py`) → follow-up GraphQL fetch of the full resource (webhooks are lightweight pointers) → inline normalization into Core models inside `intake/consumers.py`
2. **Planned (staging/prod):** Healthie Webhook → ALB → SQS Queue → Fargate Worker → Normalization Engine (Terraform infra defined; worker task not yet built)
3. Populates: `raw_ingestion_records`, `intake_records`, `intake_identifiers`, `intake_assessments`, `intake_lifestyle`, `intake_female_health` / `intake_male_health`, `intake_msk`, `safety_screening`, `participation_readiness`
4. Physician Review Gate → Eligibility Check Worker → CMS Eligibility API
5. Results: `eligibility_checks`, `enrollment_decisions`, `patient_consent`

### Phase B: Alignment & Beneficiary Promotion

1. Alignment Worker → CMS Alignment API → `alignments`
2. Promotion Logic: `intake_records` → `beneficiaries`, `beneficiary_identifiers`, `external_system_identifiers`
3. Creates `care_periods`, `protocol_assignment_versions`

### Phase C: Post-Enrollment Clinical Data Collection

Sources: Healthie (implemented), Labcorp/Quest feeds (planned) → Webhook/Polling → SQS (staging/prod only; dev is direct HTTP) → Normalization. **Note:** Healthie webhooks are lightweight pointers only (resource_id, event_type, changed_fields) — worker must make a follow-up GraphQL query to fetch full resource data before normalization.
- `daily_checkin_submissions`, `daily_checkin_responses`, `patient_execution_logs`
- `lab_results`, `lab_result_items`, `source_document_records`
- `safety_signal_events` → `care_team_review_tasks` → `clinical_review_tasks`
- Clinician/Physician Review Gate → `monitoring_plan_results`

### Phase D: Data Curation Workflow

![Data Curation Workflow](../diagrams/data_curation_workflow.svg)

`raw_ingestion_records` → `data_curation_tasks` → `curated_observation_drafts` → `curation_review_events` (QC) → `clinical_review_tasks` → `clinical_observations`

### Phase E: OAP Reporting & CMS Submission

1. `clinical_observations` → `oap_measure_observations` (FK reference, not duplicate)
2. Physician Review Gate → CMS Submission Worker → CMS Data Reporting API
3. `cms_api_requests`, `cms_api_responses`, `cms_payment_files`, `recoupment_events`

### Phase F: Offshore Pseudonymous Review

1. `clinical_observations` → De-identification (**planned** — pseudonymization service is design-doc only for now) → `pseudonymous_case_records`, `pseudonymous_clinical_packets` (models implemented)
2. Offshore Reviewer UI (restricted role) → Notes forwarded to US Clinical Reviewer
3. `clinical_review_tasks` → `physician_reviews`

### Phase G: Audit & Provenance (All Phases)

**Planned — `audit_events` model implemented, writer not yet built.** Target behavior: every create/edit/approve/reject/export/view → `audit_events` (append-only, immutable)
- `changed_fields` stores field names only; PHI values are never written in plaintext
- SHA-256 hashes (`old_value_hash`/`new_value_hash`) instead of plaintext old/new values

### Phase H: Dashboard & Public Reporting (Future)

`clinical_observations` + `oap_measure_observations` → `aggregate_outcomes` → `suppression_rules` → `reporting_views` → Approval Gate → `public_report_exports`

### Polling Reconciliation (Fallback — planned; not yet implemented)

Scheduled job (every 15 min) queries Healthie APIs → compares against `raw_ingestion_records.source_record_id` → missing records stored + queued, existing skipped (idempotent). Polling uses GraphQL queries (e.g., `formAnswerGroups`, `documents`, `entries`) to fetch full resource data directly.

---

## System Boundaries

![System Boundaries & Source of Truth](../diagrams/system_boundaries.svg)