docs: update docs
Some checks failed
continuous-integration/drone/push Build is failing

This commit is contained in:
carla 2026-02-02 13:42:24 +01:00
parent f5591c392a
commit c56ca68922

View file

@ -2,7 +2,7 @@
**Version:** 1.0 **Version:** 1.0
**Last Updated:** 2026-01-13 **Last Updated:** 2026-01-13
**Status:** In Progress (Backend Complete, UI Pending) **Status:** In Progress (Backend Complete, UI Complete, Tests Pending)
**Related Documents:** **Related Documents:**
- [Feature Roadmap](./feature-roadmap.md) - Overall feature planning - [Feature Roadmap](./feature-roadmap.md) - Overall feature planning
@ -15,15 +15,15 @@
- ✅ Issue #4: Header Normalization + Per-Header Mapping - ✅ Issue #4: Header Normalization + Per-Header Mapping
- ✅ Issue #5: Validation (Required Fields) + Error Formatting - ✅ Issue #5: Validation (Required Fields) + Error Formatting
- ✅ Issue #6: Persistence via Ash Create + Per-Row Error Capture (with Error-Capping) - ✅ Issue #6: Persistence via Ash Create + Per-Row Error Capture (with Error-Capping)
- ✅ Issue #11: Custom Field Import (Backend) - ✅ Issue #7: Admin Global Settings LiveView UI (Upload + Start Import + Results + Template Links)
- ✅ Issue #8: Authorization + Limits
- ✅ Issue #11: Custom Field Import (Backend + UI)
**In Progress / Pending:** **In Progress / Pending:**
- ⏳ Issue #7: Admin Global Settings LiveView UI (Upload + Start Import + Results)
- ⏳ Issue #8: Authorization + Limits
- ⏳ Issue #9: End-to-End LiveView Tests + Fixtures - ⏳ Issue #9: End-to-End LiveView Tests + Fixtures
- ⏳ Issue #10: Documentation Polish - ⏳ Issue #10: Documentation Polish
**Latest Update:** Error-Capping in `process_chunk/4` implemented (2025-01-XX) **Latest Update:** CSV Import UI fully implemented in GlobalSettingsLive with chunk processing, progress tracking, error display, and custom field support (2026-01-13)
--- ---
@ -161,6 +161,13 @@ A **basic CSV member import feature** that allows administrators to upload a CSV
- Any custom field column using the custom field's **name** as the header (e.g., `membership_number`, `birth_date`) - Any custom field column using the custom field's **name** as the header (e.g., `membership_number`, `birth_date`)
- **Important:** Custom fields must be created in Mila before importing. The CSV header must match the custom field name exactly (same normalization as member fields). - **Important:** Custom fields must be created in Mila before importing. The CSV header must match the custom field name exactly (same normalization as member fields).
- **Behavior:** If the CSV contains custom field columns that don't exist in Mila, a warning message will be shown and those columns will be ignored during import. - **Behavior:** If the CSV contains custom field columns that don't exist in Mila, a warning message will be shown and those columns will be ignored during import.
- **Value Validation:** Custom field values are validated according to the custom field type:
- **string**: Any text value (trimmed)
- **integer**: Must be a valid integer (e.g., `42`, `-10`). Invalid values will cause a row error with the custom field name and reason.
- **boolean**: Accepts `true`, `false`, `1`, `0`, `yes`, `no`, `ja`, `nein` (case-insensitive). Invalid values will cause a row error.
- **date**: Must be in ISO-8601 format (YYYY-MM-DD, e.g., `2024-01-15`). Invalid values will cause a row error.
- **email**: Must be a valid email format (contains `@`, 5-254 characters, valid format). Invalid values will cause a row error.
- **Error Messages:** Custom field validation errors are included in the import error list with format: `custom_field: <name> <reason>` (e.g., `custom_field: Alter expected integer, got: abc`)
**Member Field Header Mapping:** **Member Field Header Mapping:**
@ -496,36 +503,51 @@ Use `Mv.Authorization.PermissionSets` (preferred) instead of hard-coded string c
**Dependencies:** Issue #6 **Dependencies:** Issue #6
**Status:** ✅ **COMPLETED**
**Goal:** UI section with upload, progress, results, and template links. **Goal:** UI section with upload, progress, results, and template links.
**Tasks:** **Tasks:**
- [ ] Render import section only for admins - [x] Render import section only for admins
- [ ] **Add prominent UI notice about custom fields:** - [x] **Add prominent UI notice about custom fields:**
- Display alert/info box: "Custom fields must be created in Mila before importing CSV files with custom field columns" - Display alert/info box: "Custom fields must be created in Mila before importing CSV files with custom field columns"
- Explain: "Use the custom field name as the CSV column header (same normalization as member fields applies)" - Explain: "Use the custom field name as the CSV column header (same normalization as member fields applies)"
- Add link to custom fields management section - Add link to custom fields management section
- [ ] Configure `allow_upload/3`: - [x] Configure `allow_upload/3`:
- `.csv` only, `max_entries: 1`, `max_file_size: 10MB`, `auto_upload: false` - `.csv` only, `max_entries: 1`, `max_file_size: 10MB`, `auto_upload: true` (auto-upload enabled for better UX)
- [ ] `handle_event("start_import", ...)`: - [x] `handle_event("start_import", ...)`:
- Admin permission check - Admin permission check
- Consume upload -> read file content - Consume upload -> read file content
- Call `MemberCSV.prepare/2` - Call `MemberCSV.prepare/2`
- Store `import_state` in assigns (chunks + column_map + metadata) - Store `import_state` in assigns (chunks + column_map + metadata)
- Initialize progress assigns - Initialize progress assigns
- `send(self(), {:process_chunk, 0})` - `send(self(), {:process_chunk, 0})`
- [ ] `handle_info({:process_chunk, idx}, socket)`: - [x] `handle_info({:process_chunk, idx}, socket)`:
- Fetch chunk from `import_state` - Fetch chunk from `import_state`
- Call `MemberCSV.process_chunk/3` - Call `MemberCSV.process_chunk/4` with error capping support
- Merge counts/errors into progress assigns (cap errors at 50 overall) - Merge counts/errors into progress assigns (cap errors at 50 overall)
- Schedule next chunk (or finish and show results) - Schedule next chunk (or finish and show results)
- [ ] Results UI: - Async task processing with SQL sandbox support for tests
- [x] Results UI:
- Success count - Success count
- Failure count - Failure count
- Error list (line number + message + field) - Error list (line number + message + field)
- **Warning messages for unknown custom field columns** (non-existent names) shown in results - **Warning messages for unknown custom field columns** (non-existent names) shown in results
- Progress indicator during import
- Error truncation notice when errors exceed limit
**Template links:** **Template links:**
- Link `/templates/member_import_en.csv` and `/templates/member_import_de.csv` via Phoenix static path helpers. - [x] Link `/templates/member_import_en.csv` and `/templates/member_import_de.csv` via Phoenix static path helpers.
**Definition of Done:**
- [x] Upload area with drag & drop support
- [x] Template download links (EN/DE)
- [x] Progress tracking during import
- [x] Results display with success/error counts
- [x] Error list with line numbers and field information
- [x] Warning display for unknown custom field columns
- [x] Admin-only access control
- [x] Async chunk processing with proper error handling
--- ---
@ -533,19 +555,32 @@ Use `Mv.Authorization.PermissionSets` (preferred) instead of hard-coded string c
**Dependencies:** None (can be parallelized) **Dependencies:** None (can be parallelized)
**Status:** ✅ **COMPLETED**
**Goal:** Ensure admin-only access and enforce limits. **Goal:** Ensure admin-only access and enforce limits.
**Tasks:** **Tasks:**
- [ ] Admin check in start import event handler - [x] Admin check in start import event handler (via `Authorization.can?/3`)
- [ ] File size enforced in upload config - [x] File size enforced in upload config (`max_file_size: 10MB`)
- [ ] Row limit enforced in `MemberCSV.prepare/2` (max_rows from config) - [x] Row limit enforced in `MemberCSV.prepare/2` (max_rows: 1000, configurable via opts)
- [ ] Configuration: - [x] Chunk size limit (200 rows per chunk)
```elixir - [x] Error limit (50 errors per import)
config :mv, csv_import: [ - [x] UI-level authorization check (import section only visible to admins)
max_file_size_mb: 10, - [x] Event-level authorization check (prevents unauthorized import attempts)
max_rows: 1000
] **Implementation Notes:**
``` - File size limit: 10 MB (10,485,760 bytes) enforced via `allow_upload/3`
- Row limit: 1,000 rows (excluding header) enforced in `MemberCSV.prepare/2`
- Chunk size: 200 rows per chunk (configurable via opts)
- Error limit: 50 errors per import (configurable via `@max_errors`)
- Authorization uses `MvWeb.Authorization.can?/3` with `:create` permission on `Mv.Membership.Member`
**Definition of Done:**
- [x] Admin-only access enforced at UI and event level
- [x] File size limit enforced
- [x] Row count limit enforced
- [x] Chunk processing with size limits
- [x] Error capping implemented
--- ---
@ -589,7 +624,7 @@ Use `Mv.Authorization.PermissionSets` (preferred) instead of hard-coded string c
**Priority:** High (Core v1 Feature) **Priority:** High (Core v1 Feature)
**Status:** ✅ **COMPLETED** (Backend Implementation) **Status:** ✅ **COMPLETED** (Backend + UI Implementation)
**Goal:** Support importing custom field values from CSV columns. Custom fields should exist in Mila before import for best results. **Goal:** Support importing custom field values from CSV columns. Custom fields should exist in Mila before import for best results.
@ -604,23 +639,26 @@ Use `Mv.Authorization.PermissionSets` (preferred) instead of hard-coded string c
- [x] Query existing custom fields during `prepare/2` to map custom field columns - [x] Query existing custom fields during `prepare/2` to map custom field columns
- [x] Collect unknown custom field columns and add warning messages (don't fail import) - [x] Collect unknown custom field columns and add warning messages (don't fail import)
- [x] Map custom field CSV values to `CustomFieldValue` creation in `process_chunk/4` - [x] Map custom field CSV values to `CustomFieldValue` creation in `process_chunk/4`
- [x] Handle custom field type validation (string, integer, boolean, date, email) - [x] Handle custom field type validation (string, integer, boolean, date, email) with proper error messages
- [x] Create `CustomFieldValue` records linked to members during import - [x] Create `CustomFieldValue` records linked to members during import
- [ ] Update error messages to include custom field validation errors (if needed) - [x] Validate custom field values and return structured errors with custom field name and reason
- [ ] Add UI help text explaining custom field requirements (pending Issue #7): - [x] UI help text and link to custom field management (implemented in Issue #7)
- [x] Update error messages to include custom field validation errors (format: `custom_field: <name> expected <type>, got: <value>`)
- [x] Add UI help text explaining custom field requirements (completed in Issue #7):
- "Custom fields must be created in Mila before importing" - "Custom fields must be created in Mila before importing"
- "Use the custom field name as the CSV column header (same normalization as member fields)" - "Use the custom field name as the CSV column header (same normalization as member fields)"
- Link to custom fields management section - Link to custom fields management section
- [ ] Update CSV templates documentation to explain custom field columns (pending Issue #1) - [x] Update CSV templates documentation to explain custom field columns (documented in Issue #1)
- [x] Add tests for custom field import (valid, invalid name, type validation, warning for unknown) - [x] Add tests for custom field import (valid, invalid name, type validation, warning for unknown)
**Definition of Done:** **Definition of Done:**
- [x] Custom field columns are recognized by name (with normalization) - [x] Custom field columns are recognized by name (with normalization)
- [x] Warning messages shown for unknown custom field columns (import continues) - [x] Warning messages shown for unknown custom field columns (import continues)
- [x] Custom field values are created and linked to members - [x] Custom field values are created and linked to members
- [x] Type validation works for all custom field types - [x] Type validation works for all custom field types (string, integer, boolean, date, email)
- [ ] UI clearly explains custom field requirements (pending Issue #7) - [x] UI clearly explains custom field requirements (completed in Issue #7)
- [x] Tests cover custom field import scenarios (including warning for unknown names) - [x] Tests cover custom field import scenarios (including warning for unknown names)
- [x] Error messages include custom field validation errors with proper formatting
**Implementation Notes:** **Implementation Notes:**
- Custom field lookup is built in `prepare/2` and passed via `custom_field_lookup` in opts - Custom field lookup is built in `prepare/2` and passed via `custom_field_lookup` in opts