From 96daf2a089a9073d5de180373b0870708e9e427c Mon Sep 17 00:00:00 2001 From: carla Date: Tue, 3 Feb 2026 14:57:45 +0100 Subject: [PATCH] docs: update changelog --- CODE_GUIDELINES.md | 96 +++++++++++++++++++- docs/database-schema-readme.md | 37 ++++++-- docs/development-progress-log.md | 147 ++++++++++++++++++++++++++++++- docs/feature-roadmap.md | 45 ++++++++-- 4 files changed, 311 insertions(+), 14 deletions(-) diff --git a/CODE_GUIDELINES.md b/CODE_GUIDELINES.md index 0a87836..c7bcfa6 100644 --- a/CODE_GUIDELINES.md +++ b/CODE_GUIDELINES.md @@ -84,6 +84,8 @@ lib/ │ ├── custom_field_value.ex # Custom field value resource │ ├── custom_field.ex # CustomFieldValue type resource │ ├── setting.ex # Global settings (singleton resource) +│ ├── group.ex # Group resource +│ ├── member_group.ex # MemberGroup join table resource │ └── email.ex # Email custom type ├── membership_fees/ # MembershipFees domain │ ├── membership_fees.ex # Domain definition @@ -149,6 +151,8 @@ lib/ │ │ ├── membership_fee_type_live/ # Membership fee type LiveViews │ │ ├── membership_fee_settings_live.ex # Membership fee settings │ │ ├── global_settings_live.ex # Global settings +│ │ ├── group_live/ # Group management LiveViews +│ │ ├── import_export_live.ex # CSV import/export LiveView │ │ └── contribution_type_live/ # Contribution types (mock-up) │ ├── auth_overrides.ex # AshAuthentication overrides │ ├── endpoint.ex # Phoenix endpoint @@ -641,7 +645,95 @@ def card(assigns) do end ``` -### 3.3 System Actor Pattern +### 3.3 CSV Import Configuration + +**CSV Import Limits:** + +CSV import functionality supports configurable limits to prevent resource exhaustion: + +```elixir +# config/config.exs +config :mv, + csv_import: [ + max_file_size_mb: 10, # Maximum file size in megabytes + max_rows: 1000 # Maximum number of data rows (excluding header) + ] +``` + +**Accessing Configuration:** + +Use `Mv.Config` helper functions: + +```elixir +# Get max file size in bytes +max_bytes = Mv.Config.csv_import_max_file_size_bytes() + +# Get max file size in megabytes +max_mb = Mv.Config.csv_import_max_file_size_mb() + +# Get max rows +max_rows = Mv.Config.csv_import_max_rows() +``` + +**Best Practices:** +- Set reasonable limits based on server resources +- Display limits to users in UI +- Validate file size before upload +- Process imports in chunks (default: 200 rows per chunk) +- Cap error collection (default: 50 errors per import) + +### 3.4 Page-Level Authorization + +**CheckPagePermission Plug:** + +Use `MvWeb.Plugs.CheckPagePermission` for page-level authorization: + +```elixir +# lib/mv_web/router.ex +defmodule MvWeb.Router do + use MvWeb, :router + + # Add plug to router pipeline + pipeline :browser do + plug :accepts, ["html"] + plug :fetch_session + plug :fetch_live_flash + plug :put_root_layout, html: {MvWeb.Layouts, :root} + plug :protect_from_forgery + plug :put_secure_browser_headers + plug MvWeb.Plugs.CheckPagePermission # Page-level authorization + end +end +``` + +**Permission Set Route Matrix:** + +Routes are mapped to permission sets: +- `own_data`: Can access `/profile` and `/members/:id` (own linked member only) +- `read_only`: Can read all data, cannot modify +- `normal_user`: Can read and modify most data +- `admin`: Full access to all routes + +**Usage in LiveViews:** + +```elixir +# Check page access before mount +def mount(_params, _session, socket) do + actor = current_actor(socket) + + if MvWeb.Authorization.can_access_page?(actor, "/admin/roles") do + {:ok, assign(socket, :roles, load_roles(actor))} + else + {:ok, redirect(socket, to: ~p"/")} + end +end +``` + +**Public Paths:** + +Public paths (login, OIDC callbacks) are excluded from permission checks automatically. + +### 3.5 System Actor Pattern **When to Use System Actor:** @@ -726,7 +818,7 @@ Two mechanisms exist for bypassing standard authorization: **See also:** `docs/roles-and-permissions-architecture.md` (Authorization Bootstrap Patterns section) -### 3.4 Ash Framework +### 3.6 Ash Framework **Resource Definition Best Practices:** diff --git a/docs/database-schema-readme.md b/docs/database-schema-readme.md index 15e4e33..6bf11de 100644 --- a/docs/database-schema-readme.md +++ b/docs/database-schema-readme.md @@ -15,10 +15,10 @@ This document provides a comprehensive overview of the Mila Membership Managemen | Metric | Count | |--------|-------| -| **Tables** | 9 | +| **Tables** | 11 | | **Domains** | 4 (Accounts, Membership, MembershipFees, Authorization) | -| **Relationships** | 7 | -| **Indexes** | 20+ | +| **Relationships** | 9 | +| **Indexes** | 25+ | | **Triggers** | 1 (Full-text search) | ## Tables Overview @@ -77,6 +77,23 @@ This document provides a comprehensive overview of the Mila Membership Managemen - Membership fee default settings - Environment variable support for club name +#### `groups` +- **Purpose:** Group definitions for organizing members +- **Rows (Estimated):** Low (typically 5-20 groups per club) +- **Key Features:** + - Unique group names (case-insensitive) + - URL-friendly slugs (auto-generated, immutable) + - Optional descriptions + - Many-to-many relationship with members + +#### `member_groups` +- **Purpose:** Join table for many-to-many relationship between members and groups +- **Rows (Estimated):** Medium to High (multiple groups per member) +- **Key Features:** + - Unique constraint on (member_id, group_id) + - CASCADE delete on both sides + - Efficient indexes for queries + ### Authorization Domain #### `roles` @@ -100,6 +117,10 @@ Member (1) → (N) MembershipFeeCycles ↓ MembershipFeeType (1) +Member (N) ←→ (N) Group + ↓ ↓ + MemberGroups (N) MemberGroups (N) + Settings (1) → MembershipFeeType (0..1) ``` @@ -145,6 +166,12 @@ Settings (1) → MembershipFeeType (0..1) - Settings can reference a default fee type - `ON DELETE SET NULL` - if fee type is deleted, setting is cleared +9. **Member ↔ Group (N:N via MemberGroup)** + - Many-to-many relationship through `member_groups` join table + - `ON DELETE CASCADE` on both sides - removing member/group removes associations + - Unique constraint on (member_id, group_id) prevents duplicates + - Groups searchable via member search vector + ## Important Business Rules ### Email Synchronization @@ -509,7 +536,7 @@ mix run priv/repo/seeds.exs --- -**Last Updated:** 2026-01-13 -**Schema Version:** 1.4 +**Last Updated:** 2026-01-27 +**Schema Version:** 1.5 **Database:** PostgreSQL 17.6 (dev) / 16 (prod) diff --git a/docs/development-progress-log.md b/docs/development-progress-log.md index 928558e..1dcf994 100644 --- a/docs/development-progress-log.md +++ b/docs/development-progress-log.md @@ -1752,8 +1752,151 @@ This project demonstrates a modern Phoenix application built with: --- -**Document Version:** 1.4 -**Last Updated:** 2026-01-13 +--- + +## Recent Updates (2026-01-13 to 2026-01-27) + +### Groups Feature Implementation (2026-01-27) + +**PR #378:** *Add groups resource* (closes #371) +- Created `Mv.Membership.Group` resource with name, slug, description +- Created `Mv.Membership.MemberGroup` join table for many-to-many relationship +- Automatic slug generation from name (immutable after creation) +- Case-insensitive name uniqueness via LOWER(name) index +- Database migration: `20260127141620_add_groups_and_member_groups.exs` + +**PR #382:** *Groups Admin UI* (closes #372) +- Groups management LiveViews (`/groups`) +- Create, edit, delete groups with confirmation +- Member count display per group +- Add/remove members from groups +- Groups displayed in member overview and detail views +- Filter and sort by groups in member list + +**Key Features:** +- Many-to-many relationship: Members can belong to multiple groups +- Groups searchable via member search vector (full-text search) +- CASCADE delete: Removing member/group removes associations +- Unique constraint prevents duplicate member-group associations + +### CSV Import Feature Implementation (2026-01-27) + +**PR #359:** *Implements CSV Import UI* (closes #335) +- Import/Export LiveView (`/import_export`) +- CSV file upload with auto-upload +- Real-time import progress tracking +- Error and warning reporting +- Chunked processing (200 rows per chunk) + +**PR #394:** *Adds config for import limits* (closes #336) +- Configurable maximum file size (default: 10 MB) +- Configurable maximum rows (default: 1000) +- Configuration via `config :mv, csv_import: [max_file_size_mb: ..., max_rows: ...]` +- UI displays limits to users + +**PR #395:** *Implements custom field CSV import* (closes #338) +- Support for importing custom field values via CSV +- Custom field mapping by slug or name +- Validation of custom field value types +- Error reporting with line numbers and field names +- CSV templates (German and English) available for download + +**Key Features:** +- Member field import (email, first_name, last_name, etc.) +- Custom field value import (all types: string, integer, boolean, date, email) +- Error capping (max 50 errors per import to prevent memory issues) +- Async chunk processing with progress updates +- Admin-only access (requires `:create` permission on Member resource) + +### Page Permission Router Plug (2026-01-27) + +**PR #390:** *Page Permission Router Plug* (closes #388) +- `MvWeb.Plugs.CheckPagePermission` plug for page-level authorization +- Route-based permission checking +- Automatic redirects for unauthorized access +- Integration with permission sets (own_data, read_only, normal_user, admin) +- Documentation: `docs/page-permission-route-coverage.md` + +**Key Features:** +- Page-level access control before LiveView mount +- Permission set-based route matrix +- Redirect targets for different permission levels +- Public paths (login, OIDC callbacks) excluded from checks + +### Resource Policies Implementation (2026-01-27) + +**PR #387:** *CustomField Resource Policies* (closes #386) +- CustomField resource policies with actor-based authorization +- Admin-only create/update/destroy operations +- Read access for authenticated users +- No system-actor fallback (explicit actor required) + +**PR #377:** *CustomFieldValue Resource Policies* (closes #369) +- CustomFieldValue resource policies +- own_data permission set: can create/update own linked member's custom field values +- Admin and normal_user: full access +- Bypass read rule for CustomFieldValue pattern (documented) + +**PR #364:** *User Resource Policies* (closes #363) +- User resource policies with scope filtering +- own_data: can read/update own user record +- Admin: full access +- Email change validation for linked members + +### System Actor Improvements (2026-01-27) + +**PR #379:** *Fix System missing system actor in prod and prevent deletion* +- System actor user creation in migrations +- Block update/destroy on system-actor user +- System user handling in UserLive forms +- Normalize system actor email + +**PR #361:** *System Actor Mode for Systemic Flows* (closes #348) +- System actor pattern for systemic operations +- Email synchronization uses system actor +- Cycle generation uses system actor +- Documentation: `docs/roles-and-permissions-architecture.md` (Authorization Bootstrap Patterns) + +**PR #367:** *Remove NoActor bypass* +- Removed NoActor bypass to prevent masking authorization bugs +- All tests now require explicit actor +- Exception: AshAuthentication bypass tests (conscious exception) + +### Email Sync Fixes (2026-01-27) + +**PR #380:** *Fix email sync (user->member) when changing password and email* +- Email sync when admin sets password via `admin_set_password` +- Bidirectional email synchronization improvements +- Validation fixes for linked user-member pairs + +### UI/UX Improvements (2026-01-27) + +**PR #389:** *Change Logo* (closes #385) +- Updated application logo +- Logo display in sidebar and navigation + +**PR #362:** *Add boolean custom field filters to member overview* (closes #309) +- Boolean custom field filtering in member list +- Filter by true/false values +- Integration with existing filter system + +### Test Performance Optimization (2026-01-27) + +**PR #384:** *Minor test refactoring to improve on performance* (closes #383) +- Moved slow tests to nightly test suite +- Optimized policy tests +- Reduced test complexity in seeds tests +- Documentation: `docs/test-performance-optimization.md` + +**Key Changes:** +- Fast tests (standard CI): Business logic, validations, data persistence +- Slow tests (nightly): Performance tests, large datasets, query optimization +- UI tests: Basic HTML rendering, navigation, translations + +--- + +**Document Version:** 1.5 +**Last Updated:** 2026-01-27 **Maintainer:** Development Team **Status:** Living Document (update as project evolves) diff --git a/docs/feature-roadmap.md b/docs/feature-roadmap.md index 1df3eb6..7e28eea 100644 --- a/docs/feature-roadmap.md +++ b/docs/feature-roadmap.md @@ -1,7 +1,7 @@ # Feature Roadmap & Implementation Plan **Project:** Mila - Membership Management System -**Last Updated:** 2026-01-13 +**Last Updated:** 2026-01-27 **Status:** Active Development --- @@ -29,6 +29,10 @@ - ✅ **OIDC account linking with password verification** (PR #192, closes #171) - ✅ **Secure OIDC email collision handling** (PR #192) - ✅ **Automatic linking for passwordless users** (PR #192) +- ✅ **Page Permission Router Plug** - Page-level authorization (PR #390, closes #388, 2026-01-27) + - Route-based permission checking + - Automatic redirects for unauthorized access + - Integration with permission sets **Closed Issues:** - ✅ [#171](https://git.local-it.org/local-it/mitgliederverwaltung/issues/171) - OIDC handling and linking (closed 2025-11-13) @@ -55,6 +59,10 @@ - ✅ [#191](https://git.local-it.org/local-it/mitgliederverwaltung/issues/191) - Implement Roles in Ash (M) - Completed - ✅ [#190](https://git.local-it.org/local-it/mitgliederverwaltung/issues/190) - Implement Permissions in Ash (M) - Completed - ✅ [#151](https://git.local-it.org/local-it/mitgliederverwaltung/issues/151) - Define implementation plan for roles and permissions (M) - Completed +- ✅ [#388](https://git.local-it.org/local-it/mitgliederverwaltung/issues/388) - Page Permission Router Plug (closed 2026-01-27) +- ✅ [#386](https://git.local-it.org/local-it/mitgliederverwaltung/issues/386) - CustomField Resource Policies (closed 2026-01-27) +- ✅ [#369](https://git.local-it.org/local-it/mitgliederverwaltung/issues/369) - CustomFieldValue Resource Policies (closed 2026-01-27) +- ✅ [#363](https://git.local-it.org/local-it/mitgliederverwaltung/issues/363) - User Resource Policies (closed 2026-01-27) --- @@ -73,9 +81,24 @@ - ✅ User-Member linking (optional 1:1) - ✅ Email synchronization between User and Member - ✅ **Bulk email copy** - Copy selected members' email addresses to clipboard (Issue #230) +- ✅ **Groups** - Organize members into groups (PR #378, #382, closes #371, #372, 2026-01-27) + - Many-to-many relationship with groups + - Groups management UI (`/groups`) + - Filter and sort by groups in member list + - Groups displayed in member overview and detail views +- ✅ **CSV Import** - Import members from CSV files (PR #359, #394, #395, closes #335, #336, #338, 2026-01-27) + - Member field import + - Custom field value import + - Real-time progress tracking + - Error reporting **Closed Issues:** - ✅ [#162](https://git.local-it.org/local-it/mitgliederverwaltung/issues/162) - Fuzzy and substring search (closed 2025-11-12) +- ✅ [#371](https://git.local-it.org/local-it/mitgliederverwaltung/issues/371) - Add groups resource (closed 2026-01-27) +- ✅ [#372](https://git.local-it.org/local-it/mitgliederverwaltung/issues/372) - Groups Admin UI (closed 2026-01-27) +- ✅ [#335](https://git.local-it.org/local-it/mitgliederverwaltung/issues/335) - CSV Import UI (closed 2026-01-27) +- ✅ [#336](https://git.local-it.org/local-it/mitgliederverwaltung/issues/336) - Config for import limits (closed 2026-01-27) +- ✅ [#338](https://git.local-it.org/local-it/mitgliederverwaltung/issues/338) - Custom field CSV import (closed 2026-01-27) **Open Issues:** - [#169](https://git.local-it.org/local-it/mitgliederverwaltung/issues/169) - Allow combined creation of Users/Members (M, Low priority) @@ -88,7 +111,7 @@ - ❌ Advanced filters (date ranges, multiple criteria) - ❌ Pagination (currently all members loaded) - ❌ Bulk operations (bulk delete, bulk update) -- ❌ Member import/export (CSV, Excel) +- ❌ Excel import for members - ❌ Member profile photos/avatars - ❌ Member history/audit log - ❌ Duplicate detection @@ -288,12 +311,24 @@ - ✅ **CSV Import Templates** - German and English templates (#329, 2026-01-13) - Template files in `priv/static/templates/member_import_de.csv` and `member_import_en.csv` - CSV specification documented in `docs/csv-member-import-v1.md` +- ✅ **CSV Import Implementation** - Full CSV import feature (#335, #336, #338, 2026-01-27) + - Import/Export LiveView (`/import_export`) + - Member field import (email, first_name, last_name, etc.) + - Custom field value import (all types: string, integer, boolean, date, email) + - Real-time progress tracking + - Error and warning reporting with line numbers + - Configurable limits (max file size, max rows) + - Chunked processing (200 rows per chunk) + - Admin-only access + +**Closed Issues:** +- ✅ [#335](https://git.local-it.org/local-it/mitgliederverwaltung/issues/335) - CSV Import UI (closed 2026-01-27) +- ✅ [#336](https://git.local-it.org/local-it/mitgliederverwaltung/issues/336) - Config for import limits (closed 2026-01-27) +- ✅ [#338](https://git.local-it.org/local-it/mitgliederverwaltung/issues/338) - Custom field CSV import (closed 2026-01-27) **Missing Features:** -- ❌ CSV import implementation (templates ready, import logic pending) - ❌ Excel import for members -- ❌ Import validation and preview -- ❌ Import error handling +- ❌ Import validation preview (before import) - ❌ Bulk data export - ❌ Backup export - ❌ Data migration tools