Cleanup of the docs closes #507 #530

Merged
moritz merged 6 commits from issue/mitgliederverwaltung-507 into main 2026-06-15 22:00:43 +02:00
5 changed files with 164 additions and 3277 deletions
Showing only changes of commit 6fddb5285b - Show all commits

File diff suppressed because it is too large Load diff

View file

@ -6,16 +6,11 @@
--- ---
## Table of Contents This is the living per-area roadmap: shipped state (coarse — see `development-progress-log.md` for detail), open issues, and the missing-features backlog. For the actual, current endpoints see `lib/mv_web/router.ex` and `docs/page-permission-route-coverage.md`.
1. [Phase 1: Feature Area Breakdown](#phase-1-feature-area-breakdown)
2. [Phase 2: API Endpoint Definition](#phase-2-api-endpoint-definition)
3. [Phase 3: Implementation Task Creation](#phase-3-implementation-task-creation)
4. [Phase 4: Task Organization and Prioritization](#phase-4-task-organization-and-prioritization)
--- ---
## Phase 1: Feature Area Breakdown ## Feature Area Breakdown
### Feature Areas ### Feature Areas
@ -49,10 +44,10 @@
- ✅ **Page-level authorization** - LiveView page access control - ✅ **Page-level authorization** - LiveView page access control
- ✅ **System role protection** - Critical roles cannot be deleted - ✅ **System role protection** - Critical roles cannot be deleted
**Planned: OIDC-only mode (TDD, tests first):** **Implemented: OIDC-only mode:**
- Admin Settings: When OIDC-only is enabled, disable "Allow direct registration" toggle and show hint (tests in `GlobalSettingsLiveTest`). - ✅ Admin Settings: when OIDC-only is enabled, the "Allow direct registration" toggle is disabled with a hint.
- Backend: Reject password sign-in and `register_with_password` when OIDC-only (tests in `AuthControllerTest`, `Accounts`). - ✅ Backend rejects password sign-in and `register_with_password` when OIDC-only is active.
- GET `/sign-in` redirect to OIDC when OIDC-only and OIDC configured (tests in `AuthControllerTest`). Implementation to follow after tests. - ✅ GET `/sign-in` redirects to OIDC when OIDC-only and OIDC are configured (`MvWeb.Plugs.OidcOnlySignInRedirect`). The `oidc_only` setting and ENV are read via `Mv.Config.oidc_only?/0`.
**Missing Features:** **Missing Features:**
- ❌ Password reset flow - ❌ Password reset flow
@ -183,6 +178,11 @@
- ✅ Navbar with profile button - ✅ Navbar with profile button
- ✅ Member list as landing page - ✅ Member list as landing page
- ✅ Breadcrumbs (basic) - ✅ Breadcrumbs (basic)
- ✅ **Flash: auto-dismiss and consistency** (Design Guidelines §9)
- Auto-dismiss implemented via the `FlashAutoDismiss` JS hook (`assets/js/app.js`) driven by the `data-auto-clear-ms` and `data-clear-flash-key` attributes on the flash component (`MvWeb.CoreComponents.flash/1`); the per-flash delay is set through the component's `auto_clear_ms` attribute, and the dismiss button is kept for accessibility.
- On timeout the hook pushes LiveView's built-in `lv:clear-flash` event (no custom `handle_event`) and hides the element.
- All flashes (including “Email copied”) use the same variants (info, success, warning, error); no special tone. See `DESIGN_GUIDELINES.md` §9.
- ❌ Per-kind default durations (info/success 46s, warning 68s, error 812s) are not built in — the delay is a single explicit `auto_clear_ms` value per flash, not a kind-based default.
**Open Issues:** **Open Issues:**
- [#188](https://git.local-it.org/local-it/mitgliederverwaltung/issues/188) - Check if searching just on typing is accessible (S, Low priority) - [#188](https://git.local-it.org/local-it/mitgliederverwaltung/issues/188) - Check if searching just on typing is accessible (S, Low priority)
@ -196,11 +196,6 @@
- ❌ Mobile navigation - ❌ Mobile navigation
- ❌ Context-sensitive help - ❌ Context-sensitive help
- ❌ Onboarding tooltips - ❌ Onboarding tooltips
- ❌ **Flash: Auto-dismiss and consistency** (Design Guidelines §9)
- Auto-dismiss: info/success 46s, warning 68s, error 812s; dismiss button kept for accessibility.
- Implement via JS hook (e.g. `FlashAutoDismiss`) + `data-dismiss-ms` (or `data-kind`) on flash component; on timeout push `lv:clear-flash` and hide element.
- LiveView: add shared `handle_event("lv:clear-flash", %{"key" => key}, socket)` (e.g. in `MvWeb` live_view quote) calling `clear_flash(socket, key)`.
- All flashes (including “Email copied”) use the same variants (info, success, warning, error); no special tone. See `DESIGN_GUIDELINES.md` §9.
--- ---
@ -308,7 +303,12 @@
#### 10. **Reporting & Analytics** 📊 #### 10. **Reporting & Analytics** 📊
**Current State:** **Current State:**
- ✅ **Statistics page (MVP)** `/statistics` with active/inactive member counts, joins/exits by year, cycle totals, open amount (2026-02-10) - ✅ **Statistics page (MVP)** `/statistics` with active/inactive member counts, joins/exits by year, cycle totals, open amount (2026-02-10). Backed by `Mv.Statistics` (read-only Ash reads on `Member` + `MembershipFeeCycle`, no new resources); displayed in `MvWeb.StatisticsLive`. Permission: read_only, normal_user, admin (own_data denied).
**MVP design decisions:**
- Charts are HTML/CSS + SVG only — no Contex, no Chart.js (deliberate).
- Open amount = total unpaid only; no overdue vs. not-yet-due split in the MVP.
- Out of scope (deferred follow-ups): export (CSV/PDF), caching, month/quarter filters, "members per fee type" / "members per group" stats, overdue split, new tables/resources.
**Missing Features:** **Missing Features:**
- ❌ Extended member statistics dashboard - ❌ Extended member statistics dashboard
@ -487,367 +487,13 @@
--- ---
--- ## Endpoints
## Phase 2: API Endpoint Definition
### Endpoint Types
Since this is a **Phoenix LiveView** application with **Ash Framework**, we have three types of endpoints:
1. **LiveView Endpoints** - Mount points and event handlers
2. **HTTP Controller Endpoints** - Traditional REST-style endpoints
3. **Ash Resource Actions** - Backend data layer API
### Authentication Requirements Legend
- 🔓 **Public** - No authentication required
- 🔐 **Authenticated** - Requires valid user session
- 👤 **User Role** - Requires specific user role
- 🛡️ **Admin Only** - Requires admin privileges
---
### 1. Authentication & Authorization Endpoints
#### HTTP Controller Endpoints
| Method | Route | Purpose | Auth | Request | Response |
|--------|-------|---------|------|---------|----------|
| `GET` | `/auth/user/password/sign_in` | Show password login form | 🔓 | - | HTML form |
| `POST` | `/auth/user/password/sign_in` | Submit password login | 🔓 | `{email, password}` | Redirect + session cookie |
| `GET` | `/auth/user/oidc` | Initiate OIDC flow | 🔓 | - | Redirect to Rauthy |
| `GET` | `/auth/user/oidc/callback` | Handle OIDC callback | 🔓 | `{code, state}` | Redirect + session cookie |
| `POST` | `/auth/user/sign_out` | Sign out user | 🔐 | - | Redirect to login |
| `GET` | `/auth/link-oidc-account` | OIDC account linking (password verification) | 🔓 | - | LiveView form | ✅ Implemented |
| `GET` | `/auth/user/password/reset` | Show password reset form | 🔓 | - | HTML form |
| `POST` | `/auth/user/password/reset` | Request password reset | 🔓 | `{email}` | Success message + email sent |
| `GET` | `/auth/user/password/reset/:token` | Show reset password form | 🔓 | - | HTML form |
| `POST` | `/auth/user/password/reset/:token` | Submit new password | 🔓 | `{password, password_confirmation}` | Redirect to login |
#### Ash Resource Actions
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `User` | `:sign_in_with_password` | Password authentication | 🔓 | `{email, password}` | `{:ok, user}` or `{:error, reason}` |
| `User` | `:sign_in_with_oidc` | OIDC authentication | 🔓 | `{oidc_id, email, user_info}` | `{:ok, user}` or `{:error, reason}` |
| `User` | `:register_with_password` | Create user with password | 🔓 | `{email, password}` | `{:ok, user}` |
| `User` | `:register_with_oidc` | Create user via OIDC | 🔓 | `{oidc_id, email}` | `{:ok, user}` |
| `User` | `:request_password_reset` | Generate reset token | 🔓 | `{email}` | `{:ok, token}` |
| `User` | `:reset_password` | Reset password with token | 🔓 | `{token, password}` | `{:ok, user}` |
| `Token` | `:revoke` | Revoke authentication token | 🔐 | `{jti}` | `{:ok, token}` |
#### **NEW: Role & Permission Actions** (Issue #191, #190, #151)
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `Role` | `:create` | Create new role | 🛡️ | `{name, description, permissions}` | `{:ok, role}` |
| `Role` | `:list` | List all roles | 🔐 | - | `[%Role{}]` |
| `Role` | `:update` | Update role | 🛡️ | `{id, name, permissions}` | `{:ok, role}` |
| `Role` | `:delete` | Delete role | 🛡️ | `{id}` | `{:ok, role}` |
| `User` | `:assign_role` | Assign role to user | 🛡️ | `{user_id, role_id}` | `{:ok, user}` |
| `User` | `:remove_role` | Remove role from user | 🛡️ | `{user_id, role_id}` | `{:ok, user}` |
| `Permission` | `:list` | List all permissions | 🔐 | - | `[%Permission{}]` |
| `Permission` | `:check` | Check user permission | 🔐 | `{user_id, resource, action}` | `{:ok, boolean}` |
---
### 2. Member Management Endpoints
#### LiveView Endpoints
| Mount | Purpose | Auth | Query Params | Events |
|-------|---------|------|--------------|--------|
| `/members` | Member list with search/sort | 🔐 | `?search=&sort_by=&sort_dir=` | `search`, `sort`, `delete`, `select` |
| `/members/new` | Create new member form | 🔐 | - | `save`, `cancel`, `add_custom_field_value` |
| `/members/:id` | Member detail view | 🔐 | - | `edit`, `delete`, `link_user` |
| `/members/:id/edit` | Edit member form | 🔐 | - | `save`, `cancel`, `add_custom_field_value`, `remove_custom_field_value` |
#### LiveView Event Handlers
| Event | Purpose | Params | Response |
|-------|---------|--------|----------|
| `search` | Trigger search | `%{"search" => query}` | Update member list |
| `sort` | Sort member list | `%{"field" => field}` | Update sorted list |
| `delete` | Delete member | `%{"id" => id}` | Redirect to list |
| `save` | Create/update member | `%{"member" => attrs}` | Redirect or show errors |
| `link_user` | Link user to member | `%{"user_id" => id}` | Update member view |
| `unlink_user` | Unlink user from member | - | Update member view |
| `add_custom_field_value` | Add custom field value | `%{"custom_field_id" => id, "value" => val}` | Update form |
| `remove_custom_field_value` | Remove custom field value | `%{"custom_field_value_id" => id}` | Update form |
#### Ash Resource Actions
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `Member` | `:create_member` | Create member | 🔐 | `{first_name, last_name, email, ...}` | `{:ok, member}` |
| `Member` | `:read` | List/search members | 🔐 | `{search, sort_by, limit, offset}` | `[%Member{}]` |
| `Member` | `:update_member` | Update member | 🔐 | `{id, attrs}` | `{:ok, member}` |
| `Member` | `:destroy` | Delete member | 🔐 | `{id}` | `{:ok, member}` |
| `Member` | `:search_fulltext` | Full-text search | 🔐 | `{query}` | `[%Member{}]` |
| `Member` | `:link_to_user` | Link member to user | 🔐 | `{member_id, user_id}` | `{:ok, member}` |
| `Member` | `:unlink_from_user` | Unlink from user | 🔐 | `{member_id}` | `{:ok, member}` |
#### **NEW: Enhanced Search & Filter Actions** (Issue #162, #154, #165)
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `Member` | `:fuzzy_search` | Fuzzy text search | 🔐 | `{query, threshold}` | `[%Member{}]` |
| `Member` | `:advanced_search` | Multi-criteria search | 🔐 | `{filters: [{field, op, value}]}` | `[%Member{}]` |
| `Member` | `:paginate` | Paginated member list | 🔐 | `{page, per_page, filters}` | `{members, total, page_info}` |
| `Member` | `:sort_by_custom_field` | Sort by custom field | 🔐 | `{custom_field_id, direction}` | `[%Member{}]` |
| `Member` | `:bulk_delete` | Delete multiple members | 🛡️ | `{ids: [id1, id2, ...]}` | `{:ok, count}` |
| `Member` | `:bulk_update` | Update multiple members | 🛡️ | `{ids, attrs}` | `{:ok, count}` |
| `Member` | `:export` | Export to CSV/Excel | 🔐 | `{format, filters}` | File download |
| `Member` | `:import` | Import from CSV | 🛡️ | `{file, mapping}` | `{:ok, imported_count, errors}` |
---
### 3. Custom Fields (CustomFieldValue System) Endpoints
#### LiveView Endpoints (✅ Implemented)
| Mount | Purpose | Auth | Events | Status |
|-------|---------|------|--------|--------|
| `/settings` | Global settings (includes custom fields management) | 🔐 | `save`, `validate` | ✅ Implemented |
| `/custom_field_values` | List all custom field values | 🔐 | `new`, `edit`, `delete` | ✅ Implemented |
| `/custom_field_values/new` | Create custom field value | 🔐 | `save`, `cancel` | ✅ Implemented |
| `/custom_field_values/:id` | Custom field value detail | 🔐 | `edit` | ✅ Implemented |
| `/custom_field_values/:id/edit` | Edit custom field value | 🔐 | `save`, `cancel` | ✅ Implemented |
| `/custom_field_values/:id/show/edit` | Edit from show page | 🔐 | `save`, `cancel` | ✅ Implemented |
**Note:** Custom fields (definitions) are managed via LiveComponent in `/settings` page, not as separate routes.
#### Ash Resource Actions
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `CustomField` | `:create` | Create custom field | 🛡️ | `{name, value_type, description, ...}` | `{:ok, custom_field}` |
| `CustomField` | `:read` | List custom fields | 🔐 | - | `[%CustomField{}]` |
| `CustomField` | `:update` | Update custom field | 🛡️ | `{id, attrs}` | `{:ok, custom_field}` |
| `CustomField` | `:destroy` | Delete custom field | 🛡️ | `{id}` | `{:ok, custom_field}` |
| `CustomFieldValue` | `:create` | Add custom field value to member | 🔐 | `{member_id, custom_field_id, value}` | `{:ok, custom_field_value}` |
| `CustomFieldValue` | `:update` | Update custom field value | 🔐 | `{id, value}` | `{:ok, custom_field_value}` |
| `CustomFieldValue` | `:destroy` | Remove custom field value | 🔐 | `{id}` | `{:ok, custom_field_value}` |
#### **NEW: Enhanced Custom Fields** (Issue #194, #157, #161, #153)
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `CustomField` | `:set_default_visibility` | Show/hide by default | 🛡️ | `{id, visible}` | `{:ok, custom_field}` |
| `CustomField` | `:set_required` | Mark as required | 🛡️ | `{id, required}` | `{:ok, custom_field}` |
| `CustomField` | `:add_validation` | Add validation rule | 🛡️ | `{id, rule_type, params}` | `{:ok, custom_field}` |
| `CustomField` | `:create_group` | Create field group | 🛡️ | `{name, custom_field_ids}` | `{:ok, group}` |
| `CustomFieldValue` | `:validate_value` | Validate custom field value | 🔐 | `{custom_field_id, value}` | `{:ok, valid}` or `{:error, reason}` |
---
### 4. User Management Endpoints
#### LiveView Endpoints
| Mount | Purpose | Auth | Events |
|-------|---------|------|--------|
| `/users` | User list | 🛡️ | `new`, `edit`, `delete`, `assign_role` |
| `/users/new` | Create user form | 🛡️ | `save`, `cancel` |
| `/users/:id` | User detail view | 🔐 | `edit`, `delete`, `change_password` |
| `/users/:id/edit` | Edit user form | 🔐 | `save`, `cancel`, `link_member` |
| `/profile` | Current user profile | 🔐 | `edit`, `change_password` |
#### Ash Resource Actions
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `User` | `:create_user` | Create user (admin) | 🛡️ | `{email, member_id?}` | `{:ok, user}` |
| `User` | `:read` | List users | 🛡️ | - | `[%User{}]` |
| `User` | `:update_user` | Update user | 🔐 | `{id, email, member_id?}` | `{:ok, user}` |
| `User` | `:destroy` | Delete user | 🛡️ | `{id}` | `{:ok, user}` |
| `User` | `:admin_set_password` | Set password (admin) | 🛡️ | `{id, password}` | `{:ok, user}` |
| `User` | `:change_password` | Change own password | 🔐 | `{current_password, new_password}` | `{:ok, user}` |
#### **NEW: Combined User/Member Management** (Issue #169, #168)
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `User` | `:create_with_member` | Create user + member together | 🛡️ | `{user: {...}, member: {...}}` | `{:ok, %{user, member}}` |
| `User` | `:invite_user` | Send invitation email | 🛡️ | `{email, role_id, member_id?}` | `{:ok, invitation}` |
| `User` | `:accept_invitation` | Accept invitation | 🔓 | `{token, password}` | `{:ok, user}` |
---
### 5. Navigation & UX Endpoints
#### LiveView Endpoints
| Mount | Purpose | Auth | Events |
|-------|---------|------|--------|
| `/` | Dashboard/Home | 🔐 | - |
| `/dashboard` | Dashboard view | 🔐 | Contextual based on role |
#### HTTP Controller Endpoints
| Method | Route | Purpose | Auth | Request | Response |
|--------|-------|---------|------|---------|----------|
| `GET` | `/health` | Health check | 🔓 | - | `{"status": "ok"}` |
| `GET` | `/` | Root redirect | - | - | Redirect to dashboard or login |
---
### 6. Internationalization Endpoints
#### HTTP Controller Endpoints (✅ Implemented)
| Method | Route | Purpose | Auth | Request | Response | Status |
|--------|-------|---------|------|---------|----------|--------|
| `POST` | `/set_locale` | Set user locale | 🔐 | `{locale: "de"}` | Redirect with cookie | ✅ Implemented |
| `GET` | `/locales` | List available locales | 🔓 | - | `["de", "en"]` | ❌ Not implemented |
**Note:** Locale is set via `/set_locale` POST endpoint and persisted in session/cookie. Supported locales: `de` (default), `en`.
---
### 7. Payment & Fees Management Endpoints
#### LiveView Endpoints (✅ Implemented)
| Mount | Purpose | Auth | Events | Status |
|-------|---------|------|--------|--------|
| `/membership_fee_types` | Membership fee type list | 🔐 | `new`, `edit`, `delete` | ✅ Implemented |
| `/membership_fee_types/new` | Create membership fee type | 🔐 | `save`, `cancel` | ✅ Implemented |
| `/membership_fee_types/:id/edit` | Edit membership fee type | 🔐 | `save`, `cancel` | ✅ Implemented |
| `/membership_fee_settings` | Global membership fee settings | 🔐 | `save` | ✅ Implemented |
| `/contributions/member/:id` | Member contribution periods (mock-up) | 🔐 | - | ⚠️ Mock-up only |
| `/contribution_types` | Contribution types (mock-up) | 🔐 | - | ⚠️ Mock-up only |
#### Ash Resource Actions (✅ Partially Implemented)
| Resource | Action | Purpose | Auth | Input | Output | Status |
|----------|--------|---------|------|-------|--------|--------|
| `MembershipFeeType` | `:create` | Create fee type | 🔐 | `{name, amount, interval, ...}` | `{:ok, fee_type}` | ✅ Implemented |
| `MembershipFeeType` | `:read` | List fee types | 🔐 | - | `[%MembershipFeeType{}]` | ✅ Implemented |
| `MembershipFeeType` | `:update` | Update fee type (name, amount, description) | 🔐 | `{id, attrs}` | `{:ok, fee_type}` | ✅ Implemented |
| `MembershipFeeType` | `:destroy` | Delete fee type (if no cycles) | 🔐 | `{id}` | `{:ok, fee_type}` | ✅ Implemented |
| `MembershipFeeCycle` | `:read` | List cycles for member | 🔐 | `{member_id}` | `[%MembershipFeeCycle{}]` | ✅ Implemented |
| `MembershipFeeCycle` | `:update` | Update cycle status | 🔐 | `{id, status}` | `{:ok, cycle}` | ✅ Implemented |
| `Payment` | `:create` | Record payment | 🔐 | `{member_id, fee_id, amount, date}` | `{:ok, payment}` | ❌ Not implemented |
| `Payment` | `:list_by_member` | Member payment history | 🔐 | `{member_id}` | `[%Payment{}]` | ❌ Not implemented |
| `Payment` | `:mark_paid` | Mark as paid | 🔐 | `{id}` | `{:ok, payment}` | ❌ Not implemented |
| `Invoice` | `:generate` | Generate invoice | 🔐 | `{member_id, fee_id, period}` | `{:ok, invoice}` | ❌ Not implemented |
| `Invoice` | `:send` | Send invoice via email | 🔐 | `{id}` | `{:ok, sent}` | ❌ Not implemented |
| `Payment` | `:import_vereinfacht` | Import from vereinfacht.digital | 🛡️ | `{transactions}` | `{:ok, count}` | ❌ Not implemented |
---
### 8. Admin Panel & Configuration Endpoints
#### LiveView Endpoints (✅ Partially Implemented)
| Mount | Purpose | Auth | Events | Status |
|-------|---------|------|--------|--------|
| `/settings` | Global settings (club name, member fields, custom fields) | 🔐 | `save`, `validate` | ✅ Implemented |
| `/admin/roles` | Role management | 🛡️ | `new`, `edit`, `delete` | ✅ Implemented |
| `/admin/roles/new` | Create role | 🛡️ | `save`, `cancel` | ✅ Implemented |
| `/admin/roles/:id` | Role detail view | 🛡️ | `edit` | ✅ Implemented |
| `/admin/roles/:id/edit` | Edit role | 🛡️ | `save`, `cancel` | ✅ Implemented |
| `/admin` | Admin dashboard | 🛡️ | - | ❌ Not implemented |
| `/admin/organization` | Organization profile | 🛡️ | `save` | ❌ Not implemented |
| `/admin/email-templates` | Email template editor | 🛡️ | `create`, `edit`, `preview` | ❌ Not implemented |
| `/admin/audit-log` | System audit log | 🛡️ | `filter`, `export` | ❌ Not implemented |
#### Ash Resource Actions (✅ Partially Implemented)
| Resource | Action | Purpose | Auth | Input | Output | Status |
|----------|--------|---------|------|-------|--------|--------|
| `Setting` | `:read` | Get settings (singleton) | 🔐 | - | `{:ok, settings}` | ✅ Implemented |
| `Setting` | `:update` | Update settings | 🔐 | `{club_name, member_field_visibility, ...}` | `{:ok, settings}` | ✅ Implemented |
| `Setting` | `:update_member_field_visibility` | Update field visibility | 🔐 | `{member_field_visibility}` | `{:ok, settings}` | ✅ Implemented |
| `Setting` | `:update_single_member_field_visibility` | Atomic field visibility update | 🔐 | `{field, show_in_overview}` | `{:ok, settings}` | ✅ Implemented |
| `Setting` | `:update_membership_fee_settings` | Update fee settings | 🔐 | `{include_joining_cycle, default_membership_fee_type_id}` | `{:ok, settings}` | ✅ Implemented |
| `Role` | `:read` | List roles | 🛡️ | - | `[%Role{}]` | ✅ Implemented |
| `Role` | `:create` | Create role | 🛡️ | `{name, permission_set_name, ...}` | `{:ok, role}` | ✅ Implemented |
| `Role` | `:update` | Update role | 🛡️ | `{id, attrs}` | `{:ok, role}` | ✅ Implemented |
| `Role` | `:destroy` | Delete role (if not system role) | 🛡️ | `{id}` | `{:ok, role}` | ✅ Implemented |
| `Organization` | `:read` | Get organization info | 🔐 | - | `%Organization{}` | ❌ Not implemented |
| `Organization` | `:update` | Update organization | 🛡️ | `{name, logo, ...}` | `{:ok, org}` | ❌ Not implemented |
| `AuditLog` | `:list` | List audit entries | 🛡️ | `{filters, pagination}` | `[%AuditLog{}]` | ❌ Not implemented |
---
### 9. Communication & Notifications Endpoints
#### LiveView Endpoints (NEW)
| Mount | Purpose | Auth | Events |
|-------|---------|------|--------|
| `/communications` | Communication history | 🔐 | `new`, `view` |
| `/communications/new` | Create email broadcast | 🔐 | `select_recipients`, `preview`, `send` |
| `/notifications` | User notifications | 🔐 | `mark_read`, `mark_all_read` |
#### Ash Resource Actions (NEW)
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `EmailBroadcast` | `:create` | Create broadcast | 🔐 | `{subject, body, recipient_filter}` | `{:ok, broadcast}` |
| `EmailBroadcast` | `:send` | Send broadcast | 🔐 | `{id}` | `{:ok, sent_count}` |
| `EmailTemplate` | `:create` | Create template | 🛡️ | `{name, subject, body}` | `{:ok, template}` |
| `EmailTemplate` | `:render` | Render template | 🔐 | `{id, variables}` | `rendered_html` |
| `Notification` | `:create` | Create notification | System | `{user_id, type, message}` | `{:ok, notification}` |
| `Notification` | `:list_for_user` | Get user notifications | 🔐 | `{user_id}` | `[%Notification{}]` |
| `Notification` | `:mark_read` | Mark as read | 🔐 | `{id}` | `{:ok, notification}` |
---
### 10. Reporting & Analytics Endpoints
#### LiveView Endpoints (NEW)
| Mount | Purpose | Auth | Events |
|-------|---------|------|--------|
| `/reports` | Reports dashboard | 🔐 | `generate`, `schedule` |
| `/reports/members` | Member statistics | 🔐 | `filter`, `export` |
| `/reports/payments` | Payment reports | 🔐 | `filter`, `export` |
| `/reports/custom` | Custom report builder | 🛡️ | `build`, `save`, `run` |
#### Ash Resource Actions (NEW)
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `Report` | `:generate_member_stats` | Member statistics | 🔐 | `{date_range, filters}` | Statistics object |
| `Report` | `:generate_payment_stats` | Payment statistics | 🔐 | `{date_range}` | Statistics object |
| `Report` | `:export_to_csv` | Export report to CSV | 🔐 | `{report_type, filters}` | CSV file |
| `Report` | `:export_to_pdf` | Export report to PDF | 🔐 | `{report_type, filters}` | PDF file |
| `Report` | `:schedule` | Schedule recurring report | 🛡️ | `{report_type, frequency, recipients}` | `{:ok, schedule}` |
---
### 11. Data Import/Export Endpoints
#### LiveView Endpoints (NEW)
| Mount | Purpose | Auth | Events |
|-------|---------|------|--------|
| `/import` | Data import wizard | 🛡️ | `upload`, `map_fields`, `preview`, `import` |
| `/export` | Data export tool | 🔐 | `select_data`, `configure`, `export` |
#### Ash Resource Actions (NEW)
| Resource | Action | Purpose | Auth | Input | Output |
|----------|--------|---------|------|-------|--------|
| `Member` | `:import_csv` | Import members from CSV | 🛡️ | `{file, field_mapping}` | `{:ok, imported, errors}` |
| `Member` | `:validate_import` | Validate import data | 🛡️ | `{file, field_mapping}` | `{:ok, validation_results}` |
| `Member` | `:export_csv` | Export members to CSV | 🔐 | `{filters}` | CSV file |
| `Member` | `:export_excel` | Export members to Excel | 🔐 | `{filters}` | Excel file |
| `Database` | `:export_backup` | Full database backup | 🛡️ | - | Backup file |
| `Database` | `:import_backup` | Restore from backup | 🛡️ | `{file}` | `{:ok, restored}` |
---
For the real, current routes and their authorization, see `lib/mv_web/router.ex` and `docs/page-permission-route-coverage.md` (the per-permission-set route matrix). The Ash resource actions are defined on each resource module under `lib/`. An earlier speculative API catalog for not-yet-existing resources (Payment, Invoice, Report, Notification, AuditLog, Organization) was removed — those are tracked above as missing features per area, not as endpoint specs.
--- ---
**References:** **References:**
- Open Issues: https://git.local-it.org/local-it/mitgliederverwaltung/issues - Open Issues: https://git.local-it.org/local-it/mitgliederverwaltung/issues
- Project Board: Sprint 8 (23.10 - 13.11)
- Architecture: See [`CODE_GUIDELINES.md`](../CODE_GUIDELINES.md) - Architecture: See [`CODE_GUIDELINES.md`](../CODE_GUIDELINES.md)
- Database Schema: See [`database-schema-readme.md`](database-schema-readme.md) - Database Schema: See [`database-schema-readme.md`](database-schema-readme.md)

View file

@ -125,25 +125,7 @@ Verify mode is set in `tls_options` for port 587 (STARTTLS). For port 465 (impli
--- ---
## 12. Summary Checklist ## 12. Follow-up / Future Work
- [x] ENV: `SMTP_HOST`, `SMTP_PORT`, `SMTP_USERNAME`, `SMTP_PASSWORD`, `SMTP_PASSWORD_FILE`, `SMTP_SSL`.
- [x] ENV: `MAIL_FROM_NAME`, `MAIL_FROM_EMAIL` for sender identity.
- [x] Settings: attributes and UI for host, port, username, password, TLS/SSL, from-name, from-email.
- [x] Password from file: `SMTP_PASSWORD_FILE` supported in `runtime.exs`.
- [x] Mailer: Swoosh SMTP adapter configured from merged ENV + Settings when SMTP is configured.
- [x] Per-request SMTP config via `Mv.Mailer.smtp_config/0` for Settings-only scenarios.
- [x] TLS certificate validation relaxed for OTP 27 (tls_options for 587; sockopts with verify only for 465).
- [x] Prod warning: clear message in Settings when SMTP is not configured.
- [x] Test email: form with recipient field, translatable content, classified success/error messages.
- [x] Join confirmation email: uses `Mailer.smtp_config/0` (same as test mail); on failure returns `{:error, :email_delivery_failed}`, error shown in JoinLive, logged for admin.
- [x] AshAuthentication senders: graceful error handling (no crash on delivery failure).
- [x] Gettext for all new UI strings, translated to German.
- [x] Docs and code guidelines updated.
---
## 13. Follow-up / Future Work
- **SMTP password at-rest encryption:** The `smtp_password` attribute is currently stored in plaintext in the `settings` table. It is excluded from default reads (same pattern as `oidc_client_secret`); both are read only via explicit select when needed. For production systems at-rest encryption (e.g. with [Cloak](https://hexdocs.pm/cloak)) should be considered and tracked as a follow-up issue. - **SMTP password at-rest encryption:** The `smtp_password` attribute is currently stored in plaintext in the `settings` table. It is excluded from default reads (same pattern as `oidc_client_secret`); both are read only via explicit select when needed. For production systems at-rest encryption (e.g. with [Cloak](https://hexdocs.pm/cloak)) should be considered and tracked as a follow-up issue.
- **Error classification:** SMTP error categorization currently uses substring matching on server messages (e.g. "535", "authentication"). A more robust approach would be to pattern-match on `gen_smtp` error tuples first where possible, and fall back to string analysis only when needed. Server wording varies; consider extending patterns as new providers are used. - **Error classification:** SMTP error categorization currently uses substring matching on server messages (e.g. "535", "authentication"). A more robust approach would be to pattern-match on `gen_smtp` error tuples first where possible, and fall back to string analysis only when needed. Server wording varies; consider extending patterns as new providers are used.

View file

@ -1,163 +0,0 @@
# Statistics Page Implementation Plan
**Project:** Mila Membership Management System
**Feature:** Statistics page at `/statistics`
**Scope:** MVP only (no export, no optional extensions)
**Last updated:** 2026-02-10
---
## Decisions (from clarification)
| Topic | Decision |
|-------|----------|
| Route | `/statistics` |
| Navigation | Top-level menu (next to Members, Fee Types) |
| Permission | read_only, normal_user, admin (same as member list) |
| Charts | HTML/CSS and SVG only (no Contex, no Chart.js) |
| MVP scope | Minimal: active/inactive, joins/exits per year, contribution sums per year, open amount |
| Open amount | Total unpaid only (no overdue vs. not-yet-due split in MVP) |
Excluded from this plan: Export (CSV/PDF), caching, month/quarter filters, “members per fee type”, “members per group”, and overdue split.
---
## 1. Statistics module (`Mv.Statistics`)
**Goal:** Central module for all statistics; LiveView only calls this API. Uses Ash reads with actor so policies apply.
**Location:** `lib/mv/statistics.ex` (new).
**Functions to implement:**
| Function | Purpose | Data source |
|----------|---------|-------------|
| `active_member_count(opts)` | Count members with `exit_date == nil` | `Member` read with filter |
| `inactive_member_count(opts)` | Count members with `exit_date != nil` | `Member` read with filter |
| `joins_by_year(year, opts)` | Count members with `join_date` in given year | `Member` read, filter by year, count |
| `exits_by_year(year, opts)` | Count members with `exit_date` in given year | `Member` read, filter by year, count |
| `cycle_totals_by_year(year, opts)` | For cycles with `cycle_start` in year: total sum, and sums/counts by status (paid, unpaid, suspended) | `MembershipFeeCycle` read (filter by year via `cycle_start`), aggregate sum(amount) and count per status in Elixir or via Ash aggregates |
| `open_amount_total(opts)` | Sum of `amount` for all cycles with `status == :unpaid` | `MembershipFeeCycle` read with filter `status == :unpaid`, sum(amount) |
All functions accept `opts` (keyword list) and pass `actor: opts[:actor]` (and `domain:` where needed) to Ash calls. No new resources; only read actions on existing `Member` and `MembershipFeeCycle`.
**Implementation notes:**
- Use `Ash.Query.filter(Member, expr(...))` for date filters; for “year”, filter `join_date >= first_day_of_year` and `join_date <= last_day_of_year` (same for `exit_date` and for `MembershipFeeCycle.cycle_start`).
- For `cycle_totals_by_year`: either multiple Ash reads (one per status) with sum aggregate, or one read of cycles in that year and `Enum.group_by(..., :status)` then sum amounts in Elixir.
- Use `Mv.MembershipFees.CalendarCycles` only if needed for interval (e.g. cycle_end); for “cycle in year” the `cycle_start` year is enough.
**Tests:** Unit tests in `test/mv/statistics_test.exs` for each function (with fixtures: members with join_date/exit_date, cycles with cycle_start/amount/status). Use `Mv.Helpers.SystemActor.get_system_actor()` in tests for Ash read authorization where appropriate.
---
## 2. Route and authorization
**Router** ([lib/mv_web/router.ex](lib/mv_web/router.ex)):
- In the same `ash_authentication_live_session` block where `/members` and `/membership_fee_types` live, add:
- `live "/statistics", StatisticsLive, :index`
**PagePaths** ([lib/mv_web/page_paths.ex](lib/mv_web/page_paths.ex)):
- Add module attribute `@statistics "/statistics"`.
- Add `def statistics, do: @statistics`.
- No change to `@admin_page_paths` (statistics is top-level).
**Page permission** (route matrix is driven by [lib/mv/authorization/permission_sets.ex](lib/mv/authorization/permission_sets.ex)):
- Add `"/statistics"` to the `pages` list of **read_only** (e.g. after `"/groups/:slug"`) and to the `pages` list of **normal_user** (e.g. after groups entries). **admin** already has `"*"` so no change.
- **own_data** must not list `/statistics` (so they cannot access it).
- Update [docs/page-permission-route-coverage.md](docs/page-permission-route-coverage.md): add row for `| /statistics | ✗ | ✓ | ✓ | ✓ |`.
- Add test in `test/mv_web/plugs/check_page_permission_test.exs`: read_only and normal_user and admin can access `/statistics`; own_data cannot.
---
## 3. Sidebar
**File:** [lib/mv_web/components/layouts/sidebar.ex](lib/mv_web/components/layouts/sidebar.ex).
- In `sidebar_menu`, after the “Fee Types” menu item and before the “Administration” block, add a conditional menu item for Statistics:
- `can_access_page?(@current_user, PagePaths.statistics())` → show link.
- `href={~p"/statistics"}`, `icon="hero-chart-bar"` (or similar), `label={gettext("Statistics")}`.
---
## 4. Statistics LiveView
**Module:** `MvWeb.StatisticsLive`
**File:** `lib/mv_web/live/statistics_live.ex`
**Mount:** `:index` only.
**Behaviour:**
- `on_mount`: use `MvWeb.LiveUserAuth, :live_user_required` and ensure role/permission check (same as other protected LiveViews). In `mount` or `handle_params`, set default selected year to current year (e.g. `Date.utc_today().year`).
- **Assigns:** `:year` (integer), `:active_count`, `:inactive_count`, `:joins_this_year`, `:exits_this_year`, `:cycle_totals` (map with keys e.g. `:total`, `:paid`, `:unpaid`, `:suspended` for the selected year), `:open_amount_total`, and any extra needed for the bar data (e.g. list of `%{year: y, joins: j, exits: e}` for a small range of years if you show a minimal bar chart).
- **Year filter:** A single select or dropdown for year (e.g. from “first year with data” to current year). On change, send event (e.g. `"set_year"`) with `%{"year" => year}`; in `handle_event` update `assigns.year` and reload data by calling `Mv.Statistics` again and re-assigning.
**Data loading:**
- In `mount` and whenever year changes, call `Mv.Statistics` with `actor: current_actor(socket)` (and optionally `year: @year` where needed). Assign results to socket. Handle errors (e.g. redirect or flash) if a call fails.
**Layout (sections):**
1. **Page title:** e.g. “Statistics” (gettext).
2. **Year filter:** One control to select year; applies to “joins/exits” and “contribution sums” for that year.
3. **Cards (top row):**
- Active members (count)
- Inactive members (count)
- Joins in selected year
- Exits in selected year
- Open amount total (sum of all unpaid cycles; format with `MvWeb.Helpers.MembershipFeeHelpers.format_currency/1`)
- Optionally: “Paid this year” (from `cycle_totals_by_year` for selected year)
4. **Contributions for selected year:** One section showing for the chosen year: total (Soll), paid, unpaid, suspended (sums and optionally counts). Use simple table or key-value list; no chart required for MVP.
5. **Joins / Exits by year (simple bar chart):** Data: e.g. last 5 or 10 years. For each year, show joins and exits as horizontal bars (HTML/CSS: e.g. `div` with `width: #{percent}%`). Pure HTML/SVG; no external chart library. Use Tailwind/DaisyUI for layout and cards.
**Accessibility:** Semantic HTML; headings (e.g. `h2`) for each section; ensure year filter has a label; format numbers in a screen-reader-friendly way (e.g. no purely visual abbreviations without aria-label).
**i18n:** All user-visible strings via gettext (e.g. “Statistics”, “Active members”, “Inactive members”, “Joins (year)”, “Exits (year)”, “Open amount”, “Contributions for year”, “Total”, “Paid”, “Unpaid”, “Suspended”). Add keys to `priv/gettext` as needed.
---
## 5. Implementation order (tasks)
Execute in this order so that each step is testable:
1. **Statistics module**
- Add `lib/mv/statistics.ex` with the six functions above and `@moduledoc`.
- Add `test/mv/statistics_test.exs` with tests for each function (use fixtures for members and cycles; pass actor in opts).
- Run tests and fix until green.
2. **Route and permission**
- Add `live "/statistics", StatisticsLive, :index` in router.
- Add `statistics/0` and `@statistics` in PagePaths.
- Add `/statistics` to page permission logic so read_only, normal_user, admin are allowed and own_data is denied.
- Update `docs/page-permission-route-coverage.md` and add/update plug tests for `/statistics`.
3. **Sidebar**
- Add Statistics link in sidebar (top-level) with `can_access_page?` and `PagePaths.statistics()`.
4. **StatisticsLive**
- Create `lib/mv_web/live/statistics_live.ex` with mount, assigns, year param, and data loading from `Mv.Statistics`.
- Implement UI: title, year filter, cards, contribution section, simple joins/exits bar (HTML).
- Add gettext keys and use them in the template.
- Optionally add a simple LiveView test (e.g. authenticated user sees statistics page and key labels).
5. **CI and docs**
- Run `just ci-dev` (or project equivalent); fix formatting, Credo, and tests.
- In [docs/feature-roadmap.md](docs/feature-roadmap.md), update “Reporting & Analytics” to reflect that a basic statistics page is implemented (MVP).
- In [CODE_GUIDELINES.md](CODE_GUIDELINES.md), add a short note under a suitable section (e.g. “Reporting” or “LiveView”) that statistics are provided by `Mv.Statistics` and displayed in `StatisticsLive`, if desired.
---
## 6. Out of scope (not in this plan)
- Export (CSV/PDF).
- Caching (ETS/GenServer/HTTP).
- Month or quarter filters.
- “Members per fee type” or “members per group” statistics.
- Overdue vs. not-yet-due split for open amount.
- Contex or Chart.js.
- New database tables or Ash resources.
These can be added later as separate tasks or follow-up plans.

View file

@ -1,877 +1,98 @@
# Test Performance Optimization # Test Performance Optimization
**Last Updated:** 2026-01-28 **Last Updated:** 2026-01-28
**Status:** ✅ Active optimization program **Status:** Implemented
This document records the test-suite performance work and — most importantly — the conventions that govern how tests are tagged and run. The seeds-test rationale in §1 is the canonical reference linked from `test/seeds_test.exs`.
Baseline result: the standard (fast) suite runs ~368 s (~6.1 min) vs. ~445 s before; the full suite (all tests) ~7.4 min. Slow-tagged tests (~25, >1 s each, ~77 s total) are excluded from standard runs and executed via promotion before merge.
--- ---
## Executive Summary ## 1. Seeds Test Suite — coverage mapping
This document provides a comprehensive overview of test performance optimizations, risk assessments, and future opportunities. The test suite execution time has been reduced through systematic analysis and targeted optimizations. The seeds tests were reduced from 13 to 4. The 9 removed tests were dropped because their assertions are already covered by domain-specific test suites — this mapping is the justification and must be preserved:
### Current Performance Metrics | Removed seeds test | Covered by |
|--------------------|-----------|
| `"at least one member has no membership fee type assigned"` | `membership_fees/*_test.exs` |
| `"each membership fee type has at least one member"` | `membership_fees/*_test.exs` |
| `"members with fee types have cycles with various statuses"` | `cycle_generator_test.exs` |
| `"creates all 5 authorization roles with correct permission sets"` | `authorization/*_test.exs` |
| `"all roles have valid permission_set_names"` | `authorization/permission_sets_test.exs` |
| `"does not change role of users who already have a role"` | merged into general idempotency test |
| `"role creation is idempotent"` (detailed) | merged into general idempotency test |
| Metric | Value | ### 4 retained critical-bootstrap tests (and why)
|--------|-------|
| **Total Execution Time** (without `:slow` tests) | ~368 seconds (~6.1 minutes) |
| **Total Tests** | 1,336 tests (+ 25 doctests) |
| **Async Execution** | 163.5 seconds |
| **Sync Execution** | 281.5 seconds |
| **Slow Tests Excluded** | 25 tests (tagged with `@tag :slow`) |
| **Top 50 Slowest Tests** | 121.9 seconds (27.4% of total time) |
### Optimization Impact Summary These guard deployment-critical invariants that nothing else covers and must stay in the **fast** suite:
| Optimization | Tests Affected | Time Saved | Status | 1. **Smoke test** — seeds run successfully and create basic data.
|--------------|----------------|------------|--------| 2. **Idempotency** — seeds can be re-run without duplicating data.
| Seeds tests reduction | 13 → 4 tests | ~10-16s | ✅ Completed | 3. **Admin bootstrap** — admin user exists with Admin role (critical for initial access).
| Performance tests tagging | 9 tests | ~3-4s per run | ✅ Completed | 4. **System-role bootstrap**`Mitglied` system role exists (critical for user registration).
| Critical test query filtering | 1 test | ~8-10s | ✅ Completed |
| Full test suite via promotion | 25 tests | ~77s per run | ✅ Completed | If a new critical bootstrap requirement appears, add a test to the "Critical bootstrap invariants" section in `test/seeds_test.exs`. Removed-test risk is low: a smoke-test failure surfaces broken seeds, and domain tests verify business logic independently of seeds content.
| **Total Saved** | | **~98-107s** | |
--- ---
## Completed Optimizations ## 2. Tagging convention: `:slow`
### 1. Seeds Test Suite Optimization Tests are split into **fast** (standard CI) and **slow** (run via promotion before merge). A test is tagged `@tag :slow` when **all** of:
**Date:** 2026-01-28 - execution time > 1 s, **and**
**Status:** ✅ Completed - low risk — does not catch critical regressions in core business logic, **and**
- it is a UI/display/formatting test, a workflow-detail test, or an edge case with a large dataset (performance tests with 50+ records always qualify).
#### What Changed **Never** tag as `:slow`:
- **Reduced test count:** From 13 tests to 4 tests (69% reduction) - Core CRUD (Member/User create/update/destroy)
- **Reduced seeds executions:** From 8-10 times to 5 times per test run - Basic authentication/authorization
- **Execution time:** From 24-30 seconds to 13-17 seconds - Critical bootstrap (admin user, system roles)
- **Time saved:** ~10-16 seconds per test run (40-50% faster) - Email synchronization
- Representative policy tests (one per permission set + action)
- A test that is merely slow due to inefficient setup or a bug — fix the setup/bug instead
- An integration test — use `@tag :integration` instead
#### Removed Tests (9 tests) Use **`@describetag :slow`** (not `@moduletag`) for describe blocks, so unrelated tests in the same module are not tagged.
Tests were removed because their functionality is covered by domain-specific test suites: One-off isolation fix worth noting as a pattern: a test that loaded *all* members was slow in full runs because of cross-test data accumulation; constraining it with a search query (`/members?query=Alice`) made it both faster and properly isolated. Prefer query filters over loading all records.
1. `"at least one member has no membership fee type assigned"` → Covered by `membership_fees/*_test.exs`
2. `"each membership fee type has at least one member"` → Covered by `membership_fees/*_test.exs`
3. `"members with fee types have cycles with various statuses"` → Covered by `cycle_generator_test.exs`
4. `"creates all 5 authorization roles with correct permission sets"` → Covered by `authorization/*_test.exs`
5. `"all roles have valid permission_set_names"` → Covered by `authorization/permission_sets_test.exs`
6. `"does not change role of users who already have a role"` → Merged into idempotency test
7. `"role creation is idempotent"` (detailed) → Merged into general idempotency test
#### Retained Tests (4 tests)
Critical deployment requirements are still covered:
1. ✅ **Smoke Test:** Seeds run successfully and create basic data
2. ✅ **Idempotency Test:** Seeds can be run multiple times without duplicating data
3. ✅ **Admin Bootstrap:** Admin user exists with Admin role (critical for initial access)
4. ✅ **System Role Bootstrap:** Mitglied system role exists (critical for user registration)
#### Risk Assessment
| Removed Test Category | Alternative Coverage | Risk Level |
|----------------------|---------------------|------------|
| Member/fee type distribution | `membership_fees/*_test.exs` | ⚠️ Low |
| Cycle status variations | `cycle_generator_test.exs` | ⚠️ Low |
| Detailed role configs | `authorization/*_test.exs` | ⚠️ Very Low |
| Permission set validation | `permission_sets_test.exs` | ⚠️ Very Low |
**Overall Risk:** ⚠️ **Low** - All removed tests have equivalent or better coverage in domain-specific test suites.
--- ---
### 2. Full Test Suite via Promotion (`@tag :slow`) ## 3. Execution model
**Date:** 2026-01-28 | Mode | Command | Contents | Time |
**Status:** ✅ Completed |------|---------|----------|------|
| Fast (default) | `just test-fast` / `mix test --exclude slow --exclude ui` | everything except `:slow` and `:ui` | ~6 min |
| Slow only | `just test-slow` / `mix test --only slow` | the ~25 `:slow` tests | ~1.3 min |
| Full | `just test` / `mix test` | all tests | ~7.4 min |
#### What Changed CI: standard pipeline (`check-fast`) runs `mix test --exclude slow --exclude ui`. The full suite (`check-full`) is triggered by promoting a Drone build to `production` and is required before merging to `main` (branch protection).
Tests with **low risk** and **execution time >1 second** are now tagged with `@tag :slow` and excluded from standard test runs. These tests are important but not critical for every commit and are run via promotion before merging to `main`. To find the slowest tests, run `mix test --slowest N` ad hoc. `test/test_helper.exs` also carries a `slowest: 10` option for `ExUnit.start/1`, but it is commented out by default — uncomment it to print the 10 slowest tests at the end of every run.
#### Tagging Criteria
**Tagged as `@tag :slow` when:**
- ✅ Test execution time >1 second
- ✅ Low risk (not critical for catching regressions in core business logic)
- ✅ UI/Display tests (formatting, rendering)
- ✅ Workflow detail tests (not core functionality)
- ✅ Edge cases with large datasets
**NOT tagged when:**
- ❌ Core CRUD operations (Member/User Create/Update/Destroy)
- ❌ Basic Authentication/Authorization
- ❌ Critical Bootstrap (Admin user, system roles)
- ❌ Email Synchronization
- ❌ Representative tests per Permission Set + Action
#### Identified Tests for Full Test Suite (25 tests)
**1. Seeds Tests (2 tests) - 18.1s**
- `"runs successfully and creates basic data"` (9.0s)
- `"is idempotent when run multiple times"` (9.1s)
- **Note:** Critical bootstrap tests remain in fast suite
**2. UserLive.ShowTest (3 tests) - 10.8s**
- `"mounts successfully with valid user ID"` (4.2s)
- `"displays linked member when present"` (2.4s)
- `"redirects to user list when viewing system actor user"` (4.2s)
**3. UserLive.IndexTest (5 tests) - 25.0s**
- `"displays users in a table"` (1.0s)
- `"initially sorts by email ascending"` (2.2s)
- `"can sort email descending by clicking sort button"` (3.4s)
- `"select all automatically checks when all individual users are selected"` (2.0s)
- `"displays linked member name in user list"` (1.9s)
**4. MemberLive.IndexCustomFieldsDisplayTest (3 tests) - 4.9s**
- `"displays custom field with show_in_overview: true"` (1.6s)
- `"formats date custom field values correctly"` (1.5s)
- `"formats email custom field values correctly"` (1.8s)
**5. MemberLive.IndexCustomFieldsEdgeCasesTest (3 tests) - 3.6s**
- `"displays custom field column even when no members have values"` (1.1s)
- `"displays very long custom field values correctly"` (1.4s)
- `"handles multiple custom fields with show_in_overview correctly"` (1.2s)
**6. RoleLive Tests (7 tests) - 7.7s**
- `role_live_test.exs`: `"mounts successfully"` (1.5s), `"deletes non-system role"` (2.1s)
- `role_live/show_test.exs`: 5 tests >1s (mount, display, navigation)
**7. MemberAvailableForLinkingTest (1 test) - 1.5s**
- `"limits results to 10 members even when more exist"` (1.5s)
**8. Performance Tests (1 test) - 3.8s**
- `"boolean filter performance with 150 members"` (3.8s)
**Total:** 25 tests, ~77 seconds saved
#### Execution Commands
**Fast Tests (Default):**
```bash
just test-fast
# or
mix test --exclude slow
```
**Slow Tests Only:**
```bash
just test-slow
# or
mix test --only slow
```
**All Tests:**
```bash
just test
# or
mix test
```
#### CI/CD Integration
- **Standard CI (`check-fast`):** Runs `mix test --exclude slow --exclude ui` for faster feedback loops (~6 minutes)
- **Full Test Suite (`check-full`):** Triggered via promotion before merge, executes `mix test` (all tests, including slow and UI) for comprehensive coverage (~7.4 minutes)
- **Pre-Merge:** Full test suite (`mix test`) runs via promotion before merging to main
- **Manual Execution:** Promote build to `production` in Drone CI to trigger full test suite
#### Risk Assessment
**Risk Level:** ✅ **Very Low**
- All tagged tests have **low risk** - they don't catch critical regressions
- Core functionality remains tested (CRUD, Auth, Bootstrap)
- Standard test runs are faster (~6 minutes vs ~7.4 minutes)
- Full test suite runs via promotion before merge ensures comprehensive coverage
- No functionality is lost, only execution timing changed
**Critical Tests Remain in Fast Suite:**
- Core CRUD operations (Member/User Create/Update/Destroy)
- Basic Authentication/Authorization
- Critical Bootstrap (Admin user, system roles)
- Email Synchronization
- Representative Policy tests (one per Permission Set + Action)
--- ---
### 3. Critical Test Optimization ## 4. Test organization
**Date:** 2026-01-28 Tests mirror the `lib/` structure:
**Status:** ✅ Completed
#### Problem Identified
The test `test respects show_in_overview config` was the slowest test in the suite:
- **Isolated execution:** 4.8 seconds
- **In full test run:** 14.7 seconds
- **Difference:** 9.9 seconds (test isolation issue)
#### Root Cause
The test loaded **all members** from the database, not just the 2 members from the test setup. In full test runs, many members from other tests were present in the database, significantly slowing down the query.
#### Solution Implemented
**Query Filtering:** Added search query parameter to filter to only the expected member.
**Code Change:**
```elixir
# Before:
{:ok, _view, html} = live(conn, "/members")
# After:
{:ok, _view, html} = live(conn, "/members?query=Alice")
```
#### Results
| Execution | Before | After | Improvement |
|-----------|--------|-------|-------------|
| **Isolated** | 4.8s | 1.1s | **-77%** (3.7s saved) |
| **In Module** | 4.2s | 0.4s | **-90%** (3.8s saved) |
| **Expected in Full Run** | 14.7s | ~4-6s | **-65% to -73%** (8-10s saved) |
#### Risk Assessment
**Risk Level:** ✅ **Very Low**
- Test functionality unchanged - only loads expected data
- All assertions still pass
- Test is now faster and more isolated
- No impact on test coverage
---
### 3. Full Test Suite Analysis and Categorization
**Date:** 2026-01-28
**Status:** ✅ Completed
#### Analysis Methodology
A comprehensive analysis was performed to identify tests suitable for the full test suite (via promotion) based on:
- **Execution time:** Tests taking >1 second
- **Risk assessment:** Tests that don't catch critical regressions
- **Test category:** UI/Display, workflow details, edge cases
#### Test Categorization
**🔴 CRITICAL - Must Stay in Fast Suite:**
- Core Business Logic (Member/User CRUD)
- Authentication & Authorization Basics
- Critical Bootstrap (Admin user, system roles)
- Email Synchronization
- Representative Policy Tests (one per Permission Set + Action)
**🟡 LOW RISK - Moved to Full Test Suite (via Promotion):**
- Seeds Tests (non-critical: smoke test, idempotency)
- LiveView Display/Formatting Tests
- UserLive.ShowTest (core functionality covered by Index/Form)
- UserLive.IndexTest UI Features (sorting, checkboxes, navigation)
- RoleLive Tests (role management, not core authorization)
- MemberLive Custom Fields Display Tests
- Edge Cases with Large Datasets
#### Risk Assessment Summary
| Category | Tests | Time Saved | Risk Level | Rationale |
|----------|-------|------------|------------|-----------|
| Seeds (non-critical) | 2 | 18.1s | ⚠️ Low | Critical bootstrap tests remain |
| UserLive.ShowTest | 3 | 10.8s | ⚠️ Low | Core CRUD covered by Index/Form |
| UserLive.IndexTest (UI) | 5 | 25.0s | ⚠️ Low | UI features, not core functionality |
| Custom Fields Display | 6 | 8.5s | ⚠️ Low | Formatting tests, visible in code review |
| RoleLive Tests | 7 | 7.7s | ⚠️ Low | Role management, not authorization |
| Edge Cases | 1 | 1.5s | ⚠️ Low | Edge case, not critical path |
| Performance Tests | 1 | 3.8s | ✅ Very Low | Explicit performance validation |
| **Total** | **25** | **~77s** | **⚠️ Low** | |
**Overall Risk:** ⚠️ **Low** - All moved tests have low risk and don't catch critical regressions. Core functionality remains fully tested.
#### Tests Excluded from Full Test Suite
The following tests were **NOT** moved to full test suite (via promotion) despite being slow:
- **Policy Tests:** Medium risk - kept in fast suite (representative tests remain)
- **UserLive.FormTest:** Medium risk - core CRUD functionality
- **Tests <1s:** Don't meet execution time threshold
- **Critical Bootstrap Tests:** High risk - deployment critical
---
## Current Performance Analysis
### Top 20 Slowest Tests (without `:slow`)
After implementing the full test suite via promotion, the remaining slowest tests are:
| Rank | Test | File | Time | Category |
|------|------|------|------|----------|
| 1 | `test Critical bootstrap invariants Mitglied system role exists` | `seeds_test.exs` | 6.7s | Critical Bootstrap |
| 2 | `test Critical bootstrap invariants Admin user has Admin role` | `seeds_test.exs` | 5.0s | Critical Bootstrap |
| 3 | `test normal_user permission set can read own user record` | `user_policies_test.exs` | 2.6s | Policy Test |
| 4 | `test normal_user permission set can create member` | `member_policies_test.exs` | 2.5s | Policy Test |
| 5-20 | Various Policy and LiveView tests | Multiple files | 1.5-2.4s each | Policy/LiveView |
**Total Top 20:** ~44 seconds (12% of total time without `:slow`)
**Note:** Many previously slow tests (UserLive.IndexTest, UserLive.ShowTest, Display/Formatting tests) are now tagged with `@tag :slow` and excluded from standard runs.
### Performance Hotspots Identified
#### 1. Seeds Tests (~16.2s for 4 tests)
**Status:** ✅ Optimized (reduced from 13 tests)
**Remaining Optimization Potential:** 3-5 seconds
**Opportunities:**
- Settings update could potentially be moved to `setup_all` (if sandbox allows)
- Seeds execution could be further optimized (less data in test mode)
- Idempotency test could be optimized (only 1x seeds instead of 2x)
#### 2. User LiveView Tests (~35.5s for 10 tests)
**Status:** ⏳ Identified for optimization
**Optimization Potential:** 15-20 seconds
**Files:**
- `test/mv_web/user_live/index_test.exs` (3 tests, ~10.2s)
- `test/mv_web/user_live/form_test.exs` (4 tests, ~15.0s)
- `test/mv_web/user_live/show_test.exs` (3 tests, ~10.3s)
**Patterns:**
- Many tests create user/member data
- LiveView mounts are expensive
- Form submissions with validations are slow
**Recommended Actions:**
- Move shared fixtures to `setup_all`
- Reduce test data volume (3-5 users instead of 10+)
- Optimize setup patterns for recurring patterns
#### 3. Policy Tests (~8.7s for 3 tests)
**Status:** ⏳ Identified for optimization
**Optimization Potential:** 5-8 seconds
**Files:**
- `test/mv/membership/member_policies_test.exs` (2 tests, ~6.1s)
- `test/mv/accounts/user_policies_test.exs` (1 test, ~2.6s)
**Pattern:**
- Each test creates new roles/users/members
- Roles are identical across tests
**Recommended Actions:**
- Create roles in `setup_all` (shared across tests)
- Reuse common fixtures
- Maintain test isolation while optimizing setup
---
## Future Optimization Opportunities
### Priority 1: User LiveView Tests Optimization
**Estimated Savings:** 14-22 seconds
**Status:** 📋 Analysis Complete - Ready for Implementation
#### Analysis Summary
Analysis of User LiveView tests identified significant optimization opportunities:
- **Framework functionality over-testing:** ~30 tests test Phoenix/Ash/Gettext core features
- **Redundant test data creation:** Each test creates users/members independently
- **Missing shared fixtures:** No `setup_all` usage for common data
#### Current Performance
**Top 20 Slowest Tests (User LiveView):**
- `index_test.exs`: ~10.2s for 3 tests in Top 20
- `form_test.exs`: ~15.0s for 4 tests in Top 20
- `show_test.exs`: ~10.3s for 3 tests in Top 20
- **Total:** ~35.5 seconds for User LiveView tests
#### Optimization Opportunities
**1. Remove Framework Functionality Tests (~30 tests, 8-12s saved)**
- Remove translation tests (Gettext framework)
- Remove navigation tests (Phoenix LiveView framework)
- Remove validation tests (Ash framework)
- Remove basic HTML rendering tests (consolidate into smoke test)
- Remove password storage tests (AshAuthentication framework)
**2. Implement Shared Fixtures (3-5s saved)**
- Use `setup_all` for common test data in `index_test.exs` and `show_test.exs`
- Share users for sorting/checkbox tests
- Share common users/members across tests
- **Note:** `form_test.exs` uses `async: false`, preventing `setup_all` usage
**3. Consolidate Redundant Tests (~10 tests → 3-4 tests, 2-3s saved)**
- Merge basic display tests into smoke test
- Merge navigation tests into integration test
- Reduce sorting tests to 1 integration test
**4. Optimize Test Data Volume (1-2s saved)**
- Use minimum required data (2 users for sorting, 2 for checkboxes)
- Share data across tests via `setup_all`
#### Tests to Keep (Business Logic)
**Index Tests:**
- `initially sorts by email ascending` - Tests default sort
- `can sort email descending by clicking sort button` - Tests sort functionality
- `select all automatically checks when all individual users are selected` - Business logic
- `does not show system actor user in list` - Business rule
- `displays linked member name in user list` - Business logic
- Edge case tests
**Form Tests:**
- `creates user without password` - Business logic
- `creates user with password when enabled` - Business logic
- `admin sets new password for user` - Business logic
- `selecting member and saving links member to user` - Business logic
- Member linking/unlinking workflow tests
**Show Tests:**
- `displays password authentication status` - Business logic
- `displays linked member when present` - Business logic
- `redirects to user list when viewing system actor user` - Business rule
#### Implementation Plan
**Phase 1: Remove Framework Tests (1-2 hours, ⚠️ Very Low Risk)**
- Remove translation, navigation, validation, and basic HTML rendering tests
- Consolidate remaining display tests into smoke test
**Phase 2: Implement Shared Fixtures (2-3 hours, ⚠️ Low Risk)**
- Add `setup_all` to `index_test.exs` and `show_test.exs`
- Update tests to use shared fixtures
- Verify test isolation maintained
**Phase 3: Consolidate Tests (1-2 hours, ⚠️ Very Low Risk)**
- Merge basic display tests into smoke test
- Merge navigation tests into integration test
- Reduce sorting tests to 1 integration test
**Risk Assessment:** ⚠️ **Low**
- Framework functionality is tested by framework maintainers
- Business logic tests remain intact
- Shared fixtures maintain test isolation
- Consolidation preserves coverage
### Priority 2: Policy Tests Optimization
**Estimated Savings:** 5.5-9 seconds
**Status:** 📋 Analysis Complete - Ready for Decision
#### Analysis Summary
Analysis of policy tests identified significant optimization opportunities:
- **Redundant fixture creation:** Roles and users created repeatedly across tests
- **Framework functionality over-testing:** Many tests verify Ash policy framework behavior
- **Test duplication:** Similar tests across different permission sets
#### Current Performance
**Policy Test Files Performance:**
- `member_policies_test.exs`: 24 tests, ~66s (top 20)
- `user_policies_test.exs`: 30 tests, ~66s (top 20)
- `custom_field_value_policies_test.exs`: 20 tests, ~66s (top 20)
- **Total:** 74 tests, ~152s total
**Top 20 Slowest Policy Tests:** ~66 seconds
#### Framework vs. Business Logic Analysis
**Framework Functionality (Should NOT Test):**
- Policy evaluation (how Ash evaluates policies)
- Permission lookup (how Ash looks up permissions)
- Scope filtering (how Ash applies scope filters)
- Auto-filter behavior (how Ash auto-filters queries)
- Forbidden vs NotFound (how Ash returns errors)
**Business Logic (Should Test):**
- Permission set definitions (what permissions each role has)
- Scope definitions (what scopes each permission set uses)
- Special cases (custom business rules)
- Permission set behavior (how our permission sets differ)
#### Optimization Opportunities
**1. Remove Framework Functionality Tests (~22-34 tests, 3-4s saved)**
- Remove "cannot" tests that verify error types (Forbidden, NotFound)
- Remove tests that verify auto-filter behavior (framework)
- Remove tests that verify permission evaluation (framework)
- **Risk:** ⚠️ Very Low - Framework functionality is tested by Ash maintainers
**2. Consolidate Redundant Tests (~6-8 tests → 2-3 tests, 1-2s saved)**
- Merge similar tests across permission sets
- Create integration tests that cover multiple permission sets
- **Risk:** ⚠️ Low - Same coverage, fewer tests
**3. Share Admin User Across Describe Blocks (1-2s saved)**
- Create admin user once in module-level `setup`
- Reuse admin user in helper functions
- **Note:** `async: false` prevents `setup_all`, but module-level `setup` works
- **Risk:** ⚠️ Low - Admin user is read-only in tests, safe to share
**4. Reduce Test Data Volume (0.5-1s saved)**
- Use minimum required data
- Share fixtures where possible
- **Risk:** ⚠️ Very Low - Still tests same functionality
#### Test Classification Summary
**Tests to Remove (Framework):**
- `member_policies_test.exs`: ~10 tests (cannot create/destroy/update, auto-filter tests)
- `user_policies_test.exs`: ~16 tests (cannot read/update/create/destroy, auto-filter tests)
- `custom_field_value_policies_test.exs`: ~8 tests (similar "cannot" tests)
**Tests to Consolidate (Redundant):**
- `user_policies_test.exs`: 6 tests → 2 tests (can read/update own user record)
**Tests to Keep (Business Logic):**
- All "can" tests that verify permission set behavior
- Special case tests (e.g., "user can always READ linked member")
- AshAuthentication bypass tests (our integration)
#### Implementation Plan
**Phase 1: Remove Framework Tests (1-2 hours, ⚠️ Very Low Risk)**
- Identify all "cannot" tests that verify error types
- Remove tests that verify Ash auto-filter behavior
- Remove tests that verify permission evaluation (framework)
**Phase 2: Consolidate Redundant Tests (1-2 hours, ⚠️ Low Risk)**
- Identify similar tests across permission sets
- Create integration tests that cover multiple permission sets
- Remove redundant individual tests
**Phase 3: Share Admin User (1-2 hours, ⚠️ Low Risk)**
- Add module-level `setup` to create admin user
- Update helper functions to accept admin user parameter
- Update all `setup` blocks to use shared admin user
**Risk Assessment:** ⚠️ **Low**
- Framework functionality is tested by Ash maintainers
- Business logic tests remain intact
- Admin user sharing maintains test isolation (read-only)
- Consolidation preserves coverage
### Priority 3: Seeds Tests Further Optimization
**Estimated Savings:** 3-5 seconds
**Actions:**
1. Investigate if settings update can be moved to `setup_all`
2. Introduce seeds mode for tests (less data in test mode)
3. Optimize idempotency test (only 1x seeds instead of 2x)
**Risk Assessment:** ⚠️ **Low to Medium**
- Sandbox limitations may prevent `setup_all` usage
- Seeds mode would require careful implementation
- Idempotency test optimization needs to maintain test validity
### Priority 4: Additional Test Isolation Improvements
**Estimated Savings:** Variable (depends on specific tests)
**Actions:**
1. Review tests that load all records (similar to the critical test fix)
2. Add query filters where appropriate
3. Ensure proper test isolation in async tests
**Risk Assessment:** ⚠️ **Very Low**
- Similar to the critical test optimization (proven approach)
- Improves test isolation and reliability
---
## Estimated Total Optimization Potential
| Priority | Optimization | Estimated Savings |
|----------|-------------|-------------------|
| 1 | User LiveView Tests | 14-22s |
| 2 | Policy Tests | 5.5-9s |
| 3 | Seeds Tests Further | 3-5s |
| 4 | Additional Isolation | Variable |
| **Total Potential** | | **22.5-36 seconds** |
**Projected Final Time:** From ~368 seconds (fast suite) to **~332-345 seconds** (~5.5-5.8 minutes) with remaining optimizations
**Note:** Detailed analysis documents available:
- User LiveView Tests: See "Priority 1: User LiveView Tests Optimization" section above
- Policy Tests: See "Priority 2: Policy Tests Optimization" section above
---
## Risk Assessment Summary
### Overall Risk Level: ⚠️ **Low**
All optimizations maintain test coverage while improving performance:
| Optimization | Risk Level | Mitigation |
|-------------|------------|------------|
| Seeds tests reduction | ⚠️ Low | Coverage mapped to domain tests |
| Performance tests tagging | ✅ Very Low | Tests still executed, just separately |
| Critical test optimization | ✅ Very Low | Functionality unchanged, better isolation |
| Future optimizations | ⚠️ Low | Careful implementation with verification |
### Monitoring Plan
#### Success Criteria
- ✅ Seeds tests execute in <20 seconds consistently
- ✅ No increase in seeds-related deployment failures
- ✅ No regression in authorization or membership fee bugs
- ✅ Top 20 slowest tests: < 60 seconds (currently ~44s)
- ✅ Total execution time (without `:slow`): < 10 minutes (currently 6.1 min)
- ⏳ Slow tests execution time: < 2 minutes (currently ~1.3 min)
#### What to Watch For
1. **Production Seeds Failures:**
- Monitor deployment logs for seeds errors
- If failures increase, consider restoring detailed tests
2. **Authorization Bugs After Seeds Changes:**
- If role/permission bugs appear after seeds modifications
- May indicate need for more seeds-specific role validation
3. **Test Performance Regression:**
- Monitor test execution times in CI
- Alert if times increase significantly
4. **Developer Feedback:**
- If developers report missing test coverage
- Adjust based on real-world experience
---
## Benchmarking and Analysis
### How to Benchmark Tests
**ExUnit Built-in Benchmarking:**
The test suite is configured to show the slowest tests automatically:
```elixir
# test/test_helper.exs
ExUnit.start(
slowest: 10 # Shows 10 slowest tests at the end of test run
)
```
**Run Benchmark Analysis:**
```bash
# Show slowest tests
mix test --slowest 20
# Exclude slow tests for faster feedback
mix test --exclude slow --slowest 20
# Run only slow tests
mix test --only slow --slowest 10
# Benchmark specific test file
mix test test/mv_web/member_live/index_member_fields_display_test.exs --slowest 5
```
### Benchmarking Best Practices
1. **Run benchmarks regularly** (e.g., monthly) to catch performance regressions
2. **Compare isolated vs. full runs** to identify test isolation issues
3. **Monitor CI execution times** to track trends over time
4. **Document significant changes** in test performance
---
## Test Suite Structure
### Test Execution Modes
**Fast Tests (Default):**
- Excludes slow tests (`@tag :slow`)
- Used for standard development workflow
- Execution time: ~6 minutes
- Command: `mix test --exclude slow` or `just test-fast`
**Slow Tests:**
- Tests tagged with `@tag :slow` or `@describetag :slow` (25 tests)
- Low risk, >1 second execution time
- UI/Display tests, workflow details, edge cases, performance tests
- Execution time: ~1.3 minutes
- Command: `mix test --only slow` or `just test-slow`
- Excluded from standard CI runs
**Full Test Suite (via Promotion):**
- Triggered by promoting a build to `production` in Drone CI
- Runs all tests (`mix test`) for comprehensive coverage
- Execution time: ~7.4 minutes
- Required before merging to `main` (enforced via branch protection)
**All Tests:**
- Includes both fast and slow tests
- Used for comprehensive validation (pre-merge)
- Execution time: ~7.4 minutes
- Command: `mix test` or `just test`
### Test Organization
Tests are organized to mirror the `lib/` directory structure:
``` ```
test/ test/
├── accounts/ # Accounts domain tests ├── accounts/ # Accounts domain
├── membership/ # Membership domain tests ├── membership/ # Membership domain
├── membership_fees/ # Membership fees domain tests ├── membership_fees/ # Membership fees domain
├── mv/ # Core application tests ├── mv/ # Core (accounts, membership, authorization)
│ ├── accounts/ # User-related tests ├── mv_web/ # Web layer (controllers, live, components)
│ ├── membership/ # Member-related tests └── support/ # conn_case.ex, data_case.ex
│ └── authorization/ # Authorization tests
├── mv_web/ # Web layer tests
│ ├── controllers/ # Controller tests
│ ├── live/ # LiveView tests
│ └── components/ # Component tests
└── support/ # Test helpers
├── conn_case.ex # Controller test setup
└── data_case.ex # Database test setup
``` ```
--- ---
## Best Practices for Test Performance
### When Writing New Tests
1. **Use `async: true`** when possible (for parallel execution)
2. **Filter queries** to only load necessary data
3. **Share fixtures** in `setup_all` when appropriate
4. **Tag performance tests** with `@tag :slow` if they use large datasets
5. **Keep test data minimal** - only create what's needed for the test
### When Optimizing Existing Tests
1. **Measure first** - Use `mix test --slowest` to identify bottlenecks
2. **Compare isolated vs. full runs** - Identify test isolation issues
3. **Optimize setup** - Move shared data to `setup_all` where possible
4. **Filter queries** - Only load data needed for the test
5. **Verify coverage** - Ensure optimizations don't reduce test coverage
### Test Tagging Guidelines
#### Tag as `@tag :slow` when:
1. **Performance Tests:**
- Explicitly testing performance characteristics
- Using large datasets (50+ records)
- Testing scalability or query optimization
- Validating N+1 query prevention
2. **Low-Risk Tests (>1s):**
- UI/Display/Formatting tests (not critical for every commit)
- Workflow detail tests (not core functionality)
- Edge cases with large datasets
- Show page tests (core functionality covered by Index/Form tests)
- Non-critical seeds tests (smoke tests, idempotency)
#### Do NOT tag as `@tag :slow` when:
- ❌ Test is slow due to inefficient setup (fix the setup instead)
- ❌ Test is slow due to bugs (fix the bug instead)
- ❌ Core CRUD operations (Member/User Create/Update/Destroy)
- ❌ Basic Authentication/Authorization
- ❌ Critical Bootstrap (Admin user, system roles)
- ❌ Email Synchronization
- ❌ Representative Policy tests (one per Permission Set + Action)
- ❌ It's an integration test (use `@tag :integration` instead)
---
## Changelog
### 2026-01-28: Initial Optimization Phase
**Completed:**
- ✅ Reduced seeds tests from 13 to 4 tests
- ✅ Tagged 9 performance tests with `@tag :slow`
- ✅ Optimized critical test with query filtering
- ✅ Created slow test suite infrastructure
- ✅ Updated CI/CD to exclude slow tests from standard runs
- ✅ Added promotion-based full test suite pipeline (`check-full`)
**Time Saved:** ~21-30 seconds per test run
### 2026-01-28: Full Test Suite via Promotion Implementation
**Completed:**
- ✅ Analyzed all tests for full test suite candidates
- ✅ Identified 36 tests with low risk and >1s execution time
- ✅ Tagged 25 tests with `@tag :slow` for full test suite (via promotion)
- ✅ Categorized tests by risk level and execution time
- ✅ Documented tagging criteria and guidelines
**Tests Tagged:**
- 2 Seeds tests (non-critical) - 18.1s
- 3 UserLive.ShowTest tests - 10.8s
- 5 UserLive.IndexTest tests - 25.0s
- 3 MemberLive.IndexCustomFieldsDisplayTest tests - 4.9s
- 3 MemberLive.IndexCustomFieldsEdgeCasesTest tests - 3.6s
- 7 RoleLive tests - 7.7s
- 1 MemberAvailableForLinkingTest - 1.5s
- 1 Performance test (already tagged) - 3.8s
**Time Saved:** ~77 seconds per test run
**Total Optimization Impact:**
- **Before:** ~445 seconds (7.4 minutes)
- **After (fast suite):** ~368 seconds (6.1 minutes)
- **Time saved:** ~77 seconds (17% reduction)
**Next Steps:**
- ⏳ Monitor full test suite execution via promotion in CI
- ⏳ Optimize remaining slow tests (Policy tests, etc.)
- ⏳ Further optimize Seeds tests (Priority 3)
---
## References ## References
- **Testing Standards:** `CODE_GUIDELINES.md` - Section 4 (Testing Standards) - Testing Standards: `CODE_GUIDELINES.md` §4
- **CI/CD Configuration:** `.drone.yml` - CI/CD: `.drone.jsonnet`
- **Test Helper:** `test/test_helper.exs` - Test helper: `test/test_helper.exs`
- **Justfile Commands:** `Justfile` (test-fast, test-slow, test-all) - Just commands: `Justfile` (`test-fast`, `test-slow`, `test`)
---
## Questions & Answers
**Q: What if seeds create wrong data and break the system?**
A: The smoke test will fail if seeds raise errors. Domain tests ensure business logic is correct regardless of seeds content.
**Q: What if we add a new critical bootstrap requirement?**
A: Add a new test to the "Critical bootstrap invariants" section in `test/seeds_test.exs`.
**Q: How do we know the removed tests aren't needed?**
A: Monitor for 2-3 months. If no seeds-related bugs appear that would have been caught by removed tests, they were redundant.
**Q: Should we restore the tests for important releases?**
A: Consider running the full test suite (including slow tests) before major releases. Daily development uses the optimized suite.
**Q: How do I add a new performance test?**
A: Tag it with `@tag :slow` for individual tests or `@describetag :slow` for describe blocks. Use `@describetag` instead of `@moduletag` to avoid tagging unrelated tests. Include measurable performance assertions (query counts, timing with tolerance, etc.). See "Performance Test Guidelines" section above.
**Q: Can I run slow tests locally?**
A: Yes, use `just test-slow` or `mix test --only slow`. They're excluded from standard runs for faster feedback.
**Q: What is the "full test suite"?**
A: The full test suite runs **all tests** (`mix test`), including slow and UI tests. Tests tagged with `@tag :slow` or `@describetag :slow` are excluded from standard CI runs (`check-fast`) for faster feedback, but are included when promoting a build to `production` (`check-full`) before merging to `main`.
**Q: Which tests should I tag as `:slow`?**
A: Tag tests with `@tag :slow` if they: (1) take >1 second, (2) have low risk (not critical for catching regressions), and (3) test UI/Display/Formatting or workflow details. See "Test Tagging Guidelines" section for details.
**Q: What if a slow test fails in the full test suite?**
A: If a test in the full test suite fails, investigate the failure. If it indicates a critical regression, consider moving it back to the fast suite. If it's a flaky test, fix the test itself. The merge will be blocked until all tests pass.