docs(project): condense progress log, roadmap and test/infra docs
This commit is contained in:
parent
3797bc8fae
commit
6fddb5285b
5 changed files with 164 additions and 3277 deletions
File diff suppressed because it is too large
Load diff
|
|
@ -6,16 +6,11 @@
|
|||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Phase 1: Feature Area Breakdown](#phase-1-feature-area-breakdown)
|
||||
2. [Phase 2: API Endpoint Definition](#phase-2-api-endpoint-definition)
|
||||
3. [Phase 3: Implementation Task Creation](#phase-3-implementation-task-creation)
|
||||
4. [Phase 4: Task Organization and Prioritization](#phase-4-task-organization-and-prioritization)
|
||||
This is the living per-area roadmap: shipped state (coarse — see `development-progress-log.md` for detail), open issues, and the missing-features backlog. For the actual, current endpoints see `lib/mv_web/router.ex` and `docs/page-permission-route-coverage.md`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Feature Area Breakdown
|
||||
## Feature Area Breakdown
|
||||
|
||||
### Feature Areas
|
||||
|
||||
|
|
@ -49,10 +44,10 @@
|
|||
- ✅ **Page-level authorization** - LiveView page access control
|
||||
- ✅ **System role protection** - Critical roles cannot be deleted
|
||||
|
||||
**Planned: OIDC-only mode (TDD, tests first):**
|
||||
- Admin Settings: When OIDC-only is enabled, disable "Allow direct registration" toggle and show hint (tests in `GlobalSettingsLiveTest`).
|
||||
- Backend: Reject password sign-in and `register_with_password` when OIDC-only (tests in `AuthControllerTest`, `Accounts`).
|
||||
- GET `/sign-in` redirect to OIDC when OIDC-only and OIDC configured (tests in `AuthControllerTest`). Implementation to follow after tests.
|
||||
**Implemented: OIDC-only mode:**
|
||||
- ✅ Admin Settings: when OIDC-only is enabled, the "Allow direct registration" toggle is disabled with a hint.
|
||||
- ✅ Backend rejects password sign-in and `register_with_password` when OIDC-only is active.
|
||||
- ✅ GET `/sign-in` redirects to OIDC when OIDC-only and OIDC are configured (`MvWeb.Plugs.OidcOnlySignInRedirect`). The `oidc_only` setting and ENV are read via `Mv.Config.oidc_only?/0`.
|
||||
|
||||
**Missing Features:**
|
||||
- ❌ Password reset flow
|
||||
|
|
@ -183,6 +178,11 @@
|
|||
- ✅ Navbar with profile button
|
||||
- ✅ Member list as landing page
|
||||
- ✅ Breadcrumbs (basic)
|
||||
- ✅ **Flash: auto-dismiss and consistency** (Design Guidelines §9)
|
||||
- Auto-dismiss implemented via the `FlashAutoDismiss` JS hook (`assets/js/app.js`) driven by the `data-auto-clear-ms` and `data-clear-flash-key` attributes on the flash component (`MvWeb.CoreComponents.flash/1`); the per-flash delay is set through the component's `auto_clear_ms` attribute, and the dismiss button is kept for accessibility.
|
||||
- On timeout the hook pushes LiveView's built-in `lv:clear-flash` event (no custom `handle_event`) and hides the element.
|
||||
- All flashes (including “Email copied”) use the same variants (info, success, warning, error); no special tone. See `DESIGN_GUIDELINES.md` §9.
|
||||
- ❌ Per-kind default durations (info/success 4–6s, warning 6–8s, error 8–12s) are not built in — the delay is a single explicit `auto_clear_ms` value per flash, not a kind-based default.
|
||||
|
||||
**Open Issues:**
|
||||
- [#188](https://git.local-it.org/local-it/mitgliederverwaltung/issues/188) - Check if searching just on typing is accessible (S, Low priority)
|
||||
|
|
@ -196,11 +196,6 @@
|
|||
- ❌ Mobile navigation
|
||||
- ❌ Context-sensitive help
|
||||
- ❌ Onboarding tooltips
|
||||
- ❌ **Flash: Auto-dismiss and consistency** (Design Guidelines §9)
|
||||
- Auto-dismiss: info/success 4–6s, warning 6–8s, error 8–12s; dismiss button kept for accessibility.
|
||||
- Implement via JS hook (e.g. `FlashAutoDismiss`) + `data-dismiss-ms` (or `data-kind`) on flash component; on timeout push `lv:clear-flash` and hide element.
|
||||
- LiveView: add shared `handle_event("lv:clear-flash", %{"key" => key}, socket)` (e.g. in `MvWeb` live_view quote) calling `clear_flash(socket, key)`.
|
||||
- All flashes (including “Email copied”) use the same variants (info, success, warning, error); no special tone. See `DESIGN_GUIDELINES.md` §9.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -308,7 +303,12 @@
|
|||
#### 10. **Reporting & Analytics** 📊
|
||||
|
||||
**Current State:**
|
||||
- ✅ **Statistics page (MVP)** – `/statistics` with active/inactive member counts, joins/exits by year, cycle totals, open amount (2026-02-10)
|
||||
- ✅ **Statistics page (MVP)** – `/statistics` with active/inactive member counts, joins/exits by year, cycle totals, open amount (2026-02-10). Backed by `Mv.Statistics` (read-only Ash reads on `Member` + `MembershipFeeCycle`, no new resources); displayed in `MvWeb.StatisticsLive`. Permission: read_only, normal_user, admin (own_data denied).
|
||||
|
||||
**MVP design decisions:**
|
||||
- Charts are HTML/CSS + SVG only — no Contex, no Chart.js (deliberate).
|
||||
- Open amount = total unpaid only; no overdue vs. not-yet-due split in the MVP.
|
||||
- Out of scope (deferred follow-ups): export (CSV/PDF), caching, month/quarter filters, "members per fee type" / "members per group" stats, overdue split, new tables/resources.
|
||||
|
||||
**Missing Features:**
|
||||
- ❌ Extended member statistics dashboard
|
||||
|
|
@ -487,367 +487,13 @@
|
|||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: API Endpoint Definition
|
||||
|
||||
### Endpoint Types
|
||||
|
||||
Since this is a **Phoenix LiveView** application with **Ash Framework**, we have three types of endpoints:
|
||||
|
||||
1. **LiveView Endpoints** - Mount points and event handlers
|
||||
2. **HTTP Controller Endpoints** - Traditional REST-style endpoints
|
||||
3. **Ash Resource Actions** - Backend data layer API
|
||||
|
||||
### Authentication Requirements Legend
|
||||
|
||||
- 🔓 **Public** - No authentication required
|
||||
- 🔐 **Authenticated** - Requires valid user session
|
||||
- 👤 **User Role** - Requires specific user role
|
||||
- 🛡️ **Admin Only** - Requires admin privileges
|
||||
|
||||
---
|
||||
|
||||
### 1. Authentication & Authorization Endpoints
|
||||
|
||||
#### HTTP Controller Endpoints
|
||||
|
||||
| Method | Route | Purpose | Auth | Request | Response |
|
||||
|--------|-------|---------|------|---------|----------|
|
||||
| `GET` | `/auth/user/password/sign_in` | Show password login form | 🔓 | - | HTML form |
|
||||
| `POST` | `/auth/user/password/sign_in` | Submit password login | 🔓 | `{email, password}` | Redirect + session cookie |
|
||||
| `GET` | `/auth/user/oidc` | Initiate OIDC flow | 🔓 | - | Redirect to Rauthy |
|
||||
| `GET` | `/auth/user/oidc/callback` | Handle OIDC callback | 🔓 | `{code, state}` | Redirect + session cookie |
|
||||
| `POST` | `/auth/user/sign_out` | Sign out user | 🔐 | - | Redirect to login |
|
||||
| `GET` | `/auth/link-oidc-account` | OIDC account linking (password verification) | 🔓 | - | LiveView form | ✅ Implemented |
|
||||
| `GET` | `/auth/user/password/reset` | Show password reset form | 🔓 | - | HTML form |
|
||||
| `POST` | `/auth/user/password/reset` | Request password reset | 🔓 | `{email}` | Success message + email sent |
|
||||
| `GET` | `/auth/user/password/reset/:token` | Show reset password form | 🔓 | - | HTML form |
|
||||
| `POST` | `/auth/user/password/reset/:token` | Submit new password | 🔓 | `{password, password_confirmation}` | Redirect to login |
|
||||
|
||||
#### Ash Resource Actions
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `User` | `:sign_in_with_password` | Password authentication | 🔓 | `{email, password}` | `{:ok, user}` or `{:error, reason}` |
|
||||
| `User` | `:sign_in_with_oidc` | OIDC authentication | 🔓 | `{oidc_id, email, user_info}` | `{:ok, user}` or `{:error, reason}` |
|
||||
| `User` | `:register_with_password` | Create user with password | 🔓 | `{email, password}` | `{:ok, user}` |
|
||||
| `User` | `:register_with_oidc` | Create user via OIDC | 🔓 | `{oidc_id, email}` | `{:ok, user}` |
|
||||
| `User` | `:request_password_reset` | Generate reset token | 🔓 | `{email}` | `{:ok, token}` |
|
||||
| `User` | `:reset_password` | Reset password with token | 🔓 | `{token, password}` | `{:ok, user}` |
|
||||
| `Token` | `:revoke` | Revoke authentication token | 🔐 | `{jti}` | `{:ok, token}` |
|
||||
|
||||
#### **NEW: Role & Permission Actions** (Issue #191, #190, #151)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `Role` | `:create` | Create new role | 🛡️ | `{name, description, permissions}` | `{:ok, role}` |
|
||||
| `Role` | `:list` | List all roles | 🔐 | - | `[%Role{}]` |
|
||||
| `Role` | `:update` | Update role | 🛡️ | `{id, name, permissions}` | `{:ok, role}` |
|
||||
| `Role` | `:delete` | Delete role | 🛡️ | `{id}` | `{:ok, role}` |
|
||||
| `User` | `:assign_role` | Assign role to user | 🛡️ | `{user_id, role_id}` | `{:ok, user}` |
|
||||
| `User` | `:remove_role` | Remove role from user | 🛡️ | `{user_id, role_id}` | `{:ok, user}` |
|
||||
| `Permission` | `:list` | List all permissions | 🔐 | - | `[%Permission{}]` |
|
||||
| `Permission` | `:check` | Check user permission | 🔐 | `{user_id, resource, action}` | `{:ok, boolean}` |
|
||||
|
||||
---
|
||||
|
||||
### 2. Member Management Endpoints
|
||||
|
||||
#### LiveView Endpoints
|
||||
|
||||
| Mount | Purpose | Auth | Query Params | Events |
|
||||
|-------|---------|------|--------------|--------|
|
||||
| `/members` | Member list with search/sort | 🔐 | `?search=&sort_by=&sort_dir=` | `search`, `sort`, `delete`, `select` |
|
||||
| `/members/new` | Create new member form | 🔐 | - | `save`, `cancel`, `add_custom_field_value` |
|
||||
| `/members/:id` | Member detail view | 🔐 | - | `edit`, `delete`, `link_user` |
|
||||
| `/members/:id/edit` | Edit member form | 🔐 | - | `save`, `cancel`, `add_custom_field_value`, `remove_custom_field_value` |
|
||||
|
||||
#### LiveView Event Handlers
|
||||
|
||||
| Event | Purpose | Params | Response |
|
||||
|-------|---------|--------|----------|
|
||||
| `search` | Trigger search | `%{"search" => query}` | Update member list |
|
||||
| `sort` | Sort member list | `%{"field" => field}` | Update sorted list |
|
||||
| `delete` | Delete member | `%{"id" => id}` | Redirect to list |
|
||||
| `save` | Create/update member | `%{"member" => attrs}` | Redirect or show errors |
|
||||
| `link_user` | Link user to member | `%{"user_id" => id}` | Update member view |
|
||||
| `unlink_user` | Unlink user from member | - | Update member view |
|
||||
| `add_custom_field_value` | Add custom field value | `%{"custom_field_id" => id, "value" => val}` | Update form |
|
||||
| `remove_custom_field_value` | Remove custom field value | `%{"custom_field_value_id" => id}` | Update form |
|
||||
|
||||
#### Ash Resource Actions
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `Member` | `:create_member` | Create member | 🔐 | `{first_name, last_name, email, ...}` | `{:ok, member}` |
|
||||
| `Member` | `:read` | List/search members | 🔐 | `{search, sort_by, limit, offset}` | `[%Member{}]` |
|
||||
| `Member` | `:update_member` | Update member | 🔐 | `{id, attrs}` | `{:ok, member}` |
|
||||
| `Member` | `:destroy` | Delete member | 🔐 | `{id}` | `{:ok, member}` |
|
||||
| `Member` | `:search_fulltext` | Full-text search | 🔐 | `{query}` | `[%Member{}]` |
|
||||
| `Member` | `:link_to_user` | Link member to user | 🔐 | `{member_id, user_id}` | `{:ok, member}` |
|
||||
| `Member` | `:unlink_from_user` | Unlink from user | 🔐 | `{member_id}` | `{:ok, member}` |
|
||||
|
||||
#### **NEW: Enhanced Search & Filter Actions** (Issue #162, #154, #165)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `Member` | `:fuzzy_search` | Fuzzy text search | 🔐 | `{query, threshold}` | `[%Member{}]` |
|
||||
| `Member` | `:advanced_search` | Multi-criteria search | 🔐 | `{filters: [{field, op, value}]}` | `[%Member{}]` |
|
||||
| `Member` | `:paginate` | Paginated member list | 🔐 | `{page, per_page, filters}` | `{members, total, page_info}` |
|
||||
| `Member` | `:sort_by_custom_field` | Sort by custom field | 🔐 | `{custom_field_id, direction}` | `[%Member{}]` |
|
||||
| `Member` | `:bulk_delete` | Delete multiple members | 🛡️ | `{ids: [id1, id2, ...]}` | `{:ok, count}` |
|
||||
| `Member` | `:bulk_update` | Update multiple members | 🛡️ | `{ids, attrs}` | `{:ok, count}` |
|
||||
| `Member` | `:export` | Export to CSV/Excel | 🔐 | `{format, filters}` | File download |
|
||||
| `Member` | `:import` | Import from CSV | 🛡️ | `{file, mapping}` | `{:ok, imported_count, errors}` |
|
||||
|
||||
---
|
||||
|
||||
### 3. Custom Fields (CustomFieldValue System) Endpoints
|
||||
|
||||
#### LiveView Endpoints (✅ Implemented)
|
||||
|
||||
| Mount | Purpose | Auth | Events | Status |
|
||||
|-------|---------|------|--------|--------|
|
||||
| `/settings` | Global settings (includes custom fields management) | 🔐 | `save`, `validate` | ✅ Implemented |
|
||||
| `/custom_field_values` | List all custom field values | 🔐 | `new`, `edit`, `delete` | ✅ Implemented |
|
||||
| `/custom_field_values/new` | Create custom field value | 🔐 | `save`, `cancel` | ✅ Implemented |
|
||||
| `/custom_field_values/:id` | Custom field value detail | 🔐 | `edit` | ✅ Implemented |
|
||||
| `/custom_field_values/:id/edit` | Edit custom field value | 🔐 | `save`, `cancel` | ✅ Implemented |
|
||||
| `/custom_field_values/:id/show/edit` | Edit from show page | 🔐 | `save`, `cancel` | ✅ Implemented |
|
||||
|
||||
**Note:** Custom fields (definitions) are managed via LiveComponent in `/settings` page, not as separate routes.
|
||||
|
||||
#### Ash Resource Actions
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `CustomField` | `:create` | Create custom field | 🛡️ | `{name, value_type, description, ...}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:read` | List custom fields | 🔐 | - | `[%CustomField{}]` |
|
||||
| `CustomField` | `:update` | Update custom field | 🛡️ | `{id, attrs}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:destroy` | Delete custom field | 🛡️ | `{id}` | `{:ok, custom_field}` |
|
||||
| `CustomFieldValue` | `:create` | Add custom field value to member | 🔐 | `{member_id, custom_field_id, value}` | `{:ok, custom_field_value}` |
|
||||
| `CustomFieldValue` | `:update` | Update custom field value | 🔐 | `{id, value}` | `{:ok, custom_field_value}` |
|
||||
| `CustomFieldValue` | `:destroy` | Remove custom field value | 🔐 | `{id}` | `{:ok, custom_field_value}` |
|
||||
|
||||
#### **NEW: Enhanced Custom Fields** (Issue #194, #157, #161, #153)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `CustomField` | `:set_default_visibility` | Show/hide by default | 🛡️ | `{id, visible}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:set_required` | Mark as required | 🛡️ | `{id, required}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:add_validation` | Add validation rule | 🛡️ | `{id, rule_type, params}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:create_group` | Create field group | 🛡️ | `{name, custom_field_ids}` | `{:ok, group}` |
|
||||
| `CustomFieldValue` | `:validate_value` | Validate custom field value | 🔐 | `{custom_field_id, value}` | `{:ok, valid}` or `{:error, reason}` |
|
||||
|
||||
---
|
||||
|
||||
### 4. User Management Endpoints
|
||||
|
||||
#### LiveView Endpoints
|
||||
|
||||
| Mount | Purpose | Auth | Events |
|
||||
|-------|---------|------|--------|
|
||||
| `/users` | User list | 🛡️ | `new`, `edit`, `delete`, `assign_role` |
|
||||
| `/users/new` | Create user form | 🛡️ | `save`, `cancel` |
|
||||
| `/users/:id` | User detail view | 🔐 | `edit`, `delete`, `change_password` |
|
||||
| `/users/:id/edit` | Edit user form | 🔐 | `save`, `cancel`, `link_member` |
|
||||
| `/profile` | Current user profile | 🔐 | `edit`, `change_password` |
|
||||
|
||||
#### Ash Resource Actions
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `User` | `:create_user` | Create user (admin) | 🛡️ | `{email, member_id?}` | `{:ok, user}` |
|
||||
| `User` | `:read` | List users | 🛡️ | - | `[%User{}]` |
|
||||
| `User` | `:update_user` | Update user | 🔐 | `{id, email, member_id?}` | `{:ok, user}` |
|
||||
| `User` | `:destroy` | Delete user | 🛡️ | `{id}` | `{:ok, user}` |
|
||||
| `User` | `:admin_set_password` | Set password (admin) | 🛡️ | `{id, password}` | `{:ok, user}` |
|
||||
| `User` | `:change_password` | Change own password | 🔐 | `{current_password, new_password}` | `{:ok, user}` |
|
||||
|
||||
#### **NEW: Combined User/Member Management** (Issue #169, #168)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `User` | `:create_with_member` | Create user + member together | 🛡️ | `{user: {...}, member: {...}}` | `{:ok, %{user, member}}` |
|
||||
| `User` | `:invite_user` | Send invitation email | 🛡️ | `{email, role_id, member_id?}` | `{:ok, invitation}` |
|
||||
| `User` | `:accept_invitation` | Accept invitation | 🔓 | `{token, password}` | `{:ok, user}` |
|
||||
|
||||
---
|
||||
|
||||
### 5. Navigation & UX Endpoints
|
||||
|
||||
#### LiveView Endpoints
|
||||
|
||||
| Mount | Purpose | Auth | Events |
|
||||
|-------|---------|------|--------|
|
||||
| `/` | Dashboard/Home | 🔐 | - |
|
||||
| `/dashboard` | Dashboard view | 🔐 | Contextual based on role |
|
||||
|
||||
#### HTTP Controller Endpoints
|
||||
|
||||
| Method | Route | Purpose | Auth | Request | Response |
|
||||
|--------|-------|---------|------|---------|----------|
|
||||
| `GET` | `/health` | Health check | 🔓 | - | `{"status": "ok"}` |
|
||||
| `GET` | `/` | Root redirect | - | - | Redirect to dashboard or login |
|
||||
|
||||
---
|
||||
|
||||
### 6. Internationalization Endpoints
|
||||
|
||||
#### HTTP Controller Endpoints (✅ Implemented)
|
||||
|
||||
| Method | Route | Purpose | Auth | Request | Response | Status |
|
||||
|--------|-------|---------|------|---------|----------|--------|
|
||||
| `POST` | `/set_locale` | Set user locale | 🔐 | `{locale: "de"}` | Redirect with cookie | ✅ Implemented |
|
||||
| `GET` | `/locales` | List available locales | 🔓 | - | `["de", "en"]` | ❌ Not implemented |
|
||||
|
||||
**Note:** Locale is set via `/set_locale` POST endpoint and persisted in session/cookie. Supported locales: `de` (default), `en`.
|
||||
|
||||
---
|
||||
|
||||
### 7. Payment & Fees Management Endpoints
|
||||
|
||||
#### LiveView Endpoints (✅ Implemented)
|
||||
|
||||
| Mount | Purpose | Auth | Events | Status |
|
||||
|-------|---------|------|--------|--------|
|
||||
| `/membership_fee_types` | Membership fee type list | 🔐 | `new`, `edit`, `delete` | ✅ Implemented |
|
||||
| `/membership_fee_types/new` | Create membership fee type | 🔐 | `save`, `cancel` | ✅ Implemented |
|
||||
| `/membership_fee_types/:id/edit` | Edit membership fee type | 🔐 | `save`, `cancel` | ✅ Implemented |
|
||||
| `/membership_fee_settings` | Global membership fee settings | 🔐 | `save` | ✅ Implemented |
|
||||
| `/contributions/member/:id` | Member contribution periods (mock-up) | 🔐 | - | ⚠️ Mock-up only |
|
||||
| `/contribution_types` | Contribution types (mock-up) | 🔐 | - | ⚠️ Mock-up only |
|
||||
|
||||
#### Ash Resource Actions (✅ Partially Implemented)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output | Status |
|
||||
|----------|--------|---------|------|-------|--------|--------|
|
||||
| `MembershipFeeType` | `:create` | Create fee type | 🔐 | `{name, amount, interval, ...}` | `{:ok, fee_type}` | ✅ Implemented |
|
||||
| `MembershipFeeType` | `:read` | List fee types | 🔐 | - | `[%MembershipFeeType{}]` | ✅ Implemented |
|
||||
| `MembershipFeeType` | `:update` | Update fee type (name, amount, description) | 🔐 | `{id, attrs}` | `{:ok, fee_type}` | ✅ Implemented |
|
||||
| `MembershipFeeType` | `:destroy` | Delete fee type (if no cycles) | 🔐 | `{id}` | `{:ok, fee_type}` | ✅ Implemented |
|
||||
| `MembershipFeeCycle` | `:read` | List cycles for member | 🔐 | `{member_id}` | `[%MembershipFeeCycle{}]` | ✅ Implemented |
|
||||
| `MembershipFeeCycle` | `:update` | Update cycle status | 🔐 | `{id, status}` | `{:ok, cycle}` | ✅ Implemented |
|
||||
| `Payment` | `:create` | Record payment | 🔐 | `{member_id, fee_id, amount, date}` | `{:ok, payment}` | ❌ Not implemented |
|
||||
| `Payment` | `:list_by_member` | Member payment history | 🔐 | `{member_id}` | `[%Payment{}]` | ❌ Not implemented |
|
||||
| `Payment` | `:mark_paid` | Mark as paid | 🔐 | `{id}` | `{:ok, payment}` | ❌ Not implemented |
|
||||
| `Invoice` | `:generate` | Generate invoice | 🔐 | `{member_id, fee_id, period}` | `{:ok, invoice}` | ❌ Not implemented |
|
||||
| `Invoice` | `:send` | Send invoice via email | 🔐 | `{id}` | `{:ok, sent}` | ❌ Not implemented |
|
||||
| `Payment` | `:import_vereinfacht` | Import from vereinfacht.digital | 🛡️ | `{transactions}` | `{:ok, count}` | ❌ Not implemented |
|
||||
|
||||
---
|
||||
|
||||
### 8. Admin Panel & Configuration Endpoints
|
||||
|
||||
#### LiveView Endpoints (✅ Partially Implemented)
|
||||
|
||||
| Mount | Purpose | Auth | Events | Status |
|
||||
|-------|---------|------|--------|--------|
|
||||
| `/settings` | Global settings (club name, member fields, custom fields) | 🔐 | `save`, `validate` | ✅ Implemented |
|
||||
| `/admin/roles` | Role management | 🛡️ | `new`, `edit`, `delete` | ✅ Implemented |
|
||||
| `/admin/roles/new` | Create role | 🛡️ | `save`, `cancel` | ✅ Implemented |
|
||||
| `/admin/roles/:id` | Role detail view | 🛡️ | `edit` | ✅ Implemented |
|
||||
| `/admin/roles/:id/edit` | Edit role | 🛡️ | `save`, `cancel` | ✅ Implemented |
|
||||
| `/admin` | Admin dashboard | 🛡️ | - | ❌ Not implemented |
|
||||
| `/admin/organization` | Organization profile | 🛡️ | `save` | ❌ Not implemented |
|
||||
| `/admin/email-templates` | Email template editor | 🛡️ | `create`, `edit`, `preview` | ❌ Not implemented |
|
||||
| `/admin/audit-log` | System audit log | 🛡️ | `filter`, `export` | ❌ Not implemented |
|
||||
|
||||
#### Ash Resource Actions (✅ Partially Implemented)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output | Status |
|
||||
|----------|--------|---------|------|-------|--------|--------|
|
||||
| `Setting` | `:read` | Get settings (singleton) | 🔐 | - | `{:ok, settings}` | ✅ Implemented |
|
||||
| `Setting` | `:update` | Update settings | 🔐 | `{club_name, member_field_visibility, ...}` | `{:ok, settings}` | ✅ Implemented |
|
||||
| `Setting` | `:update_member_field_visibility` | Update field visibility | 🔐 | `{member_field_visibility}` | `{:ok, settings}` | ✅ Implemented |
|
||||
| `Setting` | `:update_single_member_field_visibility` | Atomic field visibility update | 🔐 | `{field, show_in_overview}` | `{:ok, settings}` | ✅ Implemented |
|
||||
| `Setting` | `:update_membership_fee_settings` | Update fee settings | 🔐 | `{include_joining_cycle, default_membership_fee_type_id}` | `{:ok, settings}` | ✅ Implemented |
|
||||
| `Role` | `:read` | List roles | 🛡️ | - | `[%Role{}]` | ✅ Implemented |
|
||||
| `Role` | `:create` | Create role | 🛡️ | `{name, permission_set_name, ...}` | `{:ok, role}` | ✅ Implemented |
|
||||
| `Role` | `:update` | Update role | 🛡️ | `{id, attrs}` | `{:ok, role}` | ✅ Implemented |
|
||||
| `Role` | `:destroy` | Delete role (if not system role) | 🛡️ | `{id}` | `{:ok, role}` | ✅ Implemented |
|
||||
| `Organization` | `:read` | Get organization info | 🔐 | - | `%Organization{}` | ❌ Not implemented |
|
||||
| `Organization` | `:update` | Update organization | 🛡️ | `{name, logo, ...}` | `{:ok, org}` | ❌ Not implemented |
|
||||
| `AuditLog` | `:list` | List audit entries | 🛡️ | `{filters, pagination}` | `[%AuditLog{}]` | ❌ Not implemented |
|
||||
|
||||
---
|
||||
|
||||
### 9. Communication & Notifications Endpoints
|
||||
|
||||
#### LiveView Endpoints (NEW)
|
||||
|
||||
| Mount | Purpose | Auth | Events |
|
||||
|-------|---------|------|--------|
|
||||
| `/communications` | Communication history | 🔐 | `new`, `view` |
|
||||
| `/communications/new` | Create email broadcast | 🔐 | `select_recipients`, `preview`, `send` |
|
||||
| `/notifications` | User notifications | 🔐 | `mark_read`, `mark_all_read` |
|
||||
|
||||
#### Ash Resource Actions (NEW)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `EmailBroadcast` | `:create` | Create broadcast | 🔐 | `{subject, body, recipient_filter}` | `{:ok, broadcast}` |
|
||||
| `EmailBroadcast` | `:send` | Send broadcast | 🔐 | `{id}` | `{:ok, sent_count}` |
|
||||
| `EmailTemplate` | `:create` | Create template | 🛡️ | `{name, subject, body}` | `{:ok, template}` |
|
||||
| `EmailTemplate` | `:render` | Render template | 🔐 | `{id, variables}` | `rendered_html` |
|
||||
| `Notification` | `:create` | Create notification | System | `{user_id, type, message}` | `{:ok, notification}` |
|
||||
| `Notification` | `:list_for_user` | Get user notifications | 🔐 | `{user_id}` | `[%Notification{}]` |
|
||||
| `Notification` | `:mark_read` | Mark as read | 🔐 | `{id}` | `{:ok, notification}` |
|
||||
|
||||
---
|
||||
|
||||
### 10. Reporting & Analytics Endpoints
|
||||
|
||||
#### LiveView Endpoints (NEW)
|
||||
|
||||
| Mount | Purpose | Auth | Events |
|
||||
|-------|---------|------|--------|
|
||||
| `/reports` | Reports dashboard | 🔐 | `generate`, `schedule` |
|
||||
| `/reports/members` | Member statistics | 🔐 | `filter`, `export` |
|
||||
| `/reports/payments` | Payment reports | 🔐 | `filter`, `export` |
|
||||
| `/reports/custom` | Custom report builder | 🛡️ | `build`, `save`, `run` |
|
||||
|
||||
#### Ash Resource Actions (NEW)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `Report` | `:generate_member_stats` | Member statistics | 🔐 | `{date_range, filters}` | Statistics object |
|
||||
| `Report` | `:generate_payment_stats` | Payment statistics | 🔐 | `{date_range}` | Statistics object |
|
||||
| `Report` | `:export_to_csv` | Export report to CSV | 🔐 | `{report_type, filters}` | CSV file |
|
||||
| `Report` | `:export_to_pdf` | Export report to PDF | 🔐 | `{report_type, filters}` | PDF file |
|
||||
| `Report` | `:schedule` | Schedule recurring report | 🛡️ | `{report_type, frequency, recipients}` | `{:ok, schedule}` |
|
||||
|
||||
---
|
||||
|
||||
### 11. Data Import/Export Endpoints
|
||||
|
||||
#### LiveView Endpoints (NEW)
|
||||
|
||||
| Mount | Purpose | Auth | Events |
|
||||
|-------|---------|------|--------|
|
||||
| `/import` | Data import wizard | 🛡️ | `upload`, `map_fields`, `preview`, `import` |
|
||||
| `/export` | Data export tool | 🔐 | `select_data`, `configure`, `export` |
|
||||
|
||||
#### Ash Resource Actions (NEW)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `Member` | `:import_csv` | Import members from CSV | 🛡️ | `{file, field_mapping}` | `{:ok, imported, errors}` |
|
||||
| `Member` | `:validate_import` | Validate import data | 🛡️ | `{file, field_mapping}` | `{:ok, validation_results}` |
|
||||
| `Member` | `:export_csv` | Export members to CSV | 🔐 | `{filters}` | CSV file |
|
||||
| `Member` | `:export_excel` | Export members to Excel | 🔐 | `{filters}` | Excel file |
|
||||
| `Database` | `:export_backup` | Full database backup | 🛡️ | - | Backup file |
|
||||
| `Database` | `:import_backup` | Restore from backup | 🛡️ | `{file}` | `{:ok, restored}` |
|
||||
|
||||
---
|
||||
## Endpoints
|
||||
|
||||
For the real, current routes and their authorization, see `lib/mv_web/router.ex` and `docs/page-permission-route-coverage.md` (the per-permission-set route matrix). The Ash resource actions are defined on each resource module under `lib/`. An earlier speculative API catalog for not-yet-existing resources (Payment, Invoice, Report, Notification, AuditLog, Organization) was removed — those are tracked above as missing features per area, not as endpoint specs.
|
||||
|
||||
---
|
||||
|
||||
**References:**
|
||||
- Open Issues: https://git.local-it.org/local-it/mitgliederverwaltung/issues
|
||||
- Project Board: Sprint 8 (23.10 - 13.11)
|
||||
- Architecture: See [`CODE_GUIDELINES.md`](../CODE_GUIDELINES.md)
|
||||
- Database Schema: See [`database-schema-readme.md`](database-schema-readme.md)
|
||||
|
||||
|
|
|
|||
|
|
@ -125,25 +125,7 @@ Verify mode is set in `tls_options` for port 587 (STARTTLS). For port 465 (impli
|
|||
|
||||
---
|
||||
|
||||
## 12. Summary Checklist
|
||||
|
||||
- [x] ENV: `SMTP_HOST`, `SMTP_PORT`, `SMTP_USERNAME`, `SMTP_PASSWORD`, `SMTP_PASSWORD_FILE`, `SMTP_SSL`.
|
||||
- [x] ENV: `MAIL_FROM_NAME`, `MAIL_FROM_EMAIL` for sender identity.
|
||||
- [x] Settings: attributes and UI for host, port, username, password, TLS/SSL, from-name, from-email.
|
||||
- [x] Password from file: `SMTP_PASSWORD_FILE` supported in `runtime.exs`.
|
||||
- [x] Mailer: Swoosh SMTP adapter configured from merged ENV + Settings when SMTP is configured.
|
||||
- [x] Per-request SMTP config via `Mv.Mailer.smtp_config/0` for Settings-only scenarios.
|
||||
- [x] TLS certificate validation relaxed for OTP 27 (tls_options for 587; sockopts with verify only for 465).
|
||||
- [x] Prod warning: clear message in Settings when SMTP is not configured.
|
||||
- [x] Test email: form with recipient field, translatable content, classified success/error messages.
|
||||
- [x] Join confirmation email: uses `Mailer.smtp_config/0` (same as test mail); on failure returns `{:error, :email_delivery_failed}`, error shown in JoinLive, logged for admin.
|
||||
- [x] AshAuthentication senders: graceful error handling (no crash on delivery failure).
|
||||
- [x] Gettext for all new UI strings, translated to German.
|
||||
- [x] Docs and code guidelines updated.
|
||||
|
||||
---
|
||||
|
||||
## 13. Follow-up / Future Work
|
||||
## 12. Follow-up / Future Work
|
||||
|
||||
- **SMTP password at-rest encryption:** The `smtp_password` attribute is currently stored in plaintext in the `settings` table. It is excluded from default reads (same pattern as `oidc_client_secret`); both are read only via explicit select when needed. For production systems at-rest encryption (e.g. with [Cloak](https://hexdocs.pm/cloak)) should be considered and tracked as a follow-up issue.
|
||||
- **Error classification:** SMTP error categorization currently uses substring matching on server messages (e.g. "535", "authentication"). A more robust approach would be to pattern-match on `gen_smtp` error tuples first where possible, and fall back to string analysis only when needed. Server wording varies; consider extending patterns as new providers are used.
|
||||
|
|
|
|||
|
|
@ -1,163 +0,0 @@
|
|||
# Statistics Page – Implementation Plan
|
||||
|
||||
**Project:** Mila – Membership Management System
|
||||
**Feature:** Statistics page at `/statistics`
|
||||
**Scope:** MVP only (no export, no optional extensions)
|
||||
**Last updated:** 2026-02-10
|
||||
|
||||
---
|
||||
|
||||
## Decisions (from clarification)
|
||||
|
||||
| Topic | Decision |
|
||||
|-------|----------|
|
||||
| Route | `/statistics` |
|
||||
| Navigation | Top-level menu (next to Members, Fee Types) |
|
||||
| Permission | read_only, normal_user, admin (same as member list) |
|
||||
| Charts | HTML/CSS and SVG only (no Contex, no Chart.js) |
|
||||
| MVP scope | Minimal: active/inactive, joins/exits per year, contribution sums per year, open amount |
|
||||
| Open amount | Total unpaid only (no overdue vs. not-yet-due split in MVP) |
|
||||
|
||||
Excluded from this plan: Export (CSV/PDF), caching, month/quarter filters, “members per fee type”, “members per group”, and overdue split.
|
||||
|
||||
---
|
||||
|
||||
## 1. Statistics module (`Mv.Statistics`)
|
||||
|
||||
**Goal:** Central module for all statistics; LiveView only calls this API. Uses Ash reads with actor so policies apply.
|
||||
|
||||
**Location:** `lib/mv/statistics.ex` (new).
|
||||
|
||||
**Functions to implement:**
|
||||
|
||||
| Function | Purpose | Data source |
|
||||
|----------|---------|-------------|
|
||||
| `active_member_count(opts)` | Count members with `exit_date == nil` | `Member` read with filter |
|
||||
| `inactive_member_count(opts)` | Count members with `exit_date != nil` | `Member` read with filter |
|
||||
| `joins_by_year(year, opts)` | Count members with `join_date` in given year | `Member` read, filter by year, count |
|
||||
| `exits_by_year(year, opts)` | Count members with `exit_date` in given year | `Member` read, filter by year, count |
|
||||
| `cycle_totals_by_year(year, opts)` | For cycles with `cycle_start` in year: total sum, and sums/counts by status (paid, unpaid, suspended) | `MembershipFeeCycle` read (filter by year via `cycle_start`), aggregate sum(amount) and count per status in Elixir or via Ash aggregates |
|
||||
| `open_amount_total(opts)` | Sum of `amount` for all cycles with `status == :unpaid` | `MembershipFeeCycle` read with filter `status == :unpaid`, sum(amount) |
|
||||
|
||||
All functions accept `opts` (keyword list) and pass `actor: opts[:actor]` (and `domain:` where needed) to Ash calls. No new resources; only read actions on existing `Member` and `MembershipFeeCycle`.
|
||||
|
||||
**Implementation notes:**
|
||||
|
||||
- Use `Ash.Query.filter(Member, expr(...))` for date filters; for “year”, filter `join_date >= first_day_of_year` and `join_date <= last_day_of_year` (same for `exit_date` and for `MembershipFeeCycle.cycle_start`).
|
||||
- For `cycle_totals_by_year`: either multiple Ash reads (one per status) with sum aggregate, or one read of cycles in that year and `Enum.group_by(..., :status)` then sum amounts in Elixir.
|
||||
- Use `Mv.MembershipFees.CalendarCycles` only if needed for interval (e.g. cycle_end); for “cycle in year” the `cycle_start` year is enough.
|
||||
|
||||
**Tests:** Unit tests in `test/mv/statistics_test.exs` for each function (with fixtures: members with join_date/exit_date, cycles with cycle_start/amount/status). Use `Mv.Helpers.SystemActor.get_system_actor()` in tests for Ash read authorization where appropriate.
|
||||
|
||||
---
|
||||
|
||||
## 2. Route and authorization
|
||||
|
||||
**Router** ([lib/mv_web/router.ex](lib/mv_web/router.ex)):
|
||||
|
||||
- In the same `ash_authentication_live_session` block where `/members` and `/membership_fee_types` live, add:
|
||||
- `live "/statistics", StatisticsLive, :index`
|
||||
|
||||
**PagePaths** ([lib/mv_web/page_paths.ex](lib/mv_web/page_paths.ex)):
|
||||
|
||||
- Add module attribute `@statistics "/statistics"`.
|
||||
- Add `def statistics, do: @statistics`.
|
||||
- No change to `@admin_page_paths` (statistics is top-level).
|
||||
|
||||
**Page permission** (route matrix is driven by [lib/mv/authorization/permission_sets.ex](lib/mv/authorization/permission_sets.ex)):
|
||||
|
||||
- Add `"/statistics"` to the `pages` list of **read_only** (e.g. after `"/groups/:slug"`) and to the `pages` list of **normal_user** (e.g. after groups entries). **admin** already has `"*"` so no change.
|
||||
- **own_data** must not list `/statistics` (so they cannot access it).
|
||||
- Update [docs/page-permission-route-coverage.md](docs/page-permission-route-coverage.md): add row for `| /statistics | ✗ | ✓ | ✓ | ✓ |`.
|
||||
- Add test in `test/mv_web/plugs/check_page_permission_test.exs`: read_only and normal_user and admin can access `/statistics`; own_data cannot.
|
||||
|
||||
---
|
||||
|
||||
## 3. Sidebar
|
||||
|
||||
**File:** [lib/mv_web/components/layouts/sidebar.ex](lib/mv_web/components/layouts/sidebar.ex).
|
||||
|
||||
- In `sidebar_menu`, after the “Fee Types” menu item and before the “Administration” block, add a conditional menu item for Statistics:
|
||||
- `can_access_page?(@current_user, PagePaths.statistics())` → show link.
|
||||
- `href={~p"/statistics"}`, `icon="hero-chart-bar"` (or similar), `label={gettext("Statistics")}`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Statistics LiveView
|
||||
|
||||
**Module:** `MvWeb.StatisticsLive`
|
||||
**File:** `lib/mv_web/live/statistics_live.ex`
|
||||
**Mount:** `:index` only.
|
||||
|
||||
**Behaviour:**
|
||||
|
||||
- `on_mount`: use `MvWeb.LiveUserAuth, :live_user_required` and ensure role/permission check (same as other protected LiveViews). In `mount` or `handle_params`, set default selected year to current year (e.g. `Date.utc_today().year`).
|
||||
- **Assigns:** `:year` (integer), `:active_count`, `:inactive_count`, `:joins_this_year`, `:exits_this_year`, `:cycle_totals` (map with keys e.g. `:total`, `:paid`, `:unpaid`, `:suspended` for the selected year), `:open_amount_total`, and any extra needed for the bar data (e.g. list of `%{year: y, joins: j, exits: e}` for a small range of years if you show a minimal bar chart).
|
||||
- **Year filter:** A single select or dropdown for year (e.g. from “first year with data” to current year). On change, send event (e.g. `"set_year"`) with `%{"year" => year}`; in `handle_event` update `assigns.year` and reload data by calling `Mv.Statistics` again and re-assigning.
|
||||
|
||||
**Data loading:**
|
||||
|
||||
- In `mount` and whenever year changes, call `Mv.Statistics` with `actor: current_actor(socket)` (and optionally `year: @year` where needed). Assign results to socket. Handle errors (e.g. redirect or flash) if a call fails.
|
||||
|
||||
**Layout (sections):**
|
||||
|
||||
1. **Page title:** e.g. “Statistics” (gettext).
|
||||
2. **Year filter:** One control to select year; applies to “joins/exits” and “contribution sums” for that year.
|
||||
3. **Cards (top row):**
|
||||
- Active members (count)
|
||||
- Inactive members (count)
|
||||
- Joins in selected year
|
||||
- Exits in selected year
|
||||
- Open amount total (sum of all unpaid cycles; format with `MvWeb.Helpers.MembershipFeeHelpers.format_currency/1`)
|
||||
- Optionally: “Paid this year” (from `cycle_totals_by_year` for selected year)
|
||||
4. **Contributions for selected year:** One section showing for the chosen year: total (Soll), paid, unpaid, suspended (sums and optionally counts). Use simple table or key-value list; no chart required for MVP.
|
||||
5. **Joins / Exits by year (simple bar chart):** Data: e.g. last 5 or 10 years. For each year, show joins and exits as horizontal bars (HTML/CSS: e.g. `div` with `width: #{percent}%`). Pure HTML/SVG; no external chart library. Use Tailwind/DaisyUI for layout and cards.
|
||||
|
||||
**Accessibility:** Semantic HTML; headings (e.g. `h2`) for each section; ensure year filter has a label; format numbers in a screen-reader-friendly way (e.g. no purely visual abbreviations without aria-label).
|
||||
|
||||
**i18n:** All user-visible strings via gettext (e.g. “Statistics”, “Active members”, “Inactive members”, “Joins (year)”, “Exits (year)”, “Open amount”, “Contributions for year”, “Total”, “Paid”, “Unpaid”, “Suspended”). Add keys to `priv/gettext` as needed.
|
||||
|
||||
---
|
||||
|
||||
## 5. Implementation order (tasks)
|
||||
|
||||
Execute in this order so that each step is testable:
|
||||
|
||||
1. **Statistics module**
|
||||
- Add `lib/mv/statistics.ex` with the six functions above and `@moduledoc`.
|
||||
- Add `test/mv/statistics_test.exs` with tests for each function (use fixtures for members and cycles; pass actor in opts).
|
||||
- Run tests and fix until green.
|
||||
|
||||
2. **Route and permission**
|
||||
- Add `live "/statistics", StatisticsLive, :index` in router.
|
||||
- Add `statistics/0` and `@statistics` in PagePaths.
|
||||
- Add `/statistics` to page permission logic so read_only, normal_user, admin are allowed and own_data is denied.
|
||||
- Update `docs/page-permission-route-coverage.md` and add/update plug tests for `/statistics`.
|
||||
|
||||
3. **Sidebar**
|
||||
- Add Statistics link in sidebar (top-level) with `can_access_page?` and `PagePaths.statistics()`.
|
||||
|
||||
4. **StatisticsLive**
|
||||
- Create `lib/mv_web/live/statistics_live.ex` with mount, assigns, year param, and data loading from `Mv.Statistics`.
|
||||
- Implement UI: title, year filter, cards, contribution section, simple joins/exits bar (HTML).
|
||||
- Add gettext keys and use them in the template.
|
||||
- Optionally add a simple LiveView test (e.g. authenticated user sees statistics page and key labels).
|
||||
|
||||
5. **CI and docs**
|
||||
- Run `just ci-dev` (or project equivalent); fix formatting, Credo, and tests.
|
||||
- In [docs/feature-roadmap.md](docs/feature-roadmap.md), update “Reporting & Analytics” to reflect that a basic statistics page is implemented (MVP).
|
||||
- In [CODE_GUIDELINES.md](CODE_GUIDELINES.md), add a short note under a suitable section (e.g. “Reporting” or “LiveView”) that statistics are provided by `Mv.Statistics` and displayed in `StatisticsLive`, if desired.
|
||||
|
||||
---
|
||||
|
||||
## 6. Out of scope (not in this plan)
|
||||
|
||||
- Export (CSV/PDF).
|
||||
- Caching (ETS/GenServer/HTTP).
|
||||
- Month or quarter filters.
|
||||
- “Members per fee type” or “members per group” statistics.
|
||||
- Overdue vs. not-yet-due split for open amount.
|
||||
- Contex or Chart.js.
|
||||
- New database tables or Ash resources.
|
||||
|
||||
These can be added later as separate tasks or follow-up plans.
|
||||
|
|
@ -1,877 +1,98 @@
|
|||
# Test Performance Optimization
|
||||
|
||||
**Last Updated:** 2026-01-28
|
||||
**Status:** ✅ Active optimization program
|
||||
**Status:** Implemented
|
||||
|
||||
This document records the test-suite performance work and — most importantly — the conventions that govern how tests are tagged and run. The seeds-test rationale in §1 is the canonical reference linked from `test/seeds_test.exs`.
|
||||
|
||||
Baseline result: the standard (fast) suite runs ~368 s (~6.1 min) vs. ~445 s before; the full suite (all tests) ~7.4 min. Slow-tagged tests (~25, >1 s each, ~77 s total) are excluded from standard runs and executed via promotion before merge.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
## 1. Seeds Test Suite — coverage mapping
|
||||
|
||||
This document provides a comprehensive overview of test performance optimizations, risk assessments, and future opportunities. The test suite execution time has been reduced through systematic analysis and targeted optimizations.
|
||||
The seeds tests were reduced from 13 to 4. The 9 removed tests were dropped because their assertions are already covered by domain-specific test suites — this mapping is the justification and must be preserved:
|
||||
|
||||
### Current Performance Metrics
|
||||
| Removed seeds test | Covered by |
|
||||
|--------------------|-----------|
|
||||
| `"at least one member has no membership fee type assigned"` | `membership_fees/*_test.exs` |
|
||||
| `"each membership fee type has at least one member"` | `membership_fees/*_test.exs` |
|
||||
| `"members with fee types have cycles with various statuses"` | `cycle_generator_test.exs` |
|
||||
| `"creates all 5 authorization roles with correct permission sets"` | `authorization/*_test.exs` |
|
||||
| `"all roles have valid permission_set_names"` | `authorization/permission_sets_test.exs` |
|
||||
| `"does not change role of users who already have a role"` | merged into general idempotency test |
|
||||
| `"role creation is idempotent"` (detailed) | merged into general idempotency test |
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| **Total Execution Time** (without `:slow` tests) | ~368 seconds (~6.1 minutes) |
|
||||
| **Total Tests** | 1,336 tests (+ 25 doctests) |
|
||||
| **Async Execution** | 163.5 seconds |
|
||||
| **Sync Execution** | 281.5 seconds |
|
||||
| **Slow Tests Excluded** | 25 tests (tagged with `@tag :slow`) |
|
||||
| **Top 50 Slowest Tests** | 121.9 seconds (27.4% of total time) |
|
||||
### 4 retained critical-bootstrap tests (and why)
|
||||
|
||||
### Optimization Impact Summary
|
||||
These guard deployment-critical invariants that nothing else covers and must stay in the **fast** suite:
|
||||
|
||||
| Optimization | Tests Affected | Time Saved | Status |
|
||||
|--------------|----------------|------------|--------|
|
||||
| Seeds tests reduction | 13 → 4 tests | ~10-16s | ✅ Completed |
|
||||
| Performance tests tagging | 9 tests | ~3-4s per run | ✅ Completed |
|
||||
| Critical test query filtering | 1 test | ~8-10s | ✅ Completed |
|
||||
| Full test suite via promotion | 25 tests | ~77s per run | ✅ Completed |
|
||||
| **Total Saved** | | **~98-107s** | |
|
||||
1. **Smoke test** — seeds run successfully and create basic data.
|
||||
2. **Idempotency** — seeds can be re-run without duplicating data.
|
||||
3. **Admin bootstrap** — admin user exists with Admin role (critical for initial access).
|
||||
4. **System-role bootstrap** — `Mitglied` system role exists (critical for user registration).
|
||||
|
||||
If a new critical bootstrap requirement appears, add a test to the "Critical bootstrap invariants" section in `test/seeds_test.exs`. Removed-test risk is low: a smoke-test failure surfaces broken seeds, and domain tests verify business logic independently of seeds content.
|
||||
|
||||
---
|
||||
|
||||
## Completed Optimizations
|
||||
## 2. Tagging convention: `:slow`
|
||||
|
||||
### 1. Seeds Test Suite Optimization
|
||||
Tests are split into **fast** (standard CI) and **slow** (run via promotion before merge). A test is tagged `@tag :slow` when **all** of:
|
||||
|
||||
**Date:** 2026-01-28
|
||||
**Status:** ✅ Completed
|
||||
- execution time > 1 s, **and**
|
||||
- low risk — does not catch critical regressions in core business logic, **and**
|
||||
- it is a UI/display/formatting test, a workflow-detail test, or an edge case with a large dataset (performance tests with 50+ records always qualify).
|
||||
|
||||
#### What Changed
|
||||
**Never** tag as `:slow`:
|
||||
|
||||
- **Reduced test count:** From 13 tests to 4 tests (69% reduction)
|
||||
- **Reduced seeds executions:** From 8-10 times to 5 times per test run
|
||||
- **Execution time:** From 24-30 seconds to 13-17 seconds
|
||||
- **Time saved:** ~10-16 seconds per test run (40-50% faster)
|
||||
- Core CRUD (Member/User create/update/destroy)
|
||||
- Basic authentication/authorization
|
||||
- Critical bootstrap (admin user, system roles)
|
||||
- Email synchronization
|
||||
- Representative policy tests (one per permission set + action)
|
||||
- A test that is merely slow due to inefficient setup or a bug — fix the setup/bug instead
|
||||
- An integration test — use `@tag :integration` instead
|
||||
|
||||
#### Removed Tests (9 tests)
|
||||
Use **`@describetag :slow`** (not `@moduletag`) for describe blocks, so unrelated tests in the same module are not tagged.
|
||||
|
||||
Tests were removed because their functionality is covered by domain-specific test suites:
|
||||
|
||||
1. `"at least one member has no membership fee type assigned"` → Covered by `membership_fees/*_test.exs`
|
||||
2. `"each membership fee type has at least one member"` → Covered by `membership_fees/*_test.exs`
|
||||
3. `"members with fee types have cycles with various statuses"` → Covered by `cycle_generator_test.exs`
|
||||
4. `"creates all 5 authorization roles with correct permission sets"` → Covered by `authorization/*_test.exs`
|
||||
5. `"all roles have valid permission_set_names"` → Covered by `authorization/permission_sets_test.exs`
|
||||
6. `"does not change role of users who already have a role"` → Merged into idempotency test
|
||||
7. `"role creation is idempotent"` (detailed) → Merged into general idempotency test
|
||||
|
||||
#### Retained Tests (4 tests)
|
||||
|
||||
Critical deployment requirements are still covered:
|
||||
|
||||
1. ✅ **Smoke Test:** Seeds run successfully and create basic data
|
||||
2. ✅ **Idempotency Test:** Seeds can be run multiple times without duplicating data
|
||||
3. ✅ **Admin Bootstrap:** Admin user exists with Admin role (critical for initial access)
|
||||
4. ✅ **System Role Bootstrap:** Mitglied system role exists (critical for user registration)
|
||||
|
||||
#### Risk Assessment
|
||||
|
||||
| Removed Test Category | Alternative Coverage | Risk Level |
|
||||
|----------------------|---------------------|------------|
|
||||
| Member/fee type distribution | `membership_fees/*_test.exs` | ⚠️ Low |
|
||||
| Cycle status variations | `cycle_generator_test.exs` | ⚠️ Low |
|
||||
| Detailed role configs | `authorization/*_test.exs` | ⚠️ Very Low |
|
||||
| Permission set validation | `permission_sets_test.exs` | ⚠️ Very Low |
|
||||
|
||||
**Overall Risk:** ⚠️ **Low** - All removed tests have equivalent or better coverage in domain-specific test suites.
|
||||
One-off isolation fix worth noting as a pattern: a test that loaded *all* members was slow in full runs because of cross-test data accumulation; constraining it with a search query (`/members?query=Alice`) made it both faster and properly isolated. Prefer query filters over loading all records.
|
||||
|
||||
---
|
||||
|
||||
### 2. Full Test Suite via Promotion (`@tag :slow`)
|
||||
## 3. Execution model
|
||||
|
||||
**Date:** 2026-01-28
|
||||
**Status:** ✅ Completed
|
||||
| Mode | Command | Contents | Time |
|
||||
|------|---------|----------|------|
|
||||
| Fast (default) | `just test-fast` / `mix test --exclude slow --exclude ui` | everything except `:slow` and `:ui` | ~6 min |
|
||||
| Slow only | `just test-slow` / `mix test --only slow` | the ~25 `:slow` tests | ~1.3 min |
|
||||
| Full | `just test` / `mix test` | all tests | ~7.4 min |
|
||||
|
||||
#### What Changed
|
||||
CI: standard pipeline (`check-fast`) runs `mix test --exclude slow --exclude ui`. The full suite (`check-full`) is triggered by promoting a Drone build to `production` and is required before merging to `main` (branch protection).
|
||||
|
||||
Tests with **low risk** and **execution time >1 second** are now tagged with `@tag :slow` and excluded from standard test runs. These tests are important but not critical for every commit and are run via promotion before merging to `main`.
|
||||
|
||||
#### Tagging Criteria
|
||||
|
||||
**Tagged as `@tag :slow` when:**
|
||||
- ✅ Test execution time >1 second
|
||||
- ✅ Low risk (not critical for catching regressions in core business logic)
|
||||
- ✅ UI/Display tests (formatting, rendering)
|
||||
- ✅ Workflow detail tests (not core functionality)
|
||||
- ✅ Edge cases with large datasets
|
||||
|
||||
**NOT tagged when:**
|
||||
- ❌ Core CRUD operations (Member/User Create/Update/Destroy)
|
||||
- ❌ Basic Authentication/Authorization
|
||||
- ❌ Critical Bootstrap (Admin user, system roles)
|
||||
- ❌ Email Synchronization
|
||||
- ❌ Representative tests per Permission Set + Action
|
||||
|
||||
#### Identified Tests for Full Test Suite (25 tests)
|
||||
|
||||
**1. Seeds Tests (2 tests) - 18.1s**
|
||||
- `"runs successfully and creates basic data"` (9.0s)
|
||||
- `"is idempotent when run multiple times"` (9.1s)
|
||||
- **Note:** Critical bootstrap tests remain in fast suite
|
||||
|
||||
**2. UserLive.ShowTest (3 tests) - 10.8s**
|
||||
- `"mounts successfully with valid user ID"` (4.2s)
|
||||
- `"displays linked member when present"` (2.4s)
|
||||
- `"redirects to user list when viewing system actor user"` (4.2s)
|
||||
|
||||
**3. UserLive.IndexTest (5 tests) - 25.0s**
|
||||
- `"displays users in a table"` (1.0s)
|
||||
- `"initially sorts by email ascending"` (2.2s)
|
||||
- `"can sort email descending by clicking sort button"` (3.4s)
|
||||
- `"select all automatically checks when all individual users are selected"` (2.0s)
|
||||
- `"displays linked member name in user list"` (1.9s)
|
||||
|
||||
**4. MemberLive.IndexCustomFieldsDisplayTest (3 tests) - 4.9s**
|
||||
- `"displays custom field with show_in_overview: true"` (1.6s)
|
||||
- `"formats date custom field values correctly"` (1.5s)
|
||||
- `"formats email custom field values correctly"` (1.8s)
|
||||
|
||||
**5. MemberLive.IndexCustomFieldsEdgeCasesTest (3 tests) - 3.6s**
|
||||
- `"displays custom field column even when no members have values"` (1.1s)
|
||||
- `"displays very long custom field values correctly"` (1.4s)
|
||||
- `"handles multiple custom fields with show_in_overview correctly"` (1.2s)
|
||||
|
||||
**6. RoleLive Tests (7 tests) - 7.7s**
|
||||
- `role_live_test.exs`: `"mounts successfully"` (1.5s), `"deletes non-system role"` (2.1s)
|
||||
- `role_live/show_test.exs`: 5 tests >1s (mount, display, navigation)
|
||||
|
||||
**7. MemberAvailableForLinkingTest (1 test) - 1.5s**
|
||||
- `"limits results to 10 members even when more exist"` (1.5s)
|
||||
|
||||
**8. Performance Tests (1 test) - 3.8s**
|
||||
- `"boolean filter performance with 150 members"` (3.8s)
|
||||
|
||||
**Total:** 25 tests, ~77 seconds saved
|
||||
|
||||
#### Execution Commands
|
||||
|
||||
**Fast Tests (Default):**
|
||||
```bash
|
||||
just test-fast
|
||||
# or
|
||||
mix test --exclude slow
|
||||
```
|
||||
|
||||
**Slow Tests Only:**
|
||||
```bash
|
||||
just test-slow
|
||||
# or
|
||||
mix test --only slow
|
||||
```
|
||||
|
||||
**All Tests:**
|
||||
```bash
|
||||
just test
|
||||
# or
|
||||
mix test
|
||||
```
|
||||
|
||||
#### CI/CD Integration
|
||||
|
||||
- **Standard CI (`check-fast`):** Runs `mix test --exclude slow --exclude ui` for faster feedback loops (~6 minutes)
|
||||
- **Full Test Suite (`check-full`):** Triggered via promotion before merge, executes `mix test` (all tests, including slow and UI) for comprehensive coverage (~7.4 minutes)
|
||||
- **Pre-Merge:** Full test suite (`mix test`) runs via promotion before merging to main
|
||||
- **Manual Execution:** Promote build to `production` in Drone CI to trigger full test suite
|
||||
|
||||
#### Risk Assessment
|
||||
|
||||
**Risk Level:** ✅ **Very Low**
|
||||
|
||||
- All tagged tests have **low risk** - they don't catch critical regressions
|
||||
- Core functionality remains tested (CRUD, Auth, Bootstrap)
|
||||
- Standard test runs are faster (~6 minutes vs ~7.4 minutes)
|
||||
- Full test suite runs via promotion before merge ensures comprehensive coverage
|
||||
- No functionality is lost, only execution timing changed
|
||||
|
||||
**Critical Tests Remain in Fast Suite:**
|
||||
- Core CRUD operations (Member/User Create/Update/Destroy)
|
||||
- Basic Authentication/Authorization
|
||||
- Critical Bootstrap (Admin user, system roles)
|
||||
- Email Synchronization
|
||||
- Representative Policy tests (one per Permission Set + Action)
|
||||
To find the slowest tests, run `mix test --slowest N` ad hoc. `test/test_helper.exs` also carries a `slowest: 10` option for `ExUnit.start/1`, but it is commented out by default — uncomment it to print the 10 slowest tests at the end of every run.
|
||||
|
||||
---
|
||||
|
||||
### 3. Critical Test Optimization
|
||||
## 4. Test organization
|
||||
|
||||
**Date:** 2026-01-28
|
||||
**Status:** ✅ Completed
|
||||
|
||||
#### Problem Identified
|
||||
|
||||
The test `test respects show_in_overview config` was the slowest test in the suite:
|
||||
- **Isolated execution:** 4.8 seconds
|
||||
- **In full test run:** 14.7 seconds
|
||||
- **Difference:** 9.9 seconds (test isolation issue)
|
||||
|
||||
#### Root Cause
|
||||
|
||||
The test loaded **all members** from the database, not just the 2 members from the test setup. In full test runs, many members from other tests were present in the database, significantly slowing down the query.
|
||||
|
||||
#### Solution Implemented
|
||||
|
||||
**Query Filtering:** Added search query parameter to filter to only the expected member.
|
||||
|
||||
**Code Change:**
|
||||
```elixir
|
||||
# Before:
|
||||
{:ok, _view, html} = live(conn, "/members")
|
||||
|
||||
# After:
|
||||
{:ok, _view, html} = live(conn, "/members?query=Alice")
|
||||
```
|
||||
|
||||
#### Results
|
||||
|
||||
| Execution | Before | After | Improvement |
|
||||
|-----------|--------|-------|-------------|
|
||||
| **Isolated** | 4.8s | 1.1s | **-77%** (3.7s saved) |
|
||||
| **In Module** | 4.2s | 0.4s | **-90%** (3.8s saved) |
|
||||
| **Expected in Full Run** | 14.7s | ~4-6s | **-65% to -73%** (8-10s saved) |
|
||||
|
||||
#### Risk Assessment
|
||||
|
||||
**Risk Level:** ✅ **Very Low**
|
||||
|
||||
- Test functionality unchanged - only loads expected data
|
||||
- All assertions still pass
|
||||
- Test is now faster and more isolated
|
||||
- No impact on test coverage
|
||||
|
||||
---
|
||||
|
||||
### 3. Full Test Suite Analysis and Categorization
|
||||
|
||||
**Date:** 2026-01-28
|
||||
**Status:** ✅ Completed
|
||||
|
||||
#### Analysis Methodology
|
||||
|
||||
A comprehensive analysis was performed to identify tests suitable for the full test suite (via promotion) based on:
|
||||
- **Execution time:** Tests taking >1 second
|
||||
- **Risk assessment:** Tests that don't catch critical regressions
|
||||
- **Test category:** UI/Display, workflow details, edge cases
|
||||
|
||||
#### Test Categorization
|
||||
|
||||
**🔴 CRITICAL - Must Stay in Fast Suite:**
|
||||
- Core Business Logic (Member/User CRUD)
|
||||
- Authentication & Authorization Basics
|
||||
- Critical Bootstrap (Admin user, system roles)
|
||||
- Email Synchronization
|
||||
- Representative Policy Tests (one per Permission Set + Action)
|
||||
|
||||
**🟡 LOW RISK - Moved to Full Test Suite (via Promotion):**
|
||||
- Seeds Tests (non-critical: smoke test, idempotency)
|
||||
- LiveView Display/Formatting Tests
|
||||
- UserLive.ShowTest (core functionality covered by Index/Form)
|
||||
- UserLive.IndexTest UI Features (sorting, checkboxes, navigation)
|
||||
- RoleLive Tests (role management, not core authorization)
|
||||
- MemberLive Custom Fields Display Tests
|
||||
- Edge Cases with Large Datasets
|
||||
|
||||
#### Risk Assessment Summary
|
||||
|
||||
| Category | Tests | Time Saved | Risk Level | Rationale |
|
||||
|----------|-------|------------|------------|-----------|
|
||||
| Seeds (non-critical) | 2 | 18.1s | ⚠️ Low | Critical bootstrap tests remain |
|
||||
| UserLive.ShowTest | 3 | 10.8s | ⚠️ Low | Core CRUD covered by Index/Form |
|
||||
| UserLive.IndexTest (UI) | 5 | 25.0s | ⚠️ Low | UI features, not core functionality |
|
||||
| Custom Fields Display | 6 | 8.5s | ⚠️ Low | Formatting tests, visible in code review |
|
||||
| RoleLive Tests | 7 | 7.7s | ⚠️ Low | Role management, not authorization |
|
||||
| Edge Cases | 1 | 1.5s | ⚠️ Low | Edge case, not critical path |
|
||||
| Performance Tests | 1 | 3.8s | ✅ Very Low | Explicit performance validation |
|
||||
| **Total** | **25** | **~77s** | **⚠️ Low** | |
|
||||
|
||||
**Overall Risk:** ⚠️ **Low** - All moved tests have low risk and don't catch critical regressions. Core functionality remains fully tested.
|
||||
|
||||
#### Tests Excluded from Full Test Suite
|
||||
|
||||
The following tests were **NOT** moved to full test suite (via promotion) despite being slow:
|
||||
|
||||
- **Policy Tests:** Medium risk - kept in fast suite (representative tests remain)
|
||||
- **UserLive.FormTest:** Medium risk - core CRUD functionality
|
||||
- **Tests <1s:** Don't meet execution time threshold
|
||||
- **Critical Bootstrap Tests:** High risk - deployment critical
|
||||
|
||||
---
|
||||
|
||||
## Current Performance Analysis
|
||||
|
||||
### Top 20 Slowest Tests (without `:slow`)
|
||||
|
||||
After implementing the full test suite via promotion, the remaining slowest tests are:
|
||||
|
||||
| Rank | Test | File | Time | Category |
|
||||
|------|------|------|------|----------|
|
||||
| 1 | `test Critical bootstrap invariants Mitglied system role exists` | `seeds_test.exs` | 6.7s | Critical Bootstrap |
|
||||
| 2 | `test Critical bootstrap invariants Admin user has Admin role` | `seeds_test.exs` | 5.0s | Critical Bootstrap |
|
||||
| 3 | `test normal_user permission set can read own user record` | `user_policies_test.exs` | 2.6s | Policy Test |
|
||||
| 4 | `test normal_user permission set can create member` | `member_policies_test.exs` | 2.5s | Policy Test |
|
||||
| 5-20 | Various Policy and LiveView tests | Multiple files | 1.5-2.4s each | Policy/LiveView |
|
||||
|
||||
**Total Top 20:** ~44 seconds (12% of total time without `:slow`)
|
||||
|
||||
**Note:** Many previously slow tests (UserLive.IndexTest, UserLive.ShowTest, Display/Formatting tests) are now tagged with `@tag :slow` and excluded from standard runs.
|
||||
|
||||
### Performance Hotspots Identified
|
||||
|
||||
#### 1. Seeds Tests (~16.2s for 4 tests)
|
||||
|
||||
**Status:** ✅ Optimized (reduced from 13 tests)
|
||||
**Remaining Optimization Potential:** 3-5 seconds
|
||||
|
||||
**Opportunities:**
|
||||
- Settings update could potentially be moved to `setup_all` (if sandbox allows)
|
||||
- Seeds execution could be further optimized (less data in test mode)
|
||||
- Idempotency test could be optimized (only 1x seeds instead of 2x)
|
||||
|
||||
#### 2. User LiveView Tests (~35.5s for 10 tests)
|
||||
|
||||
**Status:** ⏳ Identified for optimization
|
||||
**Optimization Potential:** 15-20 seconds
|
||||
|
||||
**Files:**
|
||||
- `test/mv_web/user_live/index_test.exs` (3 tests, ~10.2s)
|
||||
- `test/mv_web/user_live/form_test.exs` (4 tests, ~15.0s)
|
||||
- `test/mv_web/user_live/show_test.exs` (3 tests, ~10.3s)
|
||||
|
||||
**Patterns:**
|
||||
- Many tests create user/member data
|
||||
- LiveView mounts are expensive
|
||||
- Form submissions with validations are slow
|
||||
|
||||
**Recommended Actions:**
|
||||
- Move shared fixtures to `setup_all`
|
||||
- Reduce test data volume (3-5 users instead of 10+)
|
||||
- Optimize setup patterns for recurring patterns
|
||||
|
||||
#### 3. Policy Tests (~8.7s for 3 tests)
|
||||
|
||||
**Status:** ⏳ Identified for optimization
|
||||
**Optimization Potential:** 5-8 seconds
|
||||
|
||||
**Files:**
|
||||
- `test/mv/membership/member_policies_test.exs` (2 tests, ~6.1s)
|
||||
- `test/mv/accounts/user_policies_test.exs` (1 test, ~2.6s)
|
||||
|
||||
**Pattern:**
|
||||
- Each test creates new roles/users/members
|
||||
- Roles are identical across tests
|
||||
|
||||
**Recommended Actions:**
|
||||
- Create roles in `setup_all` (shared across tests)
|
||||
- Reuse common fixtures
|
||||
- Maintain test isolation while optimizing setup
|
||||
|
||||
---
|
||||
|
||||
## Future Optimization Opportunities
|
||||
|
||||
### Priority 1: User LiveView Tests Optimization
|
||||
|
||||
**Estimated Savings:** 14-22 seconds
|
||||
**Status:** 📋 Analysis Complete - Ready for Implementation
|
||||
|
||||
#### Analysis Summary
|
||||
|
||||
Analysis of User LiveView tests identified significant optimization opportunities:
|
||||
- **Framework functionality over-testing:** ~30 tests test Phoenix/Ash/Gettext core features
|
||||
- **Redundant test data creation:** Each test creates users/members independently
|
||||
- **Missing shared fixtures:** No `setup_all` usage for common data
|
||||
|
||||
#### Current Performance
|
||||
|
||||
**Top 20 Slowest Tests (User LiveView):**
|
||||
- `index_test.exs`: ~10.2s for 3 tests in Top 20
|
||||
- `form_test.exs`: ~15.0s for 4 tests in Top 20
|
||||
- `show_test.exs`: ~10.3s for 3 tests in Top 20
|
||||
- **Total:** ~35.5 seconds for User LiveView tests
|
||||
|
||||
#### Optimization Opportunities
|
||||
|
||||
**1. Remove Framework Functionality Tests (~30 tests, 8-12s saved)**
|
||||
- Remove translation tests (Gettext framework)
|
||||
- Remove navigation tests (Phoenix LiveView framework)
|
||||
- Remove validation tests (Ash framework)
|
||||
- Remove basic HTML rendering tests (consolidate into smoke test)
|
||||
- Remove password storage tests (AshAuthentication framework)
|
||||
|
||||
**2. Implement Shared Fixtures (3-5s saved)**
|
||||
- Use `setup_all` for common test data in `index_test.exs` and `show_test.exs`
|
||||
- Share users for sorting/checkbox tests
|
||||
- Share common users/members across tests
|
||||
- **Note:** `form_test.exs` uses `async: false`, preventing `setup_all` usage
|
||||
|
||||
**3. Consolidate Redundant Tests (~10 tests → 3-4 tests, 2-3s saved)**
|
||||
- Merge basic display tests into smoke test
|
||||
- Merge navigation tests into integration test
|
||||
- Reduce sorting tests to 1 integration test
|
||||
|
||||
**4. Optimize Test Data Volume (1-2s saved)**
|
||||
- Use minimum required data (2 users for sorting, 2 for checkboxes)
|
||||
- Share data across tests via `setup_all`
|
||||
|
||||
#### Tests to Keep (Business Logic)
|
||||
|
||||
**Index Tests:**
|
||||
- `initially sorts by email ascending` - Tests default sort
|
||||
- `can sort email descending by clicking sort button` - Tests sort functionality
|
||||
- `select all automatically checks when all individual users are selected` - Business logic
|
||||
- `does not show system actor user in list` - Business rule
|
||||
- `displays linked member name in user list` - Business logic
|
||||
- Edge case tests
|
||||
|
||||
**Form Tests:**
|
||||
- `creates user without password` - Business logic
|
||||
- `creates user with password when enabled` - Business logic
|
||||
- `admin sets new password for user` - Business logic
|
||||
- `selecting member and saving links member to user` - Business logic
|
||||
- Member linking/unlinking workflow tests
|
||||
|
||||
**Show Tests:**
|
||||
- `displays password authentication status` - Business logic
|
||||
- `displays linked member when present` - Business logic
|
||||
- `redirects to user list when viewing system actor user` - Business rule
|
||||
|
||||
#### Implementation Plan
|
||||
|
||||
**Phase 1: Remove Framework Tests (1-2 hours, ⚠️ Very Low Risk)**
|
||||
- Remove translation, navigation, validation, and basic HTML rendering tests
|
||||
- Consolidate remaining display tests into smoke test
|
||||
|
||||
**Phase 2: Implement Shared Fixtures (2-3 hours, ⚠️ Low Risk)**
|
||||
- Add `setup_all` to `index_test.exs` and `show_test.exs`
|
||||
- Update tests to use shared fixtures
|
||||
- Verify test isolation maintained
|
||||
|
||||
**Phase 3: Consolidate Tests (1-2 hours, ⚠️ Very Low Risk)**
|
||||
- Merge basic display tests into smoke test
|
||||
- Merge navigation tests into integration test
|
||||
- Reduce sorting tests to 1 integration test
|
||||
|
||||
**Risk Assessment:** ⚠️ **Low**
|
||||
- Framework functionality is tested by framework maintainers
|
||||
- Business logic tests remain intact
|
||||
- Shared fixtures maintain test isolation
|
||||
- Consolidation preserves coverage
|
||||
|
||||
### Priority 2: Policy Tests Optimization
|
||||
|
||||
**Estimated Savings:** 5.5-9 seconds
|
||||
**Status:** 📋 Analysis Complete - Ready for Decision
|
||||
|
||||
#### Analysis Summary
|
||||
|
||||
Analysis of policy tests identified significant optimization opportunities:
|
||||
- **Redundant fixture creation:** Roles and users created repeatedly across tests
|
||||
- **Framework functionality over-testing:** Many tests verify Ash policy framework behavior
|
||||
- **Test duplication:** Similar tests across different permission sets
|
||||
|
||||
#### Current Performance
|
||||
|
||||
**Policy Test Files Performance:**
|
||||
- `member_policies_test.exs`: 24 tests, ~66s (top 20)
|
||||
- `user_policies_test.exs`: 30 tests, ~66s (top 20)
|
||||
- `custom_field_value_policies_test.exs`: 20 tests, ~66s (top 20)
|
||||
- **Total:** 74 tests, ~152s total
|
||||
|
||||
**Top 20 Slowest Policy Tests:** ~66 seconds
|
||||
|
||||
#### Framework vs. Business Logic Analysis
|
||||
|
||||
**Framework Functionality (Should NOT Test):**
|
||||
- Policy evaluation (how Ash evaluates policies)
|
||||
- Permission lookup (how Ash looks up permissions)
|
||||
- Scope filtering (how Ash applies scope filters)
|
||||
- Auto-filter behavior (how Ash auto-filters queries)
|
||||
- Forbidden vs NotFound (how Ash returns errors)
|
||||
|
||||
**Business Logic (Should Test):**
|
||||
- Permission set definitions (what permissions each role has)
|
||||
- Scope definitions (what scopes each permission set uses)
|
||||
- Special cases (custom business rules)
|
||||
- Permission set behavior (how our permission sets differ)
|
||||
|
||||
#### Optimization Opportunities
|
||||
|
||||
**1. Remove Framework Functionality Tests (~22-34 tests, 3-4s saved)**
|
||||
- Remove "cannot" tests that verify error types (Forbidden, NotFound)
|
||||
- Remove tests that verify auto-filter behavior (framework)
|
||||
- Remove tests that verify permission evaluation (framework)
|
||||
- **Risk:** ⚠️ Very Low - Framework functionality is tested by Ash maintainers
|
||||
|
||||
**2. Consolidate Redundant Tests (~6-8 tests → 2-3 tests, 1-2s saved)**
|
||||
- Merge similar tests across permission sets
|
||||
- Create integration tests that cover multiple permission sets
|
||||
- **Risk:** ⚠️ Low - Same coverage, fewer tests
|
||||
|
||||
**3. Share Admin User Across Describe Blocks (1-2s saved)**
|
||||
- Create admin user once in module-level `setup`
|
||||
- Reuse admin user in helper functions
|
||||
- **Note:** `async: false` prevents `setup_all`, but module-level `setup` works
|
||||
- **Risk:** ⚠️ Low - Admin user is read-only in tests, safe to share
|
||||
|
||||
**4. Reduce Test Data Volume (0.5-1s saved)**
|
||||
- Use minimum required data
|
||||
- Share fixtures where possible
|
||||
- **Risk:** ⚠️ Very Low - Still tests same functionality
|
||||
|
||||
#### Test Classification Summary
|
||||
|
||||
**Tests to Remove (Framework):**
|
||||
- `member_policies_test.exs`: ~10 tests (cannot create/destroy/update, auto-filter tests)
|
||||
- `user_policies_test.exs`: ~16 tests (cannot read/update/create/destroy, auto-filter tests)
|
||||
- `custom_field_value_policies_test.exs`: ~8 tests (similar "cannot" tests)
|
||||
|
||||
**Tests to Consolidate (Redundant):**
|
||||
- `user_policies_test.exs`: 6 tests → 2 tests (can read/update own user record)
|
||||
|
||||
**Tests to Keep (Business Logic):**
|
||||
- All "can" tests that verify permission set behavior
|
||||
- Special case tests (e.g., "user can always READ linked member")
|
||||
- AshAuthentication bypass tests (our integration)
|
||||
|
||||
#### Implementation Plan
|
||||
|
||||
**Phase 1: Remove Framework Tests (1-2 hours, ⚠️ Very Low Risk)**
|
||||
- Identify all "cannot" tests that verify error types
|
||||
- Remove tests that verify Ash auto-filter behavior
|
||||
- Remove tests that verify permission evaluation (framework)
|
||||
|
||||
**Phase 2: Consolidate Redundant Tests (1-2 hours, ⚠️ Low Risk)**
|
||||
- Identify similar tests across permission sets
|
||||
- Create integration tests that cover multiple permission sets
|
||||
- Remove redundant individual tests
|
||||
|
||||
**Phase 3: Share Admin User (1-2 hours, ⚠️ Low Risk)**
|
||||
- Add module-level `setup` to create admin user
|
||||
- Update helper functions to accept admin user parameter
|
||||
- Update all `setup` blocks to use shared admin user
|
||||
|
||||
**Risk Assessment:** ⚠️ **Low**
|
||||
- Framework functionality is tested by Ash maintainers
|
||||
- Business logic tests remain intact
|
||||
- Admin user sharing maintains test isolation (read-only)
|
||||
- Consolidation preserves coverage
|
||||
|
||||
### Priority 3: Seeds Tests Further Optimization
|
||||
|
||||
**Estimated Savings:** 3-5 seconds
|
||||
|
||||
**Actions:**
|
||||
1. Investigate if settings update can be moved to `setup_all`
|
||||
2. Introduce seeds mode for tests (less data in test mode)
|
||||
3. Optimize idempotency test (only 1x seeds instead of 2x)
|
||||
|
||||
**Risk Assessment:** ⚠️ **Low to Medium**
|
||||
- Sandbox limitations may prevent `setup_all` usage
|
||||
- Seeds mode would require careful implementation
|
||||
- Idempotency test optimization needs to maintain test validity
|
||||
|
||||
### Priority 4: Additional Test Isolation Improvements
|
||||
|
||||
**Estimated Savings:** Variable (depends on specific tests)
|
||||
|
||||
**Actions:**
|
||||
1. Review tests that load all records (similar to the critical test fix)
|
||||
2. Add query filters where appropriate
|
||||
3. Ensure proper test isolation in async tests
|
||||
|
||||
**Risk Assessment:** ⚠️ **Very Low**
|
||||
- Similar to the critical test optimization (proven approach)
|
||||
- Improves test isolation and reliability
|
||||
|
||||
---
|
||||
|
||||
## Estimated Total Optimization Potential
|
||||
|
||||
| Priority | Optimization | Estimated Savings |
|
||||
|----------|-------------|-------------------|
|
||||
| 1 | User LiveView Tests | 14-22s |
|
||||
| 2 | Policy Tests | 5.5-9s |
|
||||
| 3 | Seeds Tests Further | 3-5s |
|
||||
| 4 | Additional Isolation | Variable |
|
||||
| **Total Potential** | | **22.5-36 seconds** |
|
||||
|
||||
**Projected Final Time:** From ~368 seconds (fast suite) to **~332-345 seconds** (~5.5-5.8 minutes) with remaining optimizations
|
||||
|
||||
**Note:** Detailed analysis documents available:
|
||||
- User LiveView Tests: See "Priority 1: User LiveView Tests Optimization" section above
|
||||
- Policy Tests: See "Priority 2: Policy Tests Optimization" section above
|
||||
|
||||
---
|
||||
|
||||
## Risk Assessment Summary
|
||||
|
||||
### Overall Risk Level: ⚠️ **Low**
|
||||
|
||||
All optimizations maintain test coverage while improving performance:
|
||||
|
||||
| Optimization | Risk Level | Mitigation |
|
||||
|-------------|------------|------------|
|
||||
| Seeds tests reduction | ⚠️ Low | Coverage mapped to domain tests |
|
||||
| Performance tests tagging | ✅ Very Low | Tests still executed, just separately |
|
||||
| Critical test optimization | ✅ Very Low | Functionality unchanged, better isolation |
|
||||
| Future optimizations | ⚠️ Low | Careful implementation with verification |
|
||||
|
||||
### Monitoring Plan
|
||||
|
||||
#### Success Criteria
|
||||
|
||||
- ✅ Seeds tests execute in <20 seconds consistently
|
||||
- ✅ No increase in seeds-related deployment failures
|
||||
- ✅ No regression in authorization or membership fee bugs
|
||||
- ✅ Top 20 slowest tests: < 60 seconds (currently ~44s)
|
||||
- ✅ Total execution time (without `:slow`): < 10 minutes (currently 6.1 min)
|
||||
- ⏳ Slow tests execution time: < 2 minutes (currently ~1.3 min)
|
||||
|
||||
#### What to Watch For
|
||||
|
||||
1. **Production Seeds Failures:**
|
||||
- Monitor deployment logs for seeds errors
|
||||
- If failures increase, consider restoring detailed tests
|
||||
|
||||
2. **Authorization Bugs After Seeds Changes:**
|
||||
- If role/permission bugs appear after seeds modifications
|
||||
- May indicate need for more seeds-specific role validation
|
||||
|
||||
3. **Test Performance Regression:**
|
||||
- Monitor test execution times in CI
|
||||
- Alert if times increase significantly
|
||||
|
||||
4. **Developer Feedback:**
|
||||
- If developers report missing test coverage
|
||||
- Adjust based on real-world experience
|
||||
|
||||
---
|
||||
|
||||
## Benchmarking and Analysis
|
||||
|
||||
### How to Benchmark Tests
|
||||
|
||||
**ExUnit Built-in Benchmarking:**
|
||||
|
||||
The test suite is configured to show the slowest tests automatically:
|
||||
|
||||
```elixir
|
||||
# test/test_helper.exs
|
||||
ExUnit.start(
|
||||
slowest: 10 # Shows 10 slowest tests at the end of test run
|
||||
)
|
||||
```
|
||||
|
||||
**Run Benchmark Analysis:**
|
||||
|
||||
```bash
|
||||
# Show slowest tests
|
||||
mix test --slowest 20
|
||||
|
||||
# Exclude slow tests for faster feedback
|
||||
mix test --exclude slow --slowest 20
|
||||
|
||||
# Run only slow tests
|
||||
mix test --only slow --slowest 10
|
||||
|
||||
# Benchmark specific test file
|
||||
mix test test/mv_web/member_live/index_member_fields_display_test.exs --slowest 5
|
||||
```
|
||||
|
||||
### Benchmarking Best Practices
|
||||
|
||||
1. **Run benchmarks regularly** (e.g., monthly) to catch performance regressions
|
||||
2. **Compare isolated vs. full runs** to identify test isolation issues
|
||||
3. **Monitor CI execution times** to track trends over time
|
||||
4. **Document significant changes** in test performance
|
||||
|
||||
---
|
||||
|
||||
## Test Suite Structure
|
||||
|
||||
### Test Execution Modes
|
||||
|
||||
**Fast Tests (Default):**
|
||||
- Excludes slow tests (`@tag :slow`)
|
||||
- Used for standard development workflow
|
||||
- Execution time: ~6 minutes
|
||||
- Command: `mix test --exclude slow` or `just test-fast`
|
||||
|
||||
**Slow Tests:**
|
||||
- Tests tagged with `@tag :slow` or `@describetag :slow` (25 tests)
|
||||
- Low risk, >1 second execution time
|
||||
- UI/Display tests, workflow details, edge cases, performance tests
|
||||
- Execution time: ~1.3 minutes
|
||||
- Command: `mix test --only slow` or `just test-slow`
|
||||
- Excluded from standard CI runs
|
||||
|
||||
**Full Test Suite (via Promotion):**
|
||||
- Triggered by promoting a build to `production` in Drone CI
|
||||
- Runs all tests (`mix test`) for comprehensive coverage
|
||||
- Execution time: ~7.4 minutes
|
||||
- Required before merging to `main` (enforced via branch protection)
|
||||
|
||||
**All Tests:**
|
||||
- Includes both fast and slow tests
|
||||
- Used for comprehensive validation (pre-merge)
|
||||
- Execution time: ~7.4 minutes
|
||||
- Command: `mix test` or `just test`
|
||||
|
||||
### Test Organization
|
||||
|
||||
Tests are organized to mirror the `lib/` directory structure:
|
||||
Tests mirror the `lib/` structure:
|
||||
|
||||
```
|
||||
test/
|
||||
├── accounts/ # Accounts domain tests
|
||||
├── membership/ # Membership domain tests
|
||||
├── membership_fees/ # Membership fees domain tests
|
||||
├── mv/ # Core application tests
|
||||
│ ├── accounts/ # User-related tests
|
||||
│ ├── membership/ # Member-related tests
|
||||
│ └── authorization/ # Authorization tests
|
||||
├── mv_web/ # Web layer tests
|
||||
│ ├── controllers/ # Controller tests
|
||||
│ ├── live/ # LiveView tests
|
||||
│ └── components/ # Component tests
|
||||
└── support/ # Test helpers
|
||||
├── conn_case.ex # Controller test setup
|
||||
└── data_case.ex # Database test setup
|
||||
├── accounts/ # Accounts domain
|
||||
├── membership/ # Membership domain
|
||||
├── membership_fees/ # Membership fees domain
|
||||
├── mv/ # Core (accounts, membership, authorization)
|
||||
├── mv_web/ # Web layer (controllers, live, components)
|
||||
└── support/ # conn_case.ex, data_case.ex
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices for Test Performance
|
||||
|
||||
### When Writing New Tests
|
||||
|
||||
1. **Use `async: true`** when possible (for parallel execution)
|
||||
2. **Filter queries** to only load necessary data
|
||||
3. **Share fixtures** in `setup_all` when appropriate
|
||||
4. **Tag performance tests** with `@tag :slow` if they use large datasets
|
||||
5. **Keep test data minimal** - only create what's needed for the test
|
||||
|
||||
### When Optimizing Existing Tests
|
||||
|
||||
1. **Measure first** - Use `mix test --slowest` to identify bottlenecks
|
||||
2. **Compare isolated vs. full runs** - Identify test isolation issues
|
||||
3. **Optimize setup** - Move shared data to `setup_all` where possible
|
||||
4. **Filter queries** - Only load data needed for the test
|
||||
5. **Verify coverage** - Ensure optimizations don't reduce test coverage
|
||||
|
||||
### Test Tagging Guidelines
|
||||
|
||||
#### Tag as `@tag :slow` when:
|
||||
|
||||
1. **Performance Tests:**
|
||||
- Explicitly testing performance characteristics
|
||||
- Using large datasets (50+ records)
|
||||
- Testing scalability or query optimization
|
||||
- Validating N+1 query prevention
|
||||
|
||||
2. **Low-Risk Tests (>1s):**
|
||||
- UI/Display/Formatting tests (not critical for every commit)
|
||||
- Workflow detail tests (not core functionality)
|
||||
- Edge cases with large datasets
|
||||
- Show page tests (core functionality covered by Index/Form tests)
|
||||
- Non-critical seeds tests (smoke tests, idempotency)
|
||||
|
||||
#### Do NOT tag as `@tag :slow` when:
|
||||
|
||||
- ❌ Test is slow due to inefficient setup (fix the setup instead)
|
||||
- ❌ Test is slow due to bugs (fix the bug instead)
|
||||
- ❌ Core CRUD operations (Member/User Create/Update/Destroy)
|
||||
- ❌ Basic Authentication/Authorization
|
||||
- ❌ Critical Bootstrap (Admin user, system roles)
|
||||
- ❌ Email Synchronization
|
||||
- ❌ Representative Policy tests (one per Permission Set + Action)
|
||||
- ❌ It's an integration test (use `@tag :integration` instead)
|
||||
|
||||
---
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-01-28: Initial Optimization Phase
|
||||
|
||||
**Completed:**
|
||||
- ✅ Reduced seeds tests from 13 to 4 tests
|
||||
- ✅ Tagged 9 performance tests with `@tag :slow`
|
||||
- ✅ Optimized critical test with query filtering
|
||||
- ✅ Created slow test suite infrastructure
|
||||
- ✅ Updated CI/CD to exclude slow tests from standard runs
|
||||
- ✅ Added promotion-based full test suite pipeline (`check-full`)
|
||||
|
||||
**Time Saved:** ~21-30 seconds per test run
|
||||
|
||||
### 2026-01-28: Full Test Suite via Promotion Implementation
|
||||
|
||||
**Completed:**
|
||||
- ✅ Analyzed all tests for full test suite candidates
|
||||
- ✅ Identified 36 tests with low risk and >1s execution time
|
||||
- ✅ Tagged 25 tests with `@tag :slow` for full test suite (via promotion)
|
||||
- ✅ Categorized tests by risk level and execution time
|
||||
- ✅ Documented tagging criteria and guidelines
|
||||
|
||||
**Tests Tagged:**
|
||||
- 2 Seeds tests (non-critical) - 18.1s
|
||||
- 3 UserLive.ShowTest tests - 10.8s
|
||||
- 5 UserLive.IndexTest tests - 25.0s
|
||||
- 3 MemberLive.IndexCustomFieldsDisplayTest tests - 4.9s
|
||||
- 3 MemberLive.IndexCustomFieldsEdgeCasesTest tests - 3.6s
|
||||
- 7 RoleLive tests - 7.7s
|
||||
- 1 MemberAvailableForLinkingTest - 1.5s
|
||||
- 1 Performance test (already tagged) - 3.8s
|
||||
|
||||
**Time Saved:** ~77 seconds per test run
|
||||
|
||||
**Total Optimization Impact:**
|
||||
- **Before:** ~445 seconds (7.4 minutes)
|
||||
- **After (fast suite):** ~368 seconds (6.1 minutes)
|
||||
- **Time saved:** ~77 seconds (17% reduction)
|
||||
|
||||
**Next Steps:**
|
||||
- ⏳ Monitor full test suite execution via promotion in CI
|
||||
- ⏳ Optimize remaining slow tests (Policy tests, etc.)
|
||||
- ⏳ Further optimize Seeds tests (Priority 3)
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- **Testing Standards:** `CODE_GUIDELINES.md` - Section 4 (Testing Standards)
|
||||
- **CI/CD Configuration:** `.drone.yml`
|
||||
- **Test Helper:** `test/test_helper.exs`
|
||||
- **Justfile Commands:** `Justfile` (test-fast, test-slow, test-all)
|
||||
|
||||
---
|
||||
|
||||
## Questions & Answers
|
||||
|
||||
**Q: What if seeds create wrong data and break the system?**
|
||||
A: The smoke test will fail if seeds raise errors. Domain tests ensure business logic is correct regardless of seeds content.
|
||||
|
||||
**Q: What if we add a new critical bootstrap requirement?**
|
||||
A: Add a new test to the "Critical bootstrap invariants" section in `test/seeds_test.exs`.
|
||||
|
||||
**Q: How do we know the removed tests aren't needed?**
|
||||
A: Monitor for 2-3 months. If no seeds-related bugs appear that would have been caught by removed tests, they were redundant.
|
||||
|
||||
**Q: Should we restore the tests for important releases?**
|
||||
A: Consider running the full test suite (including slow tests) before major releases. Daily development uses the optimized suite.
|
||||
|
||||
**Q: How do I add a new performance test?**
|
||||
A: Tag it with `@tag :slow` for individual tests or `@describetag :slow` for describe blocks. Use `@describetag` instead of `@moduletag` to avoid tagging unrelated tests. Include measurable performance assertions (query counts, timing with tolerance, etc.). See "Performance Test Guidelines" section above.
|
||||
|
||||
**Q: Can I run slow tests locally?**
|
||||
A: Yes, use `just test-slow` or `mix test --only slow`. They're excluded from standard runs for faster feedback.
|
||||
|
||||
**Q: What is the "full test suite"?**
|
||||
A: The full test suite runs **all tests** (`mix test`), including slow and UI tests. Tests tagged with `@tag :slow` or `@describetag :slow` are excluded from standard CI runs (`check-fast`) for faster feedback, but are included when promoting a build to `production` (`check-full`) before merging to `main`.
|
||||
|
||||
**Q: Which tests should I tag as `:slow`?**
|
||||
A: Tag tests with `@tag :slow` if they: (1) take >1 second, (2) have low risk (not critical for catching regressions), and (3) test UI/Display/Formatting or workflow details. See "Test Tagging Guidelines" section for details.
|
||||
|
||||
**Q: What if a slow test fails in the full test suite?**
|
||||
A: If a test in the full test suite fails, investigate the failure. If it indicates a critical regression, consider moving it back to the fast suite. If it's a flaky test, fix the test itself. The merge will be blocked until all tests pass.
|
||||
- Testing Standards: `CODE_GUIDELINES.md` §4
|
||||
- CI/CD: `.drone.jsonnet`
|
||||
- Test helper: `test/test_helper.exs`
|
||||
- Just commands: `Justfile` (`test-fast`, `test-slow`, `test`)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue