local-it/mitgliederverwaltung
local-it/mitgliederverwaltung
2
3
Fork 0
Code Issues 57 Pull requests 3 Projects 2 Releases 5 Packages 1 Activity Actions

docs(db): refresh, condense and align database and groups docs

Browse source
This commit is contained in:
Moritz 2026-06-15 21:53:36 +02:00
parent 5d8f173529
commit 0b36a43edc
4 changed files with 360 additions and 1875 deletions
Download patch file Download diff file Expand all files Collapse all files

240
docs/custom-fields-search-performance.md
View file

@ -2,242 +2,88 @@
## Current Implementation ## Current Implementation
The search vector includes custom field values via database triggers that: The member `search_vector` includes custom field values via database triggers that aggregate all of a member's custom field values, extract the value from each JSONB record (`value->>'_union_value'`), and add them at weight `C`.
1. Aggregate all custom field values for a member
2. Extract values from JSONB format
3. Add them to the search_vector with weight 'C'
## Performance Considerations Two triggers maintain the vector:
### 1. Trigger Performance on Member Updates - `members_search_vector_trigger()` — fires on `members` INSERT/UPDATE; runs a subquery `SELECT string_agg(...) FROM custom_field_values WHERE member_id = NEW.id`.
- `update_member_search_vector_from_custom_field_value()` — fires on `custom_field_values` INSERT/UPDATE/DELETE; re-aggregates and updates the member's `search_vector`.
**Current Implementation:** Both rely on `custom_field_values_member_id_idx`, so the per-member aggregation is an indexed lookup.
- `members_search_vector_trigger()` executes a subquery on every INSERT/UPDATE:
```sql
SELECT string_agg(...) FROM custom_field_values WHERE member_id = NEW.id
```
**Performance Impact:** ## Applied Trigger Optimizations
- ✅ **Good:** Index on `member_id` exists (`custom_field_values_member_id_idx`)
- ✅ **Good:** Subquery only runs for the affected member
- ⚠️ **Potential Issue:** With many custom fields per member (e.g., 50+), aggregation could be slower
- ⚠️ **Potential Issue:** JSONB extraction (`value->>'_union_value'`) is relatively fast but adds overhead
**Expected Performance:** `update_member_search_vector_from_custom_field_value()` was optimized:
- **Small scale (< 10 custom fields per member):** Negligible impact (< 5ms per operation)
- **Medium scale (10-30 custom fields):** Minor impact (5-20ms per operation)
- **Large scale (30+ custom fields):** Noticeable impact (20-50ms+ per operation)
### 2. Trigger Performance on Custom Field Value Changes - **Fetch only required member fields** (first_name, last_name, email, etc.) instead of the full record — reduces per-call overhead by roughly 30–50%.
- **Early return on UPDATE when the value is unchanged** — skips the expensive re-aggregation entirely.
**Current Implementation:** Measured effect per custom-field-value change:
- `update_member_search_vector_from_custom_field_value()` executes on every INSERT/UPDATE/DELETE on `custom_field_values`
- **Optimized:** Only fetches required member fields (not full record) to reduce overhead
- **Optimized:** Skips re-aggregation on UPDATE if value hasn't actually changed
- Aggregates all custom field values, then updates member search_vector
**Performance Impact:** | Case | Before | After |
- ✅ **Good:** Index on `member_id` ensures fast lookup |------|--------|-------|
- ✅ **Optimized:** Only required fields are fetched (first_name, last_name, email, etc.) instead of full record | Value changed | 5–15 ms | 3–10 ms |
- ✅ **Optimized:** UPDATE operations that don't change the value skip expensive re-aggregation (early return) | Value unchanged (UPDATE) | 5–15 ms | < 1 ms (early return) |
- ⚠️ **Note:** Re-aggregation is still necessary when values change (required for search_vector consistency)
- ⚠️ **Critical:** Bulk operations (e.g., importing 1000 members with custom fields) will trigger this for each row
**Expected Performance:** Re-aggregation is still required whenever a value actually changes — that is necessary for `search_vector` consistency.
- **Single operation (value changed):** 3-10ms per custom field value change (improved from 5-15ms)
- **Single operation (value unchanged):** <1ms (early return, no aggregation)
- **Bulk operations:** Could be slow (consider disabling trigger temporarily)
### 3. Search Vector Size ## Search Vector Size
**Current Constraints:** - String custom field values are capped at **10,000 characters each**; there is no cap on the number of custom fields per member.
- String values: max 10,000 characters per custom field - `tsvector` has no hard size limit, but very large vectors (> ~100 KB) degrade GIN index maintenance, tsvector operations, and trigger time. Worst case: 100 fields × 10,000 chars ≈ 1 MB of aggregated text for one member.
- No limit on number of custom fields per member - **Recommendation:** monitor `search_vector` size in production; consider capping total custom-field content per member if large vectors appear.
- tsvector has no explicit size limit, but very large vectors can cause issues
**Potential Issues:** ## Bulk Imports
- **Theoretical maximum:** If a member has 100 custom fields with 10,000 char strings each, the aggregated text could be ~1MB
- **Practical concern:** Very large search vectors (> 100KB) can slow down:
- Index updates (GIN index maintenance)
- Search queries (tsvector operations)
- Trigger execution time
**Recommendation:** The custom-field-value trigger fires once per row, so importing many members with custom fields is expensive. For bulk imports, **temporarily disable the `custom_field_values` trigger**, then re-aggregate `search_vector` in a batch after the import. The initial backfill migration also updates all members in a single transaction (table lock); for > 10,000 members, batch the backfill and run during a maintenance window.
- Monitor search_vector size in production
- Consider limiting total custom field content per member if needed
- PostgreSQL can handle large tsvectors, but performance degrades gradually
### 4. Initial Migration Performance ## Search Query Structure
**Current Implementation:** Full-text search uses the GIN index on `search_vector` (fast). Substring/custom-field matching adds `EXISTS (SELECT 1 FROM custom_field_values WHERE member_id = id AND ... LIKE ...)` subqueries, which are **not indexed** on the JSONB value (sequential scan) and run even when the FTS branch already matches. This is the main known weakness; it is acceptable at the current scale (< 30 custom fields/member, < 10,000 members) but is the first thing to revisit if search slows.
- Updates ALL members in a single transaction:
```sql
UPDATE members m SET search_vector = ... (subquery for each member)
```
**Performance Impact:** ## Search Filter Functions
- ⚠️ **Potential Issue:** With 10,000+ members, this could take minutes
- ⚠️ **Potential Issue:** Single transaction locks the members table
- ⚠️ **Potential Issue:** If migration fails, entire rollback required
**Recommendation:** The search query in `lib/membership/member.ex` is split into modular filter builders, combined as a single OR-chain in priority order:
- For large datasets (> 10,000 members), consider:
- Batch updates (e.g., 1000 members at a time)
- Run during maintenance window
- Monitor progress
### 5. Search Query Performance 1. `build_fts_filter/1` — full-text search (highest priority, GIN-indexed, fastest).
2. `build_substring_filter/2` — `ILIKE` substring matching on structured fields (postal_code, house_number, email, city, country).
3. `build_custom_field_filter/1` — JSONB custom-field value matching via `EXISTS` subquery.
4. `build_fuzzy_filter/2` — trigram fuzzy matching on first_name, last_name, street (pg_trgm).
**Current Implementation:** Priority: **FTS > Substring > Custom Fields > Fuzzy**.
- Full-text search uses GIN index on `search_vector` (fast)
- Additional LIKE queries on `custom_field_values` for substring matching:
```sql
EXISTS (SELECT 1 FROM custom_field_values WHERE member_id = id AND ... LIKE ...)
```
**Performance Impact:**
- ✅ **Good:** GIN index on `search_vector` is very fast
- ⚠️ **Potential Issue:** LIKE queries on JSONB are not indexed (sequential scan)
- ⚠️ **Potential Issue:** EXISTS subquery runs for every search, even if search_vector match is found
- ⚠️ **Potential Issue:** With many custom fields, the LIKE queries could be slow
**Expected Performance:**
- **With GIN index match:** Very fast (< 10ms for typical queries)
- **Without GIN index match (fallback to LIKE):** Slower (10-100ms depending on data size)
- **Worst case:** Sequential scan of all custom_field_values for all members
## Recommendations
### Short-term (Current Implementation)
1. **Monitor Performance:**
- Add logging for trigger execution time
- Monitor search_vector size distribution
- Track search query performance
2. **Index Verification:**
- Ensure `custom_field_values_member_id_idx` exists and is used
- Verify GIN index on `search_vector` is maintained
3. **Bulk Operations:**
- For bulk imports, consider temporarily disabling the custom_field_values trigger
- Re-enable and update search_vectors in batch after import
### Medium-term Optimizations
1. **✅ Optimize Trigger Function (FULLY IMPLEMENTED):**
- ✅ Only fetch required member fields instead of full record (reduces overhead)
- ✅ Skip re-aggregation on UPDATE if value hasn't actually changed (early return optimization)
2. **Limit Search Vector Size:**
- Truncate very long custom field values (e.g., first 1000 chars)
- Add warning if aggregated text exceeds threshold
3. **Optimize LIKE Queries:**
- Consider adding a generated column for searchable text
- Or use a materialized view for custom field search
### Long-term Considerations
1. **Alternative Approaches:**
- Separate search index table for custom fields
- Use Elasticsearch or similar for advanced search
- Materialized view for search optimization
2. **Scaling Strategy:**
- If performance becomes an issue with 100+ custom fields per member:
- Consider limiting which custom fields are searchable
- Use a separate search service
- Implement search result caching
## Performance Benchmarks (Estimated)
Based on typical PostgreSQL performance:
| Scenario | Members | Custom Fields/Member | Expected Impact |
|----------|---------|---------------------|-----------------|
| Small | < 1,000 | < 10 | Negligible (< 5ms per operation) |
| Medium | 1,000-10,000 | 10-30 | Minor (5-20ms per operation) |
| Large | 10,000-100,000 | 30-50 | Noticeable (20-50ms per operation) |
| Very Large | > 100,000 | 50+ | Significant (50-200ms+ per operation) |
## Monitoring Queries ## Monitoring Queries
```sql ```sql
-- Check search_vector size distribution -- search_vector size distribution
SELECT SELECT
pg_size_pretty(octet_length(search_vector::text)) as size, pg_size_pretty(octet_length(search_vector::text)) AS size,
COUNT(*) as member_count COUNT(*) AS member_count
FROM members FROM members
WHERE search_vector IS NOT NULL WHERE search_vector IS NOT NULL
GROUP BY octet_length(search_vector::text) GROUP BY octet_length(search_vector::text)
ORDER BY octet_length(search_vector::text) DESC ORDER BY octet_length(search_vector::text) DESC
LIMIT 20; LIMIT 20;
-- Check average custom fields per member -- average / max custom fields per member
SELECT SELECT
AVG(custom_field_count) as avg_custom_fields, AVG(custom_field_count) AS avg_custom_fields,
MAX(custom_field_count) as max_custom_fields MAX(custom_field_count) AS max_custom_fields
FROM ( FROM (
SELECT member_id, COUNT(*) as custom_field_count SELECT member_id, COUNT(*) AS custom_field_count
FROM custom_field_values FROM custom_field_values
GROUP BY member_id GROUP BY member_id
) subq; ) subq;
-- Check trigger execution time (requires pg_stat_statements) -- trigger execution time (requires pg_stat_statements)
SELECT SELECT mean_exec_time, calls, query
mean_exec_time,
calls,
query
FROM pg_stat_statements FROM pg_stat_statements
WHERE query LIKE '%members_search_vector_trigger%' WHERE query LIKE '%members_search_vector_trigger%'
ORDER BY mean_exec_time DESC; ORDER BY mean_exec_time DESC;
``` ```
## Code Quality Improvements (Post-Review) ## Future Options (if scale demands)
### Refactored Search Implementation
The search query has been refactored for better maintainability and clarity:
**Before:** Single large OR-chain with mixed search types (hard to maintain)
**After:** Modular functions grouped by search type:
- `build_fts_filter/1` - Full-text search (highest priority, fastest)
- `build_substring_filter/2` - Substring matching on structured fields
- `build_custom_field_filter/1` - Custom field value search (JSONB LIKE)
- `build_fuzzy_filter/2` - Trigram/fuzzy matching for names and streets
**Benefits:**
- ✅ Clear separation of concerns
- ✅ Easier to maintain and test
- ✅ Better documentation of search priority
- ✅ Easier to optimize individual search types
**Search Priority Order:**
1. **FTS (Full-Text Search)** - Fastest, uses GIN index on search_vector
2. **Substring** - For structured fields (postal_code, phone_number, etc.)
3. **Custom Fields** - JSONB LIKE queries (fallback for substring matching)
4. **Fuzzy Matching** - Trigram similarity for names and streets
## Conclusion
The current implementation is **well-optimized for typical use cases** (< 30 custom fields per member, < 10,000 members). For larger scales, monitoring and potential optimizations may be needed.
**Key Strengths:**
- Indexed lookups (member_id index)
- Efficient GIN index for search
- Trigger-based automatic updates
- Modular, maintainable search code structure
**Key Weaknesses:**
- LIKE queries on JSONB (not indexed)
- Re-aggregation on every custom field change (necessary for consistency)
- Potential size issues with many/large custom fields
- Substring searches (contains/ILIKE) not index-optimized
**Recent Optimizations:**
- ✅ Trigger function optimized to fetch only required fields (reduces overhead by ~30-50%)
- ✅ Early return on UPDATE when value hasn't changed (skips expensive re-aggregation, <1ms vs 3-10ms)
- ✅ Improved performance for custom field value updates (3-10ms vs 5-15ms when value changes)
- Generated/searchable text column or materialized view for custom-field substring search (to escape the unindexed JSONB `LIKE`).
- Limit which custom fields are searchable, or truncate long values.
- External search service (e.g., Elasticsearch) for advanced search.

521
docs/database-schema-readme.md
View file

@ -4,105 +4,54 @@
This document provides a comprehensive overview of the Mila Membership Management System database schema. This document provides a comprehensive overview of the Mila Membership Management System database schema.
## Quick Links - **DBML file:** [`database_schema.dbml`](./database_schema.dbml) — full per-column intent notes and relationship edges.
- **Search-vector performance:** see [`custom-fields-search-performance.md`](./custom-fields-search-performance.md) for trigger cost analysis and tuning.
- **DBML File:** [`database_schema.dbml`](./database_schema.dbml) The DBML is **hand-maintained** (not auto-generated); keep it in sync with `priv/repo/migrations/`.
- **Visualize Online:**
- [dbdiagram.io](https://dbdiagram.io) - Upload the DBML file
- [dbdocs.io](https://dbdocs.io) - Generate interactive documentation
## Schema Statistics ## Schema Statistics
| Metric | Count | | Metric | Count |
|--------|-------| |--------|-------|
| **Tables** | 11 | | **Tables** | 12 |
| **Domains** | 4 (Accounts, Membership, MembershipFees, Authorization) | | **Domains** | 4 (Accounts, Membership, MembershipFees, Authorization) |
| **Relationships** | 9 | | **Triggers** | 3 (member, custom_field_values, member_groups → member search-vector) |
| **Indexes** | 25+ |
| **Triggers** | 1 (Full-text search) |
## Tables Overview ## Tables Overview
### Accounts Domain ### Accounts Domain
- **`users`** — authentication accounts. Dual auth (Password + OIDC), optional 1:1 link to a member; email is the source of truth when linked.
- **`tokens`** — JWT storage for AshAuthentication; multiple purposes, revocation by deletion.
#### `users` OIDC account linking is recorded on the `users` table via the `oidc_id` column; there is no separate `user_identities` table.
- **Purpose:** User authentication and session management
- **Rows (Estimated):** Low to Medium (typically 10-50% of members)
- **Key Features:**
- Dual authentication (Password + OIDC)
- Optional 1:1 link to members
- Email as source of truth when linked
#### `tokens`
- **Purpose:** JWT token storage for AshAuthentication
- **Rows (Estimated):** Medium to High (multiple tokens per user)
- **Key Features:**
- Token lifecycle management
- Revocation support
- Multiple token purposes
### Membership Domain ### Membership Domain
- **`members`** — club member master data. Full-text + fuzzy search, bidirectional email sync with users, flexible address/contact data, `country`, optional `vereinfacht_contact_id` (external vereinfacht.de contact).
#### `members` - **`custom_field_values`** — dynamic per-member attributes. Union-type value in JSONB; one value per custom field per member.
- **Purpose:** Club member master data - **`custom_fields`** — schema definitions for custom field values (type, `required`/`show_in_overview` flags, optional `join_description`, auto-generated slug).
- **Rows (Estimated):** High (core entity) - **`settings`** — global application settings (singleton). Club name (also via `ASSOCIATION_NAME` env), member-field visibility/required maps, fee defaults, plus OIDC, SMTP/mail-from, vereinfacht.de, public join-form, `registration_enabled`, and `oidc_only` configuration. See [Settings configuration columns](#settings-configuration-columns).
- **Key Features:** - **`groups`** — member groupings. Case-insensitive-unique names, auto-generated immutable slugs, optional descriptions; many-to-many with members.
- Complete member profile - **`member_groups`** — join table for members ↔ groups. Unique `(member_id, group_id)`, CASCADE delete on both sides (join table only).
- Full-text search via tsvector - **`join_requests`** — public join flow (onboarding, double opt-in). Status machine `pending_confirmation → submitted → approved/rejected`; confirmation token stored as hash only, ~24h retention for unconfirmed records.
- Bidirectional email sync with users
- Flexible address and contact data
#### `custom_field_values`
- **Purpose:** Dynamic custom member attributes
- **Rows (Estimated):** Variable (N per member)
- **Key Features:**
- Union type value storage (JSONB)
- Multiple data types supported
- One custom field value per custom field per member
#### `custom_fields`
- **Purpose:** Schema definitions for custom_field_values
- **Rows (Estimated):** Low (admin-defined)
- **Key Features:**
- Type definitions
- Immutable and required flags
- Centralized custom field management
#### `settings`
- **Purpose:** Global application settings (singleton resource)
- **Rows (Estimated):** 1 (singleton pattern)
- **Key Features:**
- Club name configuration
- Member field visibility settings
- Membership fee default settings
- Environment variable support for club name
#### `groups`
- **Purpose:** Group definitions for organizing members
- **Rows (Estimated):** Low (typically 5-20 groups per club)
- **Key Features:**
- Unique group names (case-insensitive)
- URL-friendly slugs (auto-generated, immutable)
- Optional descriptions
- Many-to-many relationship with members
#### `member_groups`
- **Purpose:** Join table for many-to-many relationship between members and groups
- **Rows (Estimated):** Medium to High (multiple groups per member)
- **Key Features:**
- Unique constraint on (member_id, group_id)
- CASCADE delete on both sides
- Efficient indexes for queries
### Authorization Domain ### Authorization Domain
- **`roles`** — RBAC. Links users to one of four hardcoded permission sets (`own_data`, `read_only`, `normal_user`, `admin`); system roles are deletion-protected.
#### `roles` ### MembershipFees Domain
- **Purpose:** Role-based access control (RBAC) - **`membership_fee_types`** — fee types with immutable billing interval.
- **Rows (Estimated):** Low (typically 3-10 roles) - **`membership_fee_cycles`** — per-member billing cycles with payment status.
- **Key Features:**
- Links users to permission sets ## Settings configuration columns
- System role protection
- Four hardcoded permission sets: own_data, read_only, normal_user, admin The singleton `settings` row carries runtime configuration (all nullable unless noted). Grouped by area:
- **Member overview:** `member_field_visibility` (JSONB; absent key = visible), `member_field_required` (JSONB).
- **Membership fees:** `include_joining_cycle` (bool, NOT NULL, default true), `default_membership_fee_type_id` (FK → membership_fee_types, ON DELETE SET NULL).
- **Registration / login:** `registration_enabled` (bool, NOT NULL, default true), `oidc_only` (bool, NOT NULL, default false).
- **OIDC:** `oidc_client_id`, `oidc_client_secret`, `oidc_base_url`, `oidc_redirect_uri`, `oidc_admin_group_name`, `oidc_groups_claim`.
- **SMTP / mail-from:** `smtp_host`, `smtp_port` (bigint), `smtp_username`, `smtp_password`, `smtp_ssl`, `smtp_from_name`, `smtp_from_email`.
- **vereinfacht.de:** `vereinfacht_api_url`, `vereinfacht_api_key`, `vereinfacht_club_id`, `vereinfacht_app_url`.
- **Public join form:** `join_form_enabled` (bool, NOT NULL, default false), `join_form_field_ids` (text[]), `join_form_field_required` (JSONB).
## Key Relationships ## Key Relationships
@ -124,123 +73,54 @@ Member (N) ←→ (N) Group
Settings (1) → MembershipFeeType (0..1) Settings (1) → MembershipFeeType (0..1)
``` ```
### Relationship Details ## Foreign Key On-Delete Behavior
1. **User ↔ Member (Optional 1:1, both sides optional)** | Relationship | On Delete | Rationale |
- A User can have 0 or 1 Member (`user.member_id` can be NULL) |--------------|-----------|-----------|
- A Member can have 0 or 1 User (optional `has_one` relationship) | `users.member_id → members.id` | SET NULL | Preserve user account when member deleted |
- Both entities can exist independently | `users.role_id → roles.id` | RESTRICT | Cannot delete a role that still has users |
- Email synchronization when linked (User.email is source of truth) | `custom_field_values.member_id → members.id` | CASCADE | Delete values with member |
- `ON DELETE SET NULL` on user side (User preserved when Member deleted) | `custom_field_values.custom_field_id → custom_fields.id` | CASCADE | Delete values when the custom field is deleted |
| `members.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type assigned to members |
| `membership_fee_cycles.member_id → members.id` | CASCADE | Cycles deleted with member |
| `membership_fee_cycles.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type with cycles |
| `settings.default_membership_fee_type_id → membership_fee_types.id` | SET NULL | Clear default if fee type deleted |
| `member_groups.member_id → members.id` | CASCADE | Association removed; member preserved |
| `member_groups.group_id → groups.id` | CASCADE | Association removed; group preserved |
2. **User → Role (N:1)** `join_requests.reviewed_by_user_id` is intentionally **unconstrained** (no FK); `reviewed_by_display` is denormalized so the UI need not load the reviewer User.
- Many users can be assigned to one role
- `ON DELETE RESTRICT` - cannot delete role if users are assigned
- Role links user to permission set for authorization
3. **Member → CustomFieldValues (1:N)** **User ↔ Member** is an optional 1:1 (both sides may be NULL; entities exist independently). **Member ↔ Group** is many-to-many through `member_groups` (CASCADE lives only on the join table).
- One member, many custom_field_values
- `ON DELETE CASCADE` - custom_field_values deleted with member
- Composite unique constraint (member_id, custom_field_id)
4. **CustomFieldValue → CustomField (N:1)**
- Custom field values reference type definition
- `ON DELETE RESTRICT` - cannot delete type if in use
- Type defines data structure
5. **Member → MembershipFeeType (N:1, optional)**
- Many members can be assigned to one fee type
- `ON DELETE RESTRICT` - cannot delete fee type if members are assigned
- Optional relationship (member can have no fee type)
6. **Member → MembershipFeeCycles (1:N)**
- One member, many billing cycles
- `ON DELETE CASCADE` - cycles deleted when member deleted
- Unique constraint (member_id, cycle_start)
7. **MembershipFeeCycle → MembershipFeeType (N:1)**
- Many cycles reference one fee type
- `ON DELETE RESTRICT` - cannot delete fee type if cycles exist
8. **Settings → MembershipFeeType (N:1, optional)**
- Settings can reference a default fee type
- `ON DELETE SET NULL` - if fee type is deleted, setting is cleared
9. **Member ↔ Group (N:N via MemberGroup)**
- Many-to-many relationship through `member_groups` join table
- `ON DELETE CASCADE` on both sides - removing member/group removes associations
- Unique constraint on (member_id, group_id) prevents duplicates
- Groups searchable via member search vector
## Important Business Rules ## Important Business Rules
### Email Synchronization ### Email Synchronization
- **User.email** is the source of truth when linked - **User.email is the source of truth when linked.** On linking, `Member.email ← User.email` (overwrite). Afterwards changes sync bidirectionally. Validation prevents email conflicts with other unlinked users.
- On linking: Member.email ← User.email (overwrite)
- After linking: Changes sync bidirectionally
- Validation prevents email conflicts
### Authentication Strategies ### Authentication Strategies
- **Password:** Email + hashed_password - **Password:** email + hashed_password. **OIDC:** email + oidc_id (Rauthy provider), the external identity recorded via the `oidc_id` column on `users`. At least one method required per user.
- **OIDC:** Email + oidc_id (Rauthy provider)
- At least one method required per user
### Member Constraints ### Member Constraints
- First name and last name required (min 1 char) - `first_name` / `last_name`: optional, but if present min 1 char.
- Email unique, validated format (5-254 chars) - `email`: unique, validated format (5–254 chars).
- Exit date must be after join date - `exit_date` must be after `join_date`.
- Phone: `+?[0-9\- ]{6,20}` - `postal_code`, `country`: optional, no format validation.
- Postal code: optional (no format validation)
- Country: optional
### CustomFieldValue System ### CustomFieldValue System
- Maximum one custom field value per custom field per member - One value per custom field per member. Value stored as a union type in JSONB: `{type: "string|integer|boolean|date|email", value: <actual_value>}`. Custom fields can be marked `required` and toggled `show_in_overview`.
- Value stored as union type in JSONB
- Supported types: string, integer, boolean, date, email
- Types can be marked as immutable or required
## Indexes
### Performance Indexes
**members:**
- `search_vector` (GIN) - Full-text search (tsvector)
- `first_name` (GIN trgm) - Fuzzy search on first name
- `last_name` (GIN trgm) - Fuzzy search on last name
- `email` (GIN trgm) - Fuzzy search on email
- `city` (GIN trgm) - Fuzzy search on city
- `street` (GIN trgm) - Fuzzy search on street
- `notes` (GIN trgm) - Fuzzy search on notes
- `email` (B-tree) - Exact email lookups
- `last_name` (B-tree) - Name sorting
- `join_date` (B-tree) - Date filtering
**custom_field_values:**
- `member_id` - Member custom field value lookups
- `custom_field_id` - Type-based queries
- Composite `(member_id, custom_field_id)` - Uniqueness
**tokens:**
- `subject` - User token lookups
- `expires_at` - Token cleanup
- `purpose` - Purpose-based queries
**users:**
- `email` (unique) - Login lookups
- `oidc_id` (unique) - OIDC authentication
- `member_id` (unique) - Member linkage
## Full-Text Search ## Full-Text Search
### Implementation ### Implementation
- **Trigger** on `members` (INSERT/UPDATE): runs function `members_search_vector_trigger()` - **Trigger** on `members` (INSERT/UPDATE): `update_search_vector` runs function `members_search_vector_trigger()`
- **Trigger** on `custom_field_values` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_custom_field_value_change` runs function `update_member_search_vector_from_custom_field_value()`
- **Trigger** on `member_groups` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_member_groups_change` runs function `update_member_search_vector_from_member_groups()` - **Trigger** on `member_groups` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_member_groups_change` runs function `update_member_search_vector_from_member_groups()`
- **Index Type:** GIN (Generalized Inverted Index) - **Index Type:** GIN (Generalized Inverted Index)
### Weighted Fields ### Weighted Fields
- **Weight A (highest):** first_name, last_name - **Weight A (highest):** first_name, last_name
- **Weight B:** email, notes, group names (from member_groups → groups) - **Weight B:** email, notes, group names (from member_groups → groups)
- **Weight C:** city, street, house_number, postal_code, country, custom_field_values - **Weight C:** city, street, house_number, postal_code, custom_field_values
- **Weight D (lowest):** join_date, exit_date - **Weight D (lowest):** join_date, exit_date
### Group Names in Search ### Group Names in Search
@ -264,285 +144,40 @@ WHERE search_vector @@ to_tsquery('simple', 'john & doe');
## Fuzzy Search (Trigram-based) ## Fuzzy Search (Trigram-based)
### Implementation - **Extension:** `pg_trgm`; GIN indexes with `gin_trgm_ops` on `first_name`, `last_name`, `email`, `city`, `street`, `notes`.
- **Extension:** `pg_trgm` (PostgreSQL Trigram) - **Similarity threshold:** 0.2 (default, configurable) — balances precision/recall.
- **Index Type:** GIN with `gin_trgm_ops` operator class - **Added:** November 2025 (PR #187, closes #162).
- **Similarity Threshold:** 0.2 (default, configurable)
- **Added:** November 2025 (PR #187, closes #162)
### How It Works Fuzzy search combines several strategies (applied as an OR-chain alongside full-text and substring matching):
Fuzzy search combines multiple search strategies:
1. **Full-text search** - Primary filter using tsvector
2. **Trigram similarity** - `similarity(field, query) > threshold`
3. **Word similarity** - `word_similarity(query, field) > threshold`
4. **Substring matching** - `LIKE` and `ILIKE` for exact substrings
5. **Modulo operator** - `query % field` for quick similarity check
### Indexed Fields for Fuzzy Search 1. Full-text search — primary filter via tsvector.
- `first_name` - GIN trigram index 2. Trigram similarity — `similarity(field, query) > threshold`.
- `last_name` - GIN trigram index 3. Word similarity — `word_similarity(query, field) > threshold`.
- `email` - GIN trigram index 4. Substring matching — `LIKE` / `ILIKE`.
- `city` - GIN trigram index 5. `%` operator — quick trigram-similarity check.
- `street` - GIN trigram index
- `notes` - GIN trigram index
### Usage Example (Ash Action) For the Elixir search action and per-strategy filter functions, see `lib/membership/member.ex` and [`custom-fields-search-performance.md`](./custom-fields-search-performance.md).
```elixir
# In LiveView or context
Member.fuzzy_search(Member, query: "john", similarity_threshold: 0.2)
# Or using Ash Query directly
Member
|> Ash.Query.for_read(:search, %{query: "john", similarity_threshold: 0.2})
|> Mv.Membership.read!()
```
### Usage Example (SQL)
```sql
-- Trigram similarity search
SELECT * FROM members
WHERE similarity(first_name, 'john') > 0.2
OR similarity(last_name, 'doe') > 0.2
ORDER BY similarity(first_name, 'john') DESC;
-- Word similarity (better for partial matches)
SELECT * FROM members
WHERE word_similarity('john', first_name) > 0.2;
-- Quick similarity check with % operator
SELECT * FROM members
WHERE 'john' % first_name;
```
### Performance Considerations
- **GIN indexes** speed up trigram operations significantly
- **Similarity threshold** of 0.2 balances precision and recall
- **Combined approach** (FTS + trigram) provides best results
- Lower threshold = more results but less specific
## Database Extensions ## Database Extensions
### Required PostgreSQL Extensions Installed extensions are defined in `Mv.Repo.installed_extensions/0`:
1. **uuid-ossp** | Extension | Purpose | Notes |
- Purpose: UUID generation functions |-----------|---------|-------|
- Used for: `gen_random_uuid()`, `uuid_generate_v7()` | `ash-functions` | Ash helper SQL functions | installed by Ash |
| `citext` | Case-insensitive text | `users.email` |
| `pg_trgm` | Trigram fuzzy search | added in `20251001141005_add_trigram_to_members.exs`; operators `%`, `similarity()`, `word_similarity()` |
2. **citext** `gen_random_uuid()` is built into PostgreSQL; `uuid_generate_v7()` is a custom SQL function defined in a migration (not provided by an extension).
- Purpose: Case-insensitive text type
- Used for: `users.email` (case-insensitive email matching)
3. **pg_trgm** ## Sensitive Data (GDPR / logging)
- Purpose: Trigram-based fuzzy text search and similarity matching
- Used for: Fuzzy member search with similarity scoring
- Operators: `%` (similarity), `word_similarity()`, `similarity()`
- Added in: Migration `20251001141005_add_trigram_to_members.exs`
### Installation - **Never log:** `users.hashed_password` (bcrypt), token fields (`jti`, `purpose`, `extra_data`), OIDC/SMTP/vereinfacht secrets in `settings`.
```sql - **Personal data:** all member fields, user email, join-request applicant data.
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE EXTENSION IF NOT EXISTS "citext";
CREATE EXTENSION IF NOT EXISTS "pg_trgm";
```
## Migration Strategy
### Ash Migrations
This project uses Ash Framework's migration system:
```bash
# Generate new migration
mix ash.codegen --name add_new_feature
# Apply migrations
mix ash.setup
# Rollback migrations
mix ash_postgres.rollback -n 1
```
### Migration Files Location
```
priv/repo/migrations/
├── 20250421101957_initialize_extensions_1.exs
├── 20250528163901_initial_migration.exs
├── 20250617090641_member_fields.exs
├── 20250620110850_add_accounts_domain.exs
├── 20250912085235_AddSearchVectorToMembers.exs
├── 20250926180341_add_unique_email_to_members.exs
├── 20251001141005_add_trigram_to_members.exs
└── 20251016130855_add_constraints_for_user_member_and_property.exs
```
## Data Integrity
### Foreign Key Behaviors
| Relationship | On Delete | Rationale |
|--------------|-----------|-----------|
| `users.member_id → members.id` | SET NULL | Preserve user account when member deleted |
| `custom_field_values.member_id → members.id` | CASCADE | Delete custom_field_values with member |
| `custom_field_values.custom_field_id → custom_fields.id` | RESTRICT | Prevent deletion of types in use |
### Validation Layers
1. **Database Level:**
- CHECK constraints
- NOT NULL constraints
- UNIQUE indexes
- Foreign key constraints
2. **Application Level (Ash):**
- Custom validators
- Email format validation (EctoCommons.EmailValidator)
- Business rule validation
- Cross-entity validation
3. **UI Level:**
- Client-side form validation
- Real-time feedback
- Error messages
## Performance Considerations
### Query Patterns
**High Frequency:**
- Member search (uses GIN index on search_vector)
- Member list with filters (uses indexes on join_date, membership_fee_type_id)
- User authentication (uses unique index on email/oidc_id)
- CustomFieldValue lookups by member (uses index on member_id)
**Medium Frequency:**
- Member CRUD operations
- CustomFieldValue updates
- Token validation
**Low Frequency:**
- CustomField management
- User-Member linking
- Bulk operations
### Optimization Tips
1. **Use indexes:** All critical query paths have indexes
2. **Preload relationships:** Use Ash's `load` to avoid N+1
3. **Pagination:** Use keyset pagination (configured by default)
4. **GIN indexes:** Full-text search and fuzzy search on multiple fields
5. **Search optimization:** Full-text search via tsvector, not LIKE
## Visualization
### Using dbdiagram.io
1. Visit [https://dbdiagram.io](https://dbdiagram.io)
2. Click "Import" → "From file"
3. Upload `database_schema.dbml`
4. View interactive diagram with relationships
### Using dbdocs.io
1. Install dbdocs CLI: `npm install -g dbdocs`
2. Generate docs: `dbdocs build database_schema.dbml`
3. View generated documentation
### VS Code Extension
Install "DBML Language" extension to view/edit DBML files with:
- Syntax highlighting
- Inline documentation
- Error checking
## Security Considerations
### Sensitive Data
**Encrypted:**
- `users.hashed_password` (bcrypt)
**Should Not Log:**
- hashed_password
- tokens (jti, purpose, extra_data)
**Personal Data (GDPR):**
- All member fields (name, email, address)
- User email
- Token subject
### Access Control
- Implement through Ash policies
- Row-level security considerations for future
- Audit logging for sensitive operations
## Backup Recommendations
### Critical Tables (Priority 1)
- `members` - Core business data
- `users` - Authentication data
- `custom_fields` - Schema definitions
### Important Tables (Priority 2)
- `custom_field_values` - Member custom data
- `tokens` - Can be regenerated but good to backup
### Backup Strategy
```bash
# Full database backup
pg_dump -Fc mv_prod > backup_$(date +%Y%m%d).dump
# Restore
pg_restore -d mv_prod backup_20251110.dump
```
## Testing
### Test Database
- Separate test database: `mv_test`
- Sandbox mode via Ecto.Adapters.SQL.Sandbox
- Reset between tests
### Seed Data
```bash
# Load seed data
mix run priv/repo/seeds.exs
```
## Future Considerations
### Potential Additions
1. **Audit Log Table**
- Track changes to members
- Compliance and history tracking
2. **Payment Tracking**
- Payment history table
- Transaction records
- Fee calculation
3. **Document Storage**
- Member documents/attachments
- File metadata table
4. **Email Queue**
- Outbound email tracking
- Delivery status
5. **Roles & Permissions**
- User roles (admin, treasurer, member)
- Permission management
## Resources
- **Ash Framework:** [https://hexdocs.pm/ash](https://hexdocs.pm/ash)
- **AshPostgres:** [https://hexdocs.pm/ash_postgres](https://hexdocs.pm/ash_postgres)
- **DBML Specification:** [https://dbml.dbdiagram.io](https://dbml.dbdiagram.io)
- **PostgreSQL Docs:** [https://www.postgresql.org/docs/](https://www.postgresql.org/docs/)
--- ---
**Last Updated:** 2026-01-27 **Last Updated:** 2026-06-15
**Schema Version:** 1.5 **Schema Version:** 1.6 (12 tables)
**Database:** PostgreSQL 17.6 (dev) / 16 (prod) **Database:** PostgreSQL 17.6 (dev) / 16 (prod)

202
docs/database_schema.dbml
View file

@ -6,8 +6,9 @@
// - https://dbdocs.io // - https://dbdocs.io
// - VS Code Extensions: "DBML Language" or "dbdiagram.io" // - VS Code Extensions: "DBML Language" or "dbdiagram.io"
// //
// Version: 1.4 // Version: 1.6
// Last Updated: 2026-01-13 // Last Updated: 2026-06-15
// Hand-maintained (NOT auto-generated). 12 tables.
Project mila_membership_management { Project mila_membership_management {
database_type: 'PostgreSQL' database_type: 'PostgreSQL'
@ -25,15 +26,16 @@ Project mila_membership_management {
- GDPR-compliant data management - GDPR-compliant data management
## Domains: ## Domains:
- **Accounts**: User authentication and session management - **Accounts**: User authentication, sessions, OIDC strategy identities
- **Membership**: Club member data and custom fields - **Membership**: Club member data, custom fields, groups, settings, public join requests
- **MembershipFees**: Membership fee types and billing cycles - **MembershipFees**: Membership fee types and billing cycles
- **Authorization**: Role-based access control (RBAC) - **Authorization**: Role-based access control (RBAC)
## Required PostgreSQL Extensions: ## Required PostgreSQL Extensions (see Mv.Repo.installed_extensions/0):
- uuid-ossp (UUID generation) - ash-functions (Ash helper SQL functions)
- citext (case-insensitive text) - citext (case-insensitive text)
- pg_trgm (trigram-based fuzzy search) - pg_trgm (trigram-based fuzzy search)
UUIDv7 ids use uuid_generate_v7(), a custom SQL function defined in a migration (not an extension).
''' '''
} }
@ -135,6 +137,7 @@ Table members {
search_vector tsvector [null, note: 'Full-text search index (auto-generated)'] search_vector tsvector [null, note: 'Full-text search index (auto-generated)']
membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - assigned fee type'] membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - assigned fee type']
membership_fee_start_date date [null, note: 'Date from which membership fees should be calculated'] membership_fee_start_date date [null, note: 'Date from which membership fees should be calculated']
vereinfacht_contact_id text [null, note: 'External contact id from the vereinfacht.de API (no FK; null if unlinked)']
indexes { indexes {
email [unique, name: 'members_unique_email_index'] email [unique, name: 'members_unique_email_index']
@ -169,7 +172,8 @@ Table members {
**Search Capabilities:** **Search Capabilities:**
1. Full-Text Search (tsvector): 1. Full-Text Search (tsvector):
- `search_vector` is auto-updated via trigger - `search_vector` is auto-updated via trigger
- Weighted fields: first_name (A), last_name (A), email (B), notes (B) - Weighted fields (A/B/C/D map): see the "Weighted Fields" section of
database-schema-readme.md (single source of truth, matches the search trigger)
- GIN index for fast text search - GIN index for fast text search
2. Fuzzy Search (pg_trgm): 2. Fuzzy Search (pg_trgm):
@ -225,7 +229,7 @@ Table custom_field_values {
**Constraints:** **Constraints:**
- Each member can have only ONE custom field value per custom field - Each member can have only ONE custom field value per custom field
- Custom field values are deleted when member is deleted (CASCADE) - Custom field values are deleted when member is deleted (CASCADE)
- Custom field cannot be deleted if custom field values exist (RESTRICT) - Custom field values are deleted when the custom field is deleted (CASCADE)
**Use Cases:** **Use Cases:**
- Custom membership numbers - Custom membership numbers
@ -241,8 +245,9 @@ Table custom_fields {
slug text [not null, unique, note: 'URL-friendly, immutable identifier (e.g., "membership-number"). Auto-generated from name.'] slug text [not null, unique, note: 'URL-friendly, immutable identifier (e.g., "membership-number"). Auto-generated from name.']
value_type text [not null, note: 'Data type: string | integer | boolean | date | email'] value_type text [not null, note: 'Data type: string | integer | boolean | date | email']
description text [null, note: 'Human-readable description'] description text [null, note: 'Human-readable description']
immutable boolean [not null, default: false, note: 'If true, value cannot be changed after creation'] join_description text [null, note: 'Optional label shown for this field on the public join form (e.g., a GDPR confirmation text); supports inline external links. Falls back to name when null.']
required boolean [not null, default: false, note: 'If true, all members must have this custom field'] required boolean [not null, default: false, note: 'If true, all members must have this custom field']
show_in_overview boolean [not null, default: true, note: 'If true, this custom field is displayed in the member overview table and can be sorted']
indexes { indexes {
name [unique, name: 'custom_fields_unique_name_index'] name [unique, name: 'custom_fields_unique_name_index']
@ -259,8 +264,9 @@ Table custom_fields {
- `slug`: URL-friendly, human-readable identifier (auto-generated, immutable) - `slug`: URL-friendly, human-readable identifier (auto-generated, immutable)
- `value_type`: Enforces data type consistency - `value_type`: Enforces data type consistency
- `description`: Documentation for users/admins - `description`: Documentation for users/admins
- `immutable`: Prevents changes after initial creation (e.g., membership numbers) - `join_description`: Optional label shown for this field on the public join form (falls back to `name` when null)
- `required`: Enforces that all members must have this custom field - `required`: Enforces that all members must have this custom field
- `show_in_overview`: When true, the field is shown in the member overview table and can be sorted
**Slug Generation:** **Slug Generation:**
- Automatically generated from `name` on creation - Automatically generated from `name` on creation
@ -275,13 +281,13 @@ Table custom_fields {
- `name` must be unique across all custom fields - `name` must be unique across all custom fields
- `slug` must be unique across all custom fields - `slug` must be unique across all custom fields
- `slug` cannot be empty (validated on creation) - `slug` cannot be empty (validated on creation)
- Cannot be deleted if custom_field_values reference it (ON DELETE RESTRICT) - Deleting a custom field cascades: its custom_field_values are deleted too (ON DELETE CASCADE)
**Examples:** **Examples:**
- Membership Number (string, immutable, required) → slug: "membership-number" - Membership Number (string, required) → slug: "membership-number"
- Emergency Contact (string, mutable, optional) → slug: "emergency-contact" - Emergency Contact (string, optional) → slug: "emergency-contact"
- Certified Trainer (boolean, mutable, optional) → slug: "certified-trainer" - Certified Trainer (boolean, optional) → slug: "certified-trainer"
- Certification Date (date, immutable, optional) → slug: "certification-date" - Certification Date (date, optional) → slug: "certification-date"
''' '''
} }
@ -399,8 +405,8 @@ Ref: custom_field_values.member_id > members.id [delete: cascade]
// CustomFieldValue → CustomField (N:1) // CustomFieldValue → CustomField (N:1)
// - Many custom_field_values can reference one custom field // - Many custom_field_values can reference one custom field
// - CustomFieldValue type defines the schema/behavior // - CustomFieldValue type defines the schema/behavior
// - ON DELETE RESTRICT: Cannot delete type if custom_field_values exist // - ON DELETE CASCADE: deleting the custom field deletes its custom_field_values
Ref: custom_field_values.custom_field_id > custom_fields.id [delete: restrict] Ref: custom_field_values.custom_field_id > custom_fields.id [delete: cascade]
// Member → MembershipFeeType (N:1) // Member → MembershipFeeType (N:1)
// - Many members can be assigned to one fee type // - Many members can be assigned to one fee type
@ -467,20 +473,9 @@ TableGroup accounts_domain {
**Accounts Domain** **Accounts Domain**
Handles user authentication and session management using AshAuthentication. Handles user authentication and session management using AshAuthentication.
Supports multiple authentication strategies (Password, OIDC). Supports multiple authentication strategies (Password, OIDC). OIDC linking
''' is recorded on the users table via the oidc_id column (there is no separate
} user_identities table).
TableGroup membership_domain {
members
custom_field_values
custom_fields
Note: '''
**Membership Domain**
Core business logic for club membership management.
Supports flexible, extensible member data model.
''' '''
} }
@ -550,9 +545,32 @@ Table roles {
Table settings { Table settings {
id uuid [pk, not null, default: `gen_random_uuid()`, note: 'Primary identifier'] id uuid [pk, not null, default: `gen_random_uuid()`, note: 'Primary identifier']
club_name text [not null, note: 'The name of the association/club (min length: 1)'] club_name text [not null, note: 'The name of the association/club (min length: 1)']
member_field_visibility jsonb [null, note: 'Visibility configuration for member fields in overview (JSONB map)'] member_field_visibility jsonb [null, note: 'Visibility config for member fields in overview (JSONB map; absent key = visible)']
member_field_required jsonb [null, note: 'Required-field config for member fields (JSONB map)']
include_joining_cycle boolean [not null, default: true, note: 'Whether to include the joining cycle in membership fee generation'] include_joining_cycle boolean [not null, default: true, note: 'Whether to include the joining cycle in membership fee generation']
default_membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - default fee type for new members'] default_membership_fee_type_id uuid [null, note: 'Logical reference to membership_fee_types (default fee type for new members) - app-enforced, NO DB foreign key']
registration_enabled boolean [not null, default: true, note: 'Whether self-service user registration is enabled']
oidc_only boolean [not null, default: false, note: 'If true, only OIDC login is offered (password login hidden)']
oidc_client_id text [null, note: 'OIDC client id']
oidc_client_secret text [null, note: 'OIDC client secret']
oidc_base_url text [null, note: 'OIDC provider base URL (e.g., Rauthy)']
oidc_redirect_uri text [null, note: 'OIDC redirect URI']
oidc_admin_group_name text [null, note: 'Provider group name mapped to admin role on login']
oidc_groups_claim text [null, note: 'JWT claim carrying the user groups for role sync']
smtp_host text [null, note: 'Outbound SMTP host']
smtp_port bigint [null, note: 'Outbound SMTP port']
smtp_username text [null, note: 'SMTP auth username']
smtp_password text [null, note: 'SMTP auth password (secret)']
smtp_ssl text [null, note: 'SMTP TLS/SSL mode']
smtp_from_name text [null, note: 'Display name for the From header (mail_from)']
smtp_from_email text [null, note: 'Email address for the From header (mail_from)']
vereinfacht_api_url text [null, note: 'vereinfacht.de API base URL']
vereinfacht_api_key text [null, note: 'vereinfacht.de API key (secret)']
vereinfacht_club_id text [null, note: 'vereinfacht.de club identifier']
vereinfacht_app_url text [null, note: 'vereinfacht.de app URL (for links)']
join_form_enabled boolean [not null, default: false, note: 'Whether the public join form is enabled']
join_form_field_ids text[] [null, note: 'Ordered custom_field ids shown on the public join form']
join_form_field_required jsonb [null, note: 'Per-field required config for the join form (JSONB map)']
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)'] inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)'] updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
@ -590,19 +608,123 @@ Table settings {
''' '''
} }
// ============================================
// MEMBERSHIP DOMAIN — Groups
// ============================================
Table groups {
id uuid [pk, not null, default: `uuid_generate_v7()`, note: 'UUIDv7 primary key']
name text [not null, note: 'Group name (unique case-insensitively via LOWER(name))']
slug text [not null, unique, note: 'URL-friendly, immutable identifier auto-generated from name (shared GenerateSlug change)']
description text [null, note: 'Optional description']
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
indexes {
slug [unique, name: 'groups_unique_slug_index', note: 'Case-sensitive unique slug']
name [unique, name: 'groups_unique_name_lower_index', note: 'UNIQUE on LOWER(name) - case-insensitive name uniqueness']
}
Note: '''
**Member Groups**
Flat groupings of members (no hierarchy in current schema). Many-to-many
with members via member_groups. `slug` is generated by the shared
Mv.Membership.Changes.GenerateSlug change (same as custom_fields) and is
used for URL routing (/groups/:slug). Group names feed the member
search_vector at weight B (see member_groups note).
**Future extension path (not yet in schema):**
- parent_group_id (self-referential, nullable) + circular-ref guard + path calc for hierarchy
- member_group_roles table linking MemberGroup to a Role (position within a group)
'''
}
Table member_groups {
id uuid [pk, not null, default: `uuid_generate_v7()`, note: 'UUIDv7 primary key']
member_id uuid [not null, note: 'FK to members']
group_id uuid [not null, note: 'FK to groups']
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
indexes {
(member_id, group_id) [unique, name: 'member_groups_unique_member_group_index', note: 'One association per member per group']
member_id [name: 'member_groups_member_id_index']
group_id [name: 'member_groups_group_id_index']
}
Note: '''
**Member ↔ Group Join Table**
CASCADE delete on BOTH foreign keys (the cascade lives only on the join
table; members and groups themselves are never deleted by association
removal). INSERT/UPDATE/DELETE here fires the trigger that refreshes the
affected member's search_vector so group names (weight B) stay current.
'''
}
// ============================================
// MEMBERSHIP DOMAIN — Public Join Requests
// ============================================
Table join_requests {
id uuid [pk, not null, default: `uuid_generate_v7()`, note: 'UUIDv7 primary key']
status text [not null, default: 'pending_confirmation', note: 'pending_confirmation → submitted (after email confirm) → approved/rejected']
email text [not null, note: 'Applicant email']
first_name text [null, note: 'Applicant first name']
last_name text [null, note: 'Applicant last name']
form_data jsonb [null, note: 'Submitted join-form field values (custom fields)']
schema_version integer [null, note: 'Version of the join-form schema used at submission time']
confirmation_token_hash text [null, note: 'Hash of the double-opt-in token (raw token never stored)']
confirmation_token_expires_at timestamp [null, note: 'Token expiry (UTC)']
confirmation_sent_at timestamp [null, note: 'When the confirmation email was sent (UTC)']
submitted_at timestamp [null, note: 'When email was confirmed and request submitted (UTC)']
approved_at timestamp [null, note: 'When an admin approved (UTC)']
rejected_at timestamp [null, note: 'When an admin rejected (UTC)']
reviewed_by_user_id uuid [null, note: 'User who reviewed (no FK constraint)']
reviewed_by_display text [null, note: 'Reviewer display string, denormalized so the UI need not load the User']
source text [null, note: 'Origin of the request (e.g., public form)']
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
indexes {
confirmation_token_hash [unique, name: 'join_requests_confirmation_token_hash_unique', note: 'Partial unique WHERE confirmation_token_hash IS NOT NULL']
email [name: 'join_requests_email_index']
status [name: 'join_requests_status_index']
}
Note: '''
**Public Join Flow (Onboarding, Double Opt-In)**
Stores public join-form submissions. Double opt-in: the confirmation token
is stored as a hash only; unconfirmed records have a ~24h retention and are
removed by a scheduled cleanup job. `reviewed_by_user_id` is intentionally
unconstrained (no FK); `reviewed_by_display` is denormalized so showing the
reviewer does not require loading the User.
'''
}
// ============================================ // ============================================
// RELATIONSHIPS (Additional) // RELATIONSHIPS (Additional)
// ============================================ // ============================================
// MemberGroup → Member (N:1)
// - ON DELETE CASCADE (join table only): association removed, member preserved
Ref: member_groups.member_id > members.id [delete: cascade]
// MemberGroup → Group (N:1)
// - ON DELETE CASCADE (join table only): association removed, group preserved
Ref: member_groups.group_id > groups.id [delete: cascade]
// User → Role (N:1) // User → Role (N:1)
// - Many users can be assigned to one role // - Many users can be assigned to one role
// - ON DELETE RESTRICT: Cannot delete role if users are assigned // - ON DELETE RESTRICT: Cannot delete role if users are assigned
Ref: users.role_id > roles.id [delete: restrict] Ref: users.role_id > roles.id [delete: restrict]
// Settings → MembershipFeeType (N:1, optional) // Settings → MembershipFeeType (N:1, optional) — LOGICAL relationship only
// - Settings can reference a default membership fee type // - No DB foreign key (cross-domain dependency is deliberately avoided);
// - ON DELETE SET NULL: If fee type is deleted, setting is cleared // referential integrity is enforced in the app (Mv.Membership.Setting)
Ref: settings.default_membership_fee_type_id > membership_fee_types.id [delete: set null] Ref: settings.default_membership_fee_type_id > membership_fee_types.id
// ============================================ // ============================================
// TABLE GROUPS (Updated) // TABLE GROUPS (Updated)
@ -624,12 +746,16 @@ TableGroup membership_domain {
custom_field_values custom_field_values
custom_fields custom_fields
settings settings
groups
member_groups
join_requests
Note: ''' Note: '''
**Membership Domain** **Membership Domain**
Core business logic for club membership management. Core business logic for club membership management.
Supports flexible, extensible member data model. Supports flexible, extensible member data model.
Includes global application settings (singleton). Includes member groups (many-to-many), global application settings
(singleton), and the public join-request flow.
''' '''
} }

1254
docs/groups-architecture.md
View file

File diff suppressed because it is too large Load diff