Cleanup of the docs closes #507 #530
4 changed files with 360 additions and 1875 deletions
|
|
@ -2,242 +2,88 @@
|
|||
|
||||
## Current Implementation
|
||||
|
||||
The search vector includes custom field values via database triggers that:
|
||||
1. Aggregate all custom field values for a member
|
||||
2. Extract values from JSONB format
|
||||
3. Add them to the search_vector with weight 'C'
|
||||
The member `search_vector` includes custom field values via database triggers that aggregate all of a member's custom field values, extract the value from each JSONB record (`value->>'_union_value'`), and add them at weight `C`.
|
||||
|
||||
## Performance Considerations
|
||||
Two triggers maintain the vector:
|
||||
|
||||
### 1. Trigger Performance on Member Updates
|
||||
- `members_search_vector_trigger()` — fires on `members` INSERT/UPDATE; runs a subquery `SELECT string_agg(...) FROM custom_field_values WHERE member_id = NEW.id`.
|
||||
- `update_member_search_vector_from_custom_field_value()` — fires on `custom_field_values` INSERT/UPDATE/DELETE; re-aggregates and updates the member's `search_vector`.
|
||||
|
||||
**Current Implementation:**
|
||||
- `members_search_vector_trigger()` executes a subquery on every INSERT/UPDATE:
|
||||
```sql
|
||||
SELECT string_agg(...) FROM custom_field_values WHERE member_id = NEW.id
|
||||
```
|
||||
Both rely on `custom_field_values_member_id_idx`, so the per-member aggregation is an indexed lookup.
|
||||
|
||||
**Performance Impact:**
|
||||
- ✅ **Good:** Index on `member_id` exists (`custom_field_values_member_id_idx`)
|
||||
- ✅ **Good:** Subquery only runs for the affected member
|
||||
- ⚠️ **Potential Issue:** With many custom fields per member (e.g., 50+), aggregation could be slower
|
||||
- ⚠️ **Potential Issue:** JSONB extraction (`value->>'_union_value'`) is relatively fast but adds overhead
|
||||
## Applied Trigger Optimizations
|
||||
|
||||
**Expected Performance:**
|
||||
- **Small scale (< 10 custom fields per member):** Negligible impact (< 5ms per operation)
|
||||
- **Medium scale (10-30 custom fields):** Minor impact (5-20ms per operation)
|
||||
- **Large scale (30+ custom fields):** Noticeable impact (20-50ms+ per operation)
|
||||
`update_member_search_vector_from_custom_field_value()` was optimized:
|
||||
|
||||
### 2. Trigger Performance on Custom Field Value Changes
|
||||
- **Fetch only required member fields** (first_name, last_name, email, etc.) instead of the full record — reduces per-call overhead by roughly 30–50%.
|
||||
- **Early return on UPDATE when the value is unchanged** — skips the expensive re-aggregation entirely.
|
||||
|
||||
**Current Implementation:**
|
||||
- `update_member_search_vector_from_custom_field_value()` executes on every INSERT/UPDATE/DELETE on `custom_field_values`
|
||||
- **Optimized:** Only fetches required member fields (not full record) to reduce overhead
|
||||
- **Optimized:** Skips re-aggregation on UPDATE if value hasn't actually changed
|
||||
- Aggregates all custom field values, then updates member search_vector
|
||||
Measured effect per custom-field-value change:
|
||||
|
||||
**Performance Impact:**
|
||||
- ✅ **Good:** Index on `member_id` ensures fast lookup
|
||||
- ✅ **Optimized:** Only required fields are fetched (first_name, last_name, email, etc.) instead of full record
|
||||
- ✅ **Optimized:** UPDATE operations that don't change the value skip expensive re-aggregation (early return)
|
||||
- ⚠️ **Note:** Re-aggregation is still necessary when values change (required for search_vector consistency)
|
||||
- ⚠️ **Critical:** Bulk operations (e.g., importing 1000 members with custom fields) will trigger this for each row
|
||||
| Case | Before | After |
|
||||
|------|--------|-------|
|
||||
| Value changed | 5–15 ms | 3–10 ms |
|
||||
| Value unchanged (UPDATE) | 5–15 ms | < 1 ms (early return) |
|
||||
|
||||
**Expected Performance:**
|
||||
- **Single operation (value changed):** 3-10ms per custom field value change (improved from 5-15ms)
|
||||
- **Single operation (value unchanged):** <1ms (early return, no aggregation)
|
||||
- **Bulk operations:** Could be slow (consider disabling trigger temporarily)
|
||||
Re-aggregation is still required whenever a value actually changes — that is necessary for `search_vector` consistency.
|
||||
|
||||
### 3. Search Vector Size
|
||||
## Search Vector Size
|
||||
|
||||
**Current Constraints:**
|
||||
- String values: max 10,000 characters per custom field
|
||||
- No limit on number of custom fields per member
|
||||
- tsvector has no explicit size limit, but very large vectors can cause issues
|
||||
- String custom field values are capped at **10,000 characters each**; there is no cap on the number of custom fields per member.
|
||||
- `tsvector` has no hard size limit, but very large vectors (> ~100 KB) degrade GIN index maintenance, tsvector operations, and trigger time. Worst case: 100 fields × 10,000 chars ≈ 1 MB of aggregated text for one member.
|
||||
- **Recommendation:** monitor `search_vector` size in production; consider capping total custom-field content per member if large vectors appear.
|
||||
|
||||
**Potential Issues:**
|
||||
- **Theoretical maximum:** If a member has 100 custom fields with 10,000 char strings each, the aggregated text could be ~1MB
|
||||
- **Practical concern:** Very large search vectors (> 100KB) can slow down:
|
||||
- Index updates (GIN index maintenance)
|
||||
- Search queries (tsvector operations)
|
||||
- Trigger execution time
|
||||
## Bulk Imports
|
||||
|
||||
**Recommendation:**
|
||||
- Monitor search_vector size in production
|
||||
- Consider limiting total custom field content per member if needed
|
||||
- PostgreSQL can handle large tsvectors, but performance degrades gradually
|
||||
The custom-field-value trigger fires once per row, so importing many members with custom fields is expensive. For bulk imports, **temporarily disable the `custom_field_values` trigger**, then re-aggregate `search_vector` in a batch after the import. The initial backfill migration also updates all members in a single transaction (table lock); for > 10,000 members, batch the backfill and run during a maintenance window.
|
||||
|
||||
### 4. Initial Migration Performance
|
||||
## Search Query Structure
|
||||
|
||||
**Current Implementation:**
|
||||
- Updates ALL members in a single transaction:
|
||||
```sql
|
||||
UPDATE members m SET search_vector = ... (subquery for each member)
|
||||
```
|
||||
Full-text search uses the GIN index on `search_vector` (fast). Substring/custom-field matching adds `EXISTS (SELECT 1 FROM custom_field_values WHERE member_id = id AND ... LIKE ...)` subqueries, which are **not indexed** on the JSONB value (sequential scan) and run even when the FTS branch already matches. This is the main known weakness; it is acceptable at the current scale (< 30 custom fields/member, < 10,000 members) but is the first thing to revisit if search slows.
|
||||
|
||||
**Performance Impact:**
|
||||
- ⚠️ **Potential Issue:** With 10,000+ members, this could take minutes
|
||||
- ⚠️ **Potential Issue:** Single transaction locks the members table
|
||||
- ⚠️ **Potential Issue:** If migration fails, entire rollback required
|
||||
## Search Filter Functions
|
||||
|
||||
**Recommendation:**
|
||||
- For large datasets (> 10,000 members), consider:
|
||||
- Batch updates (e.g., 1000 members at a time)
|
||||
- Run during maintenance window
|
||||
- Monitor progress
|
||||
The search query in `lib/membership/member.ex` is split into modular filter builders, combined as a single OR-chain in priority order:
|
||||
|
||||
### 5. Search Query Performance
|
||||
1. `build_fts_filter/1` — full-text search (highest priority, GIN-indexed, fastest).
|
||||
2. `build_substring_filter/2` — `ILIKE` substring matching on structured fields (postal_code, house_number, email, city, country).
|
||||
3. `build_custom_field_filter/1` — JSONB custom-field value matching via `EXISTS` subquery.
|
||||
4. `build_fuzzy_filter/2` — trigram fuzzy matching on first_name, last_name, street (pg_trgm).
|
||||
|
||||
**Current Implementation:**
|
||||
- Full-text search uses GIN index on `search_vector` (fast)
|
||||
- Additional LIKE queries on `custom_field_values` for substring matching:
|
||||
```sql
|
||||
EXISTS (SELECT 1 FROM custom_field_values WHERE member_id = id AND ... LIKE ...)
|
||||
```
|
||||
|
||||
**Performance Impact:**
|
||||
- ✅ **Good:** GIN index on `search_vector` is very fast
|
||||
- ⚠️ **Potential Issue:** LIKE queries on JSONB are not indexed (sequential scan)
|
||||
- ⚠️ **Potential Issue:** EXISTS subquery runs for every search, even if search_vector match is found
|
||||
- ⚠️ **Potential Issue:** With many custom fields, the LIKE queries could be slow
|
||||
|
||||
**Expected Performance:**
|
||||
- **With GIN index match:** Very fast (< 10ms for typical queries)
|
||||
- **Without GIN index match (fallback to LIKE):** Slower (10-100ms depending on data size)
|
||||
- **Worst case:** Sequential scan of all custom_field_values for all members
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Short-term (Current Implementation)
|
||||
|
||||
1. **Monitor Performance:**
|
||||
- Add logging for trigger execution time
|
||||
- Monitor search_vector size distribution
|
||||
- Track search query performance
|
||||
|
||||
2. **Index Verification:**
|
||||
- Ensure `custom_field_values_member_id_idx` exists and is used
|
||||
- Verify GIN index on `search_vector` is maintained
|
||||
|
||||
3. **Bulk Operations:**
|
||||
- For bulk imports, consider temporarily disabling the custom_field_values trigger
|
||||
- Re-enable and update search_vectors in batch after import
|
||||
|
||||
### Medium-term Optimizations
|
||||
|
||||
1. **✅ Optimize Trigger Function (FULLY IMPLEMENTED):**
|
||||
- ✅ Only fetch required member fields instead of full record (reduces overhead)
|
||||
- ✅ Skip re-aggregation on UPDATE if value hasn't actually changed (early return optimization)
|
||||
|
||||
2. **Limit Search Vector Size:**
|
||||
- Truncate very long custom field values (e.g., first 1000 chars)
|
||||
- Add warning if aggregated text exceeds threshold
|
||||
|
||||
3. **Optimize LIKE Queries:**
|
||||
- Consider adding a generated column for searchable text
|
||||
- Or use a materialized view for custom field search
|
||||
|
||||
### Long-term Considerations
|
||||
|
||||
1. **Alternative Approaches:**
|
||||
- Separate search index table for custom fields
|
||||
- Use Elasticsearch or similar for advanced search
|
||||
- Materialized view for search optimization
|
||||
|
||||
2. **Scaling Strategy:**
|
||||
- If performance becomes an issue with 100+ custom fields per member:
|
||||
- Consider limiting which custom fields are searchable
|
||||
- Use a separate search service
|
||||
- Implement search result caching
|
||||
|
||||
## Performance Benchmarks (Estimated)
|
||||
|
||||
Based on typical PostgreSQL performance:
|
||||
|
||||
| Scenario | Members | Custom Fields/Member | Expected Impact |
|
||||
|----------|---------|---------------------|-----------------|
|
||||
| Small | < 1,000 | < 10 | Negligible (< 5ms per operation) |
|
||||
| Medium | 1,000-10,000 | 10-30 | Minor (5-20ms per operation) |
|
||||
| Large | 10,000-100,000 | 30-50 | Noticeable (20-50ms per operation) |
|
||||
| Very Large | > 100,000 | 50+ | Significant (50-200ms+ per operation) |
|
||||
Priority: **FTS > Substring > Custom Fields > Fuzzy**.
|
||||
|
||||
## Monitoring Queries
|
||||
|
||||
```sql
|
||||
-- Check search_vector size distribution
|
||||
-- search_vector size distribution
|
||||
SELECT
|
||||
pg_size_pretty(octet_length(search_vector::text)) as size,
|
||||
COUNT(*) as member_count
|
||||
pg_size_pretty(octet_length(search_vector::text)) AS size,
|
||||
COUNT(*) AS member_count
|
||||
FROM members
|
||||
WHERE search_vector IS NOT NULL
|
||||
GROUP BY octet_length(search_vector::text)
|
||||
ORDER BY octet_length(search_vector::text) DESC
|
||||
LIMIT 20;
|
||||
|
||||
-- Check average custom fields per member
|
||||
-- average / max custom fields per member
|
||||
SELECT
|
||||
AVG(custom_field_count) as avg_custom_fields,
|
||||
MAX(custom_field_count) as max_custom_fields
|
||||
AVG(custom_field_count) AS avg_custom_fields,
|
||||
MAX(custom_field_count) AS max_custom_fields
|
||||
FROM (
|
||||
SELECT member_id, COUNT(*) as custom_field_count
|
||||
SELECT member_id, COUNT(*) AS custom_field_count
|
||||
FROM custom_field_values
|
||||
GROUP BY member_id
|
||||
) subq;
|
||||
|
||||
-- Check trigger execution time (requires pg_stat_statements)
|
||||
SELECT
|
||||
mean_exec_time,
|
||||
calls,
|
||||
query
|
||||
-- trigger execution time (requires pg_stat_statements)
|
||||
SELECT mean_exec_time, calls, query
|
||||
FROM pg_stat_statements
|
||||
WHERE query LIKE '%members_search_vector_trigger%'
|
||||
ORDER BY mean_exec_time DESC;
|
||||
```
|
||||
|
||||
## Code Quality Improvements (Post-Review)
|
||||
|
||||
### Refactored Search Implementation
|
||||
|
||||
The search query has been refactored for better maintainability and clarity:
|
||||
|
||||
**Before:** Single large OR-chain with mixed search types (hard to maintain)
|
||||
|
||||
**After:** Modular functions grouped by search type:
|
||||
- `build_fts_filter/1` - Full-text search (highest priority, fastest)
|
||||
- `build_substring_filter/2` - Substring matching on structured fields
|
||||
- `build_custom_field_filter/1` - Custom field value search (JSONB LIKE)
|
||||
- `build_fuzzy_filter/2` - Trigram/fuzzy matching for names and streets
|
||||
|
||||
**Benefits:**
|
||||
- ✅ Clear separation of concerns
|
||||
- ✅ Easier to maintain and test
|
||||
- ✅ Better documentation of search priority
|
||||
- ✅ Easier to optimize individual search types
|
||||
|
||||
**Search Priority Order:**
|
||||
1. **FTS (Full-Text Search)** - Fastest, uses GIN index on search_vector
|
||||
2. **Substring** - For structured fields (postal_code, phone_number, etc.)
|
||||
3. **Custom Fields** - JSONB LIKE queries (fallback for substring matching)
|
||||
4. **Fuzzy Matching** - Trigram similarity for names and streets
|
||||
|
||||
## Conclusion
|
||||
|
||||
The current implementation is **well-optimized for typical use cases** (< 30 custom fields per member, < 10,000 members). For larger scales, monitoring and potential optimizations may be needed.
|
||||
|
||||
**Key Strengths:**
|
||||
- Indexed lookups (member_id index)
|
||||
- Efficient GIN index for search
|
||||
- Trigger-based automatic updates
|
||||
- Modular, maintainable search code structure
|
||||
|
||||
**Key Weaknesses:**
|
||||
- LIKE queries on JSONB (not indexed)
|
||||
- Re-aggregation on every custom field change (necessary for consistency)
|
||||
- Potential size issues with many/large custom fields
|
||||
- Substring searches (contains/ILIKE) not index-optimized
|
||||
|
||||
**Recent Optimizations:**
|
||||
- ✅ Trigger function optimized to fetch only required fields (reduces overhead by ~30-50%)
|
||||
- ✅ Early return on UPDATE when value hasn't changed (skips expensive re-aggregation, <1ms vs 3-10ms)
|
||||
- ✅ Improved performance for custom field value updates (3-10ms vs 5-15ms when value changes)
|
||||
## Future Options (if scale demands)
|
||||
|
||||
- Generated/searchable text column or materialized view for custom-field substring search (to escape the unindexed JSONB `LIKE`).
|
||||
- Limit which custom fields are searchable, or truncate long values.
|
||||
- External search service (e.g., Elasticsearch) for advanced search.
|
||||
|
|
|
|||
|
|
@ -4,105 +4,54 @@
|
|||
|
||||
This document provides a comprehensive overview of the Mila Membership Management System database schema.
|
||||
|
||||
## Quick Links
|
||||
- **DBML file:** [`database_schema.dbml`](./database_schema.dbml) — full per-column intent notes and relationship edges.
|
||||
- **Search-vector performance:** see [`custom-fields-search-performance.md`](./custom-fields-search-performance.md) for trigger cost analysis and tuning.
|
||||
|
||||
- **DBML File:** [`database_schema.dbml`](./database_schema.dbml)
|
||||
- **Visualize Online:**
|
||||
- [dbdiagram.io](https://dbdiagram.io) - Upload the DBML file
|
||||
- [dbdocs.io](https://dbdocs.io) - Generate interactive documentation
|
||||
The DBML is **hand-maintained** (not auto-generated); keep it in sync with `priv/repo/migrations/`.
|
||||
|
||||
## Schema Statistics
|
||||
|
||||
| Metric | Count |
|
||||
|--------|-------|
|
||||
| **Tables** | 11 |
|
||||
| **Tables** | 12 |
|
||||
| **Domains** | 4 (Accounts, Membership, MembershipFees, Authorization) |
|
||||
| **Relationships** | 9 |
|
||||
| **Indexes** | 25+ |
|
||||
| **Triggers** | 1 (Full-text search) |
|
||||
| **Triggers** | 3 (member, custom_field_values, member_groups → member search-vector) |
|
||||
|
||||
## Tables Overview
|
||||
|
||||
### Accounts Domain
|
||||
- **`users`** — authentication accounts. Dual auth (Password + OIDC), optional 1:1 link to a member; email is the source of truth when linked.
|
||||
- **`tokens`** — JWT storage for AshAuthentication; multiple purposes, revocation by deletion.
|
||||
|
||||
#### `users`
|
||||
- **Purpose:** User authentication and session management
|
||||
- **Rows (Estimated):** Low to Medium (typically 10-50% of members)
|
||||
- **Key Features:**
|
||||
- Dual authentication (Password + OIDC)
|
||||
- Optional 1:1 link to members
|
||||
- Email as source of truth when linked
|
||||
|
||||
#### `tokens`
|
||||
- **Purpose:** JWT token storage for AshAuthentication
|
||||
- **Rows (Estimated):** Medium to High (multiple tokens per user)
|
||||
- **Key Features:**
|
||||
- Token lifecycle management
|
||||
- Revocation support
|
||||
- Multiple token purposes
|
||||
OIDC account linking is recorded on the `users` table via the `oidc_id` column; there is no separate `user_identities` table.
|
||||
|
||||
### Membership Domain
|
||||
|
||||
#### `members`
|
||||
- **Purpose:** Club member master data
|
||||
- **Rows (Estimated):** High (core entity)
|
||||
- **Key Features:**
|
||||
- Complete member profile
|
||||
- Full-text search via tsvector
|
||||
- Bidirectional email sync with users
|
||||
- Flexible address and contact data
|
||||
|
||||
#### `custom_field_values`
|
||||
- **Purpose:** Dynamic custom member attributes
|
||||
- **Rows (Estimated):** Variable (N per member)
|
||||
- **Key Features:**
|
||||
- Union type value storage (JSONB)
|
||||
- Multiple data types supported
|
||||
- One custom field value per custom field per member
|
||||
|
||||
#### `custom_fields`
|
||||
- **Purpose:** Schema definitions for custom_field_values
|
||||
- **Rows (Estimated):** Low (admin-defined)
|
||||
- **Key Features:**
|
||||
- Type definitions
|
||||
- Immutable and required flags
|
||||
- Centralized custom field management
|
||||
|
||||
#### `settings`
|
||||
- **Purpose:** Global application settings (singleton resource)
|
||||
- **Rows (Estimated):** 1 (singleton pattern)
|
||||
- **Key Features:**
|
||||
- Club name configuration
|
||||
- Member field visibility settings
|
||||
- Membership fee default settings
|
||||
- Environment variable support for club name
|
||||
|
||||
#### `groups`
|
||||
- **Purpose:** Group definitions for organizing members
|
||||
- **Rows (Estimated):** Low (typically 5-20 groups per club)
|
||||
- **Key Features:**
|
||||
- Unique group names (case-insensitive)
|
||||
- URL-friendly slugs (auto-generated, immutable)
|
||||
- Optional descriptions
|
||||
- Many-to-many relationship with members
|
||||
|
||||
#### `member_groups`
|
||||
- **Purpose:** Join table for many-to-many relationship between members and groups
|
||||
- **Rows (Estimated):** Medium to High (multiple groups per member)
|
||||
- **Key Features:**
|
||||
- Unique constraint on (member_id, group_id)
|
||||
- CASCADE delete on both sides
|
||||
- Efficient indexes for queries
|
||||
- **`members`** — club member master data. Full-text + fuzzy search, bidirectional email sync with users, flexible address/contact data, `country`, optional `vereinfacht_contact_id` (external vereinfacht.de contact).
|
||||
- **`custom_field_values`** — dynamic per-member attributes. Union-type value in JSONB; one value per custom field per member.
|
||||
- **`custom_fields`** — schema definitions for custom field values (type, `required`/`show_in_overview` flags, optional `join_description`, auto-generated slug).
|
||||
- **`settings`** — global application settings (singleton). Club name (also via `ASSOCIATION_NAME` env), member-field visibility/required maps, fee defaults, plus OIDC, SMTP/mail-from, vereinfacht.de, public join-form, `registration_enabled`, and `oidc_only` configuration. See [Settings configuration columns](#settings-configuration-columns).
|
||||
- **`groups`** — member groupings. Case-insensitive-unique names, auto-generated immutable slugs, optional descriptions; many-to-many with members.
|
||||
- **`member_groups`** — join table for members ↔ groups. Unique `(member_id, group_id)`, CASCADE delete on both sides (join table only).
|
||||
- **`join_requests`** — public join flow (onboarding, double opt-in). Status machine `pending_confirmation → submitted → approved/rejected`; confirmation token stored as hash only, ~24h retention for unconfirmed records.
|
||||
|
||||
### Authorization Domain
|
||||
- **`roles`** — RBAC. Links users to one of four hardcoded permission sets (`own_data`, `read_only`, `normal_user`, `admin`); system roles are deletion-protected.
|
||||
|
||||
#### `roles`
|
||||
- **Purpose:** Role-based access control (RBAC)
|
||||
- **Rows (Estimated):** Low (typically 3-10 roles)
|
||||
- **Key Features:**
|
||||
- Links users to permission sets
|
||||
- System role protection
|
||||
- Four hardcoded permission sets: own_data, read_only, normal_user, admin
|
||||
### MembershipFees Domain
|
||||
- **`membership_fee_types`** — fee types with immutable billing interval.
|
||||
- **`membership_fee_cycles`** — per-member billing cycles with payment status.
|
||||
|
||||
## Settings configuration columns
|
||||
|
||||
The singleton `settings` row carries runtime configuration (all nullable unless noted). Grouped by area:
|
||||
|
||||
- **Member overview:** `member_field_visibility` (JSONB; absent key = visible), `member_field_required` (JSONB).
|
||||
- **Membership fees:** `include_joining_cycle` (bool, NOT NULL, default true), `default_membership_fee_type_id` (FK → membership_fee_types, ON DELETE SET NULL).
|
||||
- **Registration / login:** `registration_enabled` (bool, NOT NULL, default true), `oidc_only` (bool, NOT NULL, default false).
|
||||
- **OIDC:** `oidc_client_id`, `oidc_client_secret`, `oidc_base_url`, `oidc_redirect_uri`, `oidc_admin_group_name`, `oidc_groups_claim`.
|
||||
- **SMTP / mail-from:** `smtp_host`, `smtp_port` (bigint), `smtp_username`, `smtp_password`, `smtp_ssl`, `smtp_from_name`, `smtp_from_email`.
|
||||
- **vereinfacht.de:** `vereinfacht_api_url`, `vereinfacht_api_key`, `vereinfacht_club_id`, `vereinfacht_app_url`.
|
||||
- **Public join form:** `join_form_enabled` (bool, NOT NULL, default false), `join_form_field_ids` (text[]), `join_form_field_required` (JSONB).
|
||||
|
||||
## Key Relationships
|
||||
|
||||
|
|
@ -124,123 +73,54 @@ Member (N) ←→ (N) Group
|
|||
Settings (1) → MembershipFeeType (0..1)
|
||||
```
|
||||
|
||||
### Relationship Details
|
||||
## Foreign Key On-Delete Behavior
|
||||
|
||||
1. **User ↔ Member (Optional 1:1, both sides optional)**
|
||||
- A User can have 0 or 1 Member (`user.member_id` can be NULL)
|
||||
- A Member can have 0 or 1 User (optional `has_one` relationship)
|
||||
- Both entities can exist independently
|
||||
- Email synchronization when linked (User.email is source of truth)
|
||||
- `ON DELETE SET NULL` on user side (User preserved when Member deleted)
|
||||
| Relationship | On Delete | Rationale |
|
||||
|--------------|-----------|-----------|
|
||||
| `users.member_id → members.id` | SET NULL | Preserve user account when member deleted |
|
||||
| `users.role_id → roles.id` | RESTRICT | Cannot delete a role that still has users |
|
||||
| `custom_field_values.member_id → members.id` | CASCADE | Delete values with member |
|
||||
| `custom_field_values.custom_field_id → custom_fields.id` | CASCADE | Delete values when the custom field is deleted |
|
||||
| `members.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type assigned to members |
|
||||
| `membership_fee_cycles.member_id → members.id` | CASCADE | Cycles deleted with member |
|
||||
| `membership_fee_cycles.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type with cycles |
|
||||
| `settings.default_membership_fee_type_id → membership_fee_types.id` | SET NULL | Clear default if fee type deleted |
|
||||
| `member_groups.member_id → members.id` | CASCADE | Association removed; member preserved |
|
||||
| `member_groups.group_id → groups.id` | CASCADE | Association removed; group preserved |
|
||||
|
||||
2. **User → Role (N:1)**
|
||||
- Many users can be assigned to one role
|
||||
- `ON DELETE RESTRICT` - cannot delete role if users are assigned
|
||||
- Role links user to permission set for authorization
|
||||
`join_requests.reviewed_by_user_id` is intentionally **unconstrained** (no FK); `reviewed_by_display` is denormalized so the UI need not load the reviewer User.
|
||||
|
||||
3. **Member → CustomFieldValues (1:N)**
|
||||
- One member, many custom_field_values
|
||||
- `ON DELETE CASCADE` - custom_field_values deleted with member
|
||||
- Composite unique constraint (member_id, custom_field_id)
|
||||
|
||||
4. **CustomFieldValue → CustomField (N:1)**
|
||||
- Custom field values reference type definition
|
||||
- `ON DELETE RESTRICT` - cannot delete type if in use
|
||||
- Type defines data structure
|
||||
|
||||
5. **Member → MembershipFeeType (N:1, optional)**
|
||||
- Many members can be assigned to one fee type
|
||||
- `ON DELETE RESTRICT` - cannot delete fee type if members are assigned
|
||||
- Optional relationship (member can have no fee type)
|
||||
|
||||
6. **Member → MembershipFeeCycles (1:N)**
|
||||
- One member, many billing cycles
|
||||
- `ON DELETE CASCADE` - cycles deleted when member deleted
|
||||
- Unique constraint (member_id, cycle_start)
|
||||
|
||||
7. **MembershipFeeCycle → MembershipFeeType (N:1)**
|
||||
- Many cycles reference one fee type
|
||||
- `ON DELETE RESTRICT` - cannot delete fee type if cycles exist
|
||||
|
||||
8. **Settings → MembershipFeeType (N:1, optional)**
|
||||
- Settings can reference a default fee type
|
||||
- `ON DELETE SET NULL` - if fee type is deleted, setting is cleared
|
||||
|
||||
9. **Member ↔ Group (N:N via MemberGroup)**
|
||||
- Many-to-many relationship through `member_groups` join table
|
||||
- `ON DELETE CASCADE` on both sides - removing member/group removes associations
|
||||
- Unique constraint on (member_id, group_id) prevents duplicates
|
||||
- Groups searchable via member search vector
|
||||
**User ↔ Member** is an optional 1:1 (both sides may be NULL; entities exist independently). **Member ↔ Group** is many-to-many through `member_groups` (CASCADE lives only on the join table).
|
||||
|
||||
## Important Business Rules
|
||||
|
||||
### Email Synchronization
|
||||
- **User.email** is the source of truth when linked
|
||||
- On linking: Member.email ← User.email (overwrite)
|
||||
- After linking: Changes sync bidirectionally
|
||||
- Validation prevents email conflicts
|
||||
- **User.email is the source of truth when linked.** On linking, `Member.email ← User.email` (overwrite). Afterwards changes sync bidirectionally. Validation prevents email conflicts with other unlinked users.
|
||||
|
||||
### Authentication Strategies
|
||||
- **Password:** Email + hashed_password
|
||||
- **OIDC:** Email + oidc_id (Rauthy provider)
|
||||
- At least one method required per user
|
||||
- **Password:** email + hashed_password. **OIDC:** email + oidc_id (Rauthy provider), the external identity recorded via the `oidc_id` column on `users`. At least one method required per user.
|
||||
|
||||
### Member Constraints
|
||||
- First name and last name required (min 1 char)
|
||||
- Email unique, validated format (5-254 chars)
|
||||
- Exit date must be after join date
|
||||
- Phone: `+?[0-9\- ]{6,20}`
|
||||
- Postal code: optional (no format validation)
|
||||
- Country: optional
|
||||
- `first_name` / `last_name`: optional, but if present min 1 char.
|
||||
- `email`: unique, validated format (5–254 chars).
|
||||
- `exit_date` must be after `join_date`.
|
||||
- `postal_code`, `country`: optional, no format validation.
|
||||
|
||||
### CustomFieldValue System
|
||||
- Maximum one custom field value per custom field per member
|
||||
- Value stored as union type in JSONB
|
||||
- Supported types: string, integer, boolean, date, email
|
||||
- Types can be marked as immutable or required
|
||||
|
||||
## Indexes
|
||||
|
||||
### Performance Indexes
|
||||
|
||||
**members:**
|
||||
- `search_vector` (GIN) - Full-text search (tsvector)
|
||||
- `first_name` (GIN trgm) - Fuzzy search on first name
|
||||
- `last_name` (GIN trgm) - Fuzzy search on last name
|
||||
- `email` (GIN trgm) - Fuzzy search on email
|
||||
- `city` (GIN trgm) - Fuzzy search on city
|
||||
- `street` (GIN trgm) - Fuzzy search on street
|
||||
- `notes` (GIN trgm) - Fuzzy search on notes
|
||||
- `email` (B-tree) - Exact email lookups
|
||||
- `last_name` (B-tree) - Name sorting
|
||||
- `join_date` (B-tree) - Date filtering
|
||||
|
||||
**custom_field_values:**
|
||||
- `member_id` - Member custom field value lookups
|
||||
- `custom_field_id` - Type-based queries
|
||||
- Composite `(member_id, custom_field_id)` - Uniqueness
|
||||
|
||||
**tokens:**
|
||||
- `subject` - User token lookups
|
||||
- `expires_at` - Token cleanup
|
||||
- `purpose` - Purpose-based queries
|
||||
|
||||
**users:**
|
||||
- `email` (unique) - Login lookups
|
||||
- `oidc_id` (unique) - OIDC authentication
|
||||
- `member_id` (unique) - Member linkage
|
||||
- One value per custom field per member. Value stored as a union type in JSONB: `{type: "string|integer|boolean|date|email", value: <actual_value>}`. Custom fields can be marked `required` and toggled `show_in_overview`.
|
||||
|
||||
## Full-Text Search
|
||||
|
||||
### Implementation
|
||||
- **Trigger** on `members` (INSERT/UPDATE): runs function `members_search_vector_trigger()`
|
||||
- **Trigger** on `members` (INSERT/UPDATE): `update_search_vector` runs function `members_search_vector_trigger()`
|
||||
- **Trigger** on `custom_field_values` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_custom_field_value_change` runs function `update_member_search_vector_from_custom_field_value()`
|
||||
- **Trigger** on `member_groups` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_member_groups_change` runs function `update_member_search_vector_from_member_groups()`
|
||||
- **Index Type:** GIN (Generalized Inverted Index)
|
||||
|
||||
### Weighted Fields
|
||||
- **Weight A (highest):** first_name, last_name
|
||||
- **Weight B:** email, notes, group names (from member_groups → groups)
|
||||
- **Weight C:** city, street, house_number, postal_code, country, custom_field_values
|
||||
- **Weight C:** city, street, house_number, postal_code, custom_field_values
|
||||
- **Weight D (lowest):** join_date, exit_date
|
||||
|
||||
### Group Names in Search
|
||||
|
|
@ -264,285 +144,40 @@ WHERE search_vector @@ to_tsquery('simple', 'john & doe');
|
|||
|
||||
## Fuzzy Search (Trigram-based)
|
||||
|
||||
### Implementation
|
||||
- **Extension:** `pg_trgm` (PostgreSQL Trigram)
|
||||
- **Index Type:** GIN with `gin_trgm_ops` operator class
|
||||
- **Similarity Threshold:** 0.2 (default, configurable)
|
||||
- **Added:** November 2025 (PR #187, closes #162)
|
||||
- **Extension:** `pg_trgm`; GIN indexes with `gin_trgm_ops` on `first_name`, `last_name`, `email`, `city`, `street`, `notes`.
|
||||
- **Similarity threshold:** 0.2 (default, configurable) — balances precision/recall.
|
||||
- **Added:** November 2025 (PR #187, closes #162).
|
||||
|
||||
### How It Works
|
||||
Fuzzy search combines multiple search strategies:
|
||||
1. **Full-text search** - Primary filter using tsvector
|
||||
2. **Trigram similarity** - `similarity(field, query) > threshold`
|
||||
3. **Word similarity** - `word_similarity(query, field) > threshold`
|
||||
4. **Substring matching** - `LIKE` and `ILIKE` for exact substrings
|
||||
5. **Modulo operator** - `query % field` for quick similarity check
|
||||
Fuzzy search combines several strategies (applied as an OR-chain alongside full-text and substring matching):
|
||||
|
||||
### Indexed Fields for Fuzzy Search
|
||||
- `first_name` - GIN trigram index
|
||||
- `last_name` - GIN trigram index
|
||||
- `email` - GIN trigram index
|
||||
- `city` - GIN trigram index
|
||||
- `street` - GIN trigram index
|
||||
- `notes` - GIN trigram index
|
||||
1. Full-text search — primary filter via tsvector.
|
||||
2. Trigram similarity — `similarity(field, query) > threshold`.
|
||||
3. Word similarity — `word_similarity(query, field) > threshold`.
|
||||
4. Substring matching — `LIKE` / `ILIKE`.
|
||||
5. `%` operator — quick trigram-similarity check.
|
||||
|
||||
### Usage Example (Ash Action)
|
||||
```elixir
|
||||
# In LiveView or context
|
||||
Member.fuzzy_search(Member, query: "john", similarity_threshold: 0.2)
|
||||
|
||||
# Or using Ash Query directly
|
||||
Member
|
||||
|> Ash.Query.for_read(:search, %{query: "john", similarity_threshold: 0.2})
|
||||
|> Mv.Membership.read!()
|
||||
```
|
||||
|
||||
### Usage Example (SQL)
|
||||
```sql
|
||||
-- Trigram similarity search
|
||||
SELECT * FROM members
|
||||
WHERE similarity(first_name, 'john') > 0.2
|
||||
OR similarity(last_name, 'doe') > 0.2
|
||||
ORDER BY similarity(first_name, 'john') DESC;
|
||||
|
||||
-- Word similarity (better for partial matches)
|
||||
SELECT * FROM members
|
||||
WHERE word_similarity('john', first_name) > 0.2;
|
||||
|
||||
-- Quick similarity check with % operator
|
||||
SELECT * FROM members
|
||||
WHERE 'john' % first_name;
|
||||
```
|
||||
|
||||
### Performance Considerations
|
||||
- **GIN indexes** speed up trigram operations significantly
|
||||
- **Similarity threshold** of 0.2 balances precision and recall
|
||||
- **Combined approach** (FTS + trigram) provides best results
|
||||
- Lower threshold = more results but less specific
|
||||
For the Elixir search action and per-strategy filter functions, see `lib/membership/member.ex` and [`custom-fields-search-performance.md`](./custom-fields-search-performance.md).
|
||||
|
||||
## Database Extensions
|
||||
|
||||
### Required PostgreSQL Extensions
|
||||
Installed extensions are defined in `Mv.Repo.installed_extensions/0`:
|
||||
|
||||
1. **uuid-ossp**
|
||||
- Purpose: UUID generation functions
|
||||
- Used for: `gen_random_uuid()`, `uuid_generate_v7()`
|
||||
| Extension | Purpose | Notes |
|
||||
|-----------|---------|-------|
|
||||
| `ash-functions` | Ash helper SQL functions | installed by Ash |
|
||||
| `citext` | Case-insensitive text | `users.email` |
|
||||
| `pg_trgm` | Trigram fuzzy search | added in `20251001141005_add_trigram_to_members.exs`; operators `%`, `similarity()`, `word_similarity()` |
|
||||
|
||||
2. **citext**
|
||||
- Purpose: Case-insensitive text type
|
||||
- Used for: `users.email` (case-insensitive email matching)
|
||||
`gen_random_uuid()` is built into PostgreSQL; `uuid_generate_v7()` is a custom SQL function defined in a migration (not provided by an extension).
|
||||
|
||||
3. **pg_trgm**
|
||||
- Purpose: Trigram-based fuzzy text search and similarity matching
|
||||
- Used for: Fuzzy member search with similarity scoring
|
||||
- Operators: `%` (similarity), `word_similarity()`, `similarity()`
|
||||
- Added in: Migration `20251001141005_add_trigram_to_members.exs`
|
||||
## Sensitive Data (GDPR / logging)
|
||||
|
||||
### Installation
|
||||
```sql
|
||||
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
|
||||
CREATE EXTENSION IF NOT EXISTS "citext";
|
||||
CREATE EXTENSION IF NOT EXISTS "pg_trgm";
|
||||
```
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Ash Migrations
|
||||
This project uses Ash Framework's migration system:
|
||||
|
||||
```bash
|
||||
# Generate new migration
|
||||
mix ash.codegen --name add_new_feature
|
||||
|
||||
# Apply migrations
|
||||
mix ash.setup
|
||||
|
||||
# Rollback migrations
|
||||
mix ash_postgres.rollback -n 1
|
||||
```
|
||||
|
||||
### Migration Files Location
|
||||
```
|
||||
priv/repo/migrations/
|
||||
├── 20250421101957_initialize_extensions_1.exs
|
||||
├── 20250528163901_initial_migration.exs
|
||||
├── 20250617090641_member_fields.exs
|
||||
├── 20250620110850_add_accounts_domain.exs
|
||||
├── 20250912085235_AddSearchVectorToMembers.exs
|
||||
├── 20250926180341_add_unique_email_to_members.exs
|
||||
├── 20251001141005_add_trigram_to_members.exs
|
||||
└── 20251016130855_add_constraints_for_user_member_and_property.exs
|
||||
```
|
||||
|
||||
## Data Integrity
|
||||
|
||||
### Foreign Key Behaviors
|
||||
|
||||
| Relationship | On Delete | Rationale |
|
||||
|--------------|-----------|-----------|
|
||||
| `users.member_id → members.id` | SET NULL | Preserve user account when member deleted |
|
||||
| `custom_field_values.member_id → members.id` | CASCADE | Delete custom_field_values with member |
|
||||
| `custom_field_values.custom_field_id → custom_fields.id` | RESTRICT | Prevent deletion of types in use |
|
||||
|
||||
### Validation Layers
|
||||
|
||||
1. **Database Level:**
|
||||
- CHECK constraints
|
||||
- NOT NULL constraints
|
||||
- UNIQUE indexes
|
||||
- Foreign key constraints
|
||||
|
||||
2. **Application Level (Ash):**
|
||||
- Custom validators
|
||||
- Email format validation (EctoCommons.EmailValidator)
|
||||
- Business rule validation
|
||||
- Cross-entity validation
|
||||
|
||||
3. **UI Level:**
|
||||
- Client-side form validation
|
||||
- Real-time feedback
|
||||
- Error messages
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Query Patterns
|
||||
|
||||
**High Frequency:**
|
||||
- Member search (uses GIN index on search_vector)
|
||||
- Member list with filters (uses indexes on join_date, membership_fee_type_id)
|
||||
- User authentication (uses unique index on email/oidc_id)
|
||||
- CustomFieldValue lookups by member (uses index on member_id)
|
||||
|
||||
**Medium Frequency:**
|
||||
- Member CRUD operations
|
||||
- CustomFieldValue updates
|
||||
- Token validation
|
||||
|
||||
**Low Frequency:**
|
||||
- CustomField management
|
||||
- User-Member linking
|
||||
- Bulk operations
|
||||
|
||||
### Optimization Tips
|
||||
|
||||
1. **Use indexes:** All critical query paths have indexes
|
||||
2. **Preload relationships:** Use Ash's `load` to avoid N+1
|
||||
3. **Pagination:** Use keyset pagination (configured by default)
|
||||
4. **GIN indexes:** Full-text search and fuzzy search on multiple fields
|
||||
5. **Search optimization:** Full-text search via tsvector, not LIKE
|
||||
|
||||
## Visualization
|
||||
|
||||
### Using dbdiagram.io
|
||||
|
||||
1. Visit [https://dbdiagram.io](https://dbdiagram.io)
|
||||
2. Click "Import" → "From file"
|
||||
3. Upload `database_schema.dbml`
|
||||
4. View interactive diagram with relationships
|
||||
|
||||
### Using dbdocs.io
|
||||
|
||||
1. Install dbdocs CLI: `npm install -g dbdocs`
|
||||
2. Generate docs: `dbdocs build database_schema.dbml`
|
||||
3. View generated documentation
|
||||
|
||||
### VS Code Extension
|
||||
|
||||
Install "DBML Language" extension to view/edit DBML files with:
|
||||
- Syntax highlighting
|
||||
- Inline documentation
|
||||
- Error checking
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Sensitive Data
|
||||
|
||||
**Encrypted:**
|
||||
- `users.hashed_password` (bcrypt)
|
||||
|
||||
**Should Not Log:**
|
||||
- hashed_password
|
||||
- tokens (jti, purpose, extra_data)
|
||||
|
||||
**Personal Data (GDPR):**
|
||||
- All member fields (name, email, address)
|
||||
- User email
|
||||
- Token subject
|
||||
|
||||
### Access Control
|
||||
|
||||
- Implement through Ash policies
|
||||
- Row-level security considerations for future
|
||||
- Audit logging for sensitive operations
|
||||
|
||||
## Backup Recommendations
|
||||
|
||||
### Critical Tables (Priority 1)
|
||||
- `members` - Core business data
|
||||
- `users` - Authentication data
|
||||
- `custom_fields` - Schema definitions
|
||||
|
||||
### Important Tables (Priority 2)
|
||||
- `custom_field_values` - Member custom data
|
||||
- `tokens` - Can be regenerated but good to backup
|
||||
|
||||
### Backup Strategy
|
||||
```bash
|
||||
# Full database backup
|
||||
pg_dump -Fc mv_prod > backup_$(date +%Y%m%d).dump
|
||||
|
||||
# Restore
|
||||
pg_restore -d mv_prod backup_20251110.dump
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Test Database
|
||||
- Separate test database: `mv_test`
|
||||
- Sandbox mode via Ecto.Adapters.SQL.Sandbox
|
||||
- Reset between tests
|
||||
|
||||
### Seed Data
|
||||
```bash
|
||||
# Load seed data
|
||||
mix run priv/repo/seeds.exs
|
||||
```
|
||||
|
||||
## Future Considerations
|
||||
|
||||
### Potential Additions
|
||||
|
||||
1. **Audit Log Table**
|
||||
- Track changes to members
|
||||
- Compliance and history tracking
|
||||
|
||||
2. **Payment Tracking**
|
||||
- Payment history table
|
||||
- Transaction records
|
||||
- Fee calculation
|
||||
|
||||
3. **Document Storage**
|
||||
- Member documents/attachments
|
||||
- File metadata table
|
||||
|
||||
4. **Email Queue**
|
||||
- Outbound email tracking
|
||||
- Delivery status
|
||||
|
||||
5. **Roles & Permissions**
|
||||
- User roles (admin, treasurer, member)
|
||||
- Permission management
|
||||
|
||||
## Resources
|
||||
|
||||
- **Ash Framework:** [https://hexdocs.pm/ash](https://hexdocs.pm/ash)
|
||||
- **AshPostgres:** [https://hexdocs.pm/ash_postgres](https://hexdocs.pm/ash_postgres)
|
||||
- **DBML Specification:** [https://dbml.dbdiagram.io](https://dbml.dbdiagram.io)
|
||||
- **PostgreSQL Docs:** [https://www.postgresql.org/docs/](https://www.postgresql.org/docs/)
|
||||
- **Never log:** `users.hashed_password` (bcrypt), token fields (`jti`, `purpose`, `extra_data`), OIDC/SMTP/vereinfacht secrets in `settings`.
|
||||
- **Personal data:** all member fields, user email, join-request applicant data.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-01-27
|
||||
**Schema Version:** 1.5
|
||||
**Last Updated:** 2026-06-15
|
||||
**Schema Version:** 1.6 (12 tables)
|
||||
**Database:** PostgreSQL 17.6 (dev) / 16 (prod)
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,9 @@
|
|||
// - https://dbdocs.io
|
||||
// - VS Code Extensions: "DBML Language" or "dbdiagram.io"
|
||||
//
|
||||
// Version: 1.4
|
||||
// Last Updated: 2026-01-13
|
||||
// Version: 1.6
|
||||
// Last Updated: 2026-06-15
|
||||
// Hand-maintained (NOT auto-generated). 12 tables.
|
||||
|
||||
Project mila_membership_management {
|
||||
database_type: 'PostgreSQL'
|
||||
|
|
@ -25,15 +26,16 @@ Project mila_membership_management {
|
|||
- GDPR-compliant data management
|
||||
|
||||
## Domains:
|
||||
- **Accounts**: User authentication and session management
|
||||
- **Membership**: Club member data and custom fields
|
||||
- **Accounts**: User authentication, sessions, OIDC strategy identities
|
||||
- **Membership**: Club member data, custom fields, groups, settings, public join requests
|
||||
- **MembershipFees**: Membership fee types and billing cycles
|
||||
- **Authorization**: Role-based access control (RBAC)
|
||||
|
||||
## Required PostgreSQL Extensions:
|
||||
- uuid-ossp (UUID generation)
|
||||
## Required PostgreSQL Extensions (see Mv.Repo.installed_extensions/0):
|
||||
- ash-functions (Ash helper SQL functions)
|
||||
- citext (case-insensitive text)
|
||||
- pg_trgm (trigram-based fuzzy search)
|
||||
UUIDv7 ids use uuid_generate_v7(), a custom SQL function defined in a migration (not an extension).
|
||||
'''
|
||||
}
|
||||
|
||||
|
|
@ -135,6 +137,7 @@ Table members {
|
|||
search_vector tsvector [null, note: 'Full-text search index (auto-generated)']
|
||||
membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - assigned fee type']
|
||||
membership_fee_start_date date [null, note: 'Date from which membership fees should be calculated']
|
||||
vereinfacht_contact_id text [null, note: 'External contact id from the vereinfacht.de API (no FK; null if unlinked)']
|
||||
|
||||
indexes {
|
||||
email [unique, name: 'members_unique_email_index']
|
||||
|
|
@ -169,7 +172,8 @@ Table members {
|
|||
**Search Capabilities:**
|
||||
1. Full-Text Search (tsvector):
|
||||
- `search_vector` is auto-updated via trigger
|
||||
- Weighted fields: first_name (A), last_name (A), email (B), notes (B)
|
||||
- Weighted fields (A/B/C/D map): see the "Weighted Fields" section of
|
||||
database-schema-readme.md (single source of truth, matches the search trigger)
|
||||
- GIN index for fast text search
|
||||
|
||||
2. Fuzzy Search (pg_trgm):
|
||||
|
|
@ -225,7 +229,7 @@ Table custom_field_values {
|
|||
**Constraints:**
|
||||
- Each member can have only ONE custom field value per custom field
|
||||
- Custom field values are deleted when member is deleted (CASCADE)
|
||||
- Custom field cannot be deleted if custom field values exist (RESTRICT)
|
||||
- Custom field values are deleted when the custom field is deleted (CASCADE)
|
||||
|
||||
**Use Cases:**
|
||||
- Custom membership numbers
|
||||
|
|
@ -241,8 +245,9 @@ Table custom_fields {
|
|||
slug text [not null, unique, note: 'URL-friendly, immutable identifier (e.g., "membership-number"). Auto-generated from name.']
|
||||
value_type text [not null, note: 'Data type: string | integer | boolean | date | email']
|
||||
description text [null, note: 'Human-readable description']
|
||||
immutable boolean [not null, default: false, note: 'If true, value cannot be changed after creation']
|
||||
join_description text [null, note: 'Optional label shown for this field on the public join form (e.g., a GDPR confirmation text); supports inline external links. Falls back to name when null.']
|
||||
required boolean [not null, default: false, note: 'If true, all members must have this custom field']
|
||||
show_in_overview boolean [not null, default: true, note: 'If true, this custom field is displayed in the member overview table and can be sorted']
|
||||
|
||||
indexes {
|
||||
name [unique, name: 'custom_fields_unique_name_index']
|
||||
|
|
@ -259,8 +264,9 @@ Table custom_fields {
|
|||
- `slug`: URL-friendly, human-readable identifier (auto-generated, immutable)
|
||||
- `value_type`: Enforces data type consistency
|
||||
- `description`: Documentation for users/admins
|
||||
- `immutable`: Prevents changes after initial creation (e.g., membership numbers)
|
||||
- `join_description`: Optional label shown for this field on the public join form (falls back to `name` when null)
|
||||
- `required`: Enforces that all members must have this custom field
|
||||
- `show_in_overview`: When true, the field is shown in the member overview table and can be sorted
|
||||
|
||||
**Slug Generation:**
|
||||
- Automatically generated from `name` on creation
|
||||
|
|
@ -275,13 +281,13 @@ Table custom_fields {
|
|||
- `name` must be unique across all custom fields
|
||||
- `slug` must be unique across all custom fields
|
||||
- `slug` cannot be empty (validated on creation)
|
||||
- Cannot be deleted if custom_field_values reference it (ON DELETE RESTRICT)
|
||||
- Deleting a custom field cascades: its custom_field_values are deleted too (ON DELETE CASCADE)
|
||||
|
||||
**Examples:**
|
||||
- Membership Number (string, immutable, required) → slug: "membership-number"
|
||||
- Emergency Contact (string, mutable, optional) → slug: "emergency-contact"
|
||||
- Certified Trainer (boolean, mutable, optional) → slug: "certified-trainer"
|
||||
- Certification Date (date, immutable, optional) → slug: "certification-date"
|
||||
- Membership Number (string, required) → slug: "membership-number"
|
||||
- Emergency Contact (string, optional) → slug: "emergency-contact"
|
||||
- Certified Trainer (boolean, optional) → slug: "certified-trainer"
|
||||
- Certification Date (date, optional) → slug: "certification-date"
|
||||
'''
|
||||
}
|
||||
|
||||
|
|
@ -399,8 +405,8 @@ Ref: custom_field_values.member_id > members.id [delete: cascade]
|
|||
// CustomFieldValue → CustomField (N:1)
|
||||
// - Many custom_field_values can reference one custom field
|
||||
// - CustomFieldValue type defines the schema/behavior
|
||||
// - ON DELETE RESTRICT: Cannot delete type if custom_field_values exist
|
||||
Ref: custom_field_values.custom_field_id > custom_fields.id [delete: restrict]
|
||||
// - ON DELETE CASCADE: deleting the custom field deletes its custom_field_values
|
||||
Ref: custom_field_values.custom_field_id > custom_fields.id [delete: cascade]
|
||||
|
||||
// Member → MembershipFeeType (N:1)
|
||||
// - Many members can be assigned to one fee type
|
||||
|
|
@ -467,20 +473,9 @@ TableGroup accounts_domain {
|
|||
**Accounts Domain**
|
||||
|
||||
Handles user authentication and session management using AshAuthentication.
|
||||
Supports multiple authentication strategies (Password, OIDC).
|
||||
'''
|
||||
}
|
||||
|
||||
TableGroup membership_domain {
|
||||
members
|
||||
custom_field_values
|
||||
custom_fields
|
||||
|
||||
Note: '''
|
||||
**Membership Domain**
|
||||
|
||||
Core business logic for club membership management.
|
||||
Supports flexible, extensible member data model.
|
||||
Supports multiple authentication strategies (Password, OIDC). OIDC linking
|
||||
is recorded on the users table via the oidc_id column (there is no separate
|
||||
user_identities table).
|
||||
'''
|
||||
}
|
||||
|
||||
|
|
@ -550,9 +545,32 @@ Table roles {
|
|||
Table settings {
|
||||
id uuid [pk, not null, default: `gen_random_uuid()`, note: 'Primary identifier']
|
||||
club_name text [not null, note: 'The name of the association/club (min length: 1)']
|
||||
member_field_visibility jsonb [null, note: 'Visibility configuration for member fields in overview (JSONB map)']
|
||||
member_field_visibility jsonb [null, note: 'Visibility config for member fields in overview (JSONB map; absent key = visible)']
|
||||
member_field_required jsonb [null, note: 'Required-field config for member fields (JSONB map)']
|
||||
include_joining_cycle boolean [not null, default: true, note: 'Whether to include the joining cycle in membership fee generation']
|
||||
default_membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - default fee type for new members']
|
||||
default_membership_fee_type_id uuid [null, note: 'Logical reference to membership_fee_types (default fee type for new members) - app-enforced, NO DB foreign key']
|
||||
registration_enabled boolean [not null, default: true, note: 'Whether self-service user registration is enabled']
|
||||
oidc_only boolean [not null, default: false, note: 'If true, only OIDC login is offered (password login hidden)']
|
||||
oidc_client_id text [null, note: 'OIDC client id']
|
||||
oidc_client_secret text [null, note: 'OIDC client secret']
|
||||
oidc_base_url text [null, note: 'OIDC provider base URL (e.g., Rauthy)']
|
||||
oidc_redirect_uri text [null, note: 'OIDC redirect URI']
|
||||
oidc_admin_group_name text [null, note: 'Provider group name mapped to admin role on login']
|
||||
oidc_groups_claim text [null, note: 'JWT claim carrying the user groups for role sync']
|
||||
smtp_host text [null, note: 'Outbound SMTP host']
|
||||
smtp_port bigint [null, note: 'Outbound SMTP port']
|
||||
smtp_username text [null, note: 'SMTP auth username']
|
||||
smtp_password text [null, note: 'SMTP auth password (secret)']
|
||||
smtp_ssl text [null, note: 'SMTP TLS/SSL mode']
|
||||
smtp_from_name text [null, note: 'Display name for the From header (mail_from)']
|
||||
smtp_from_email text [null, note: 'Email address for the From header (mail_from)']
|
||||
vereinfacht_api_url text [null, note: 'vereinfacht.de API base URL']
|
||||
vereinfacht_api_key text [null, note: 'vereinfacht.de API key (secret)']
|
||||
vereinfacht_club_id text [null, note: 'vereinfacht.de club identifier']
|
||||
vereinfacht_app_url text [null, note: 'vereinfacht.de app URL (for links)']
|
||||
join_form_enabled boolean [not null, default: false, note: 'Whether the public join form is enabled']
|
||||
join_form_field_ids text[] [null, note: 'Ordered custom_field ids shown on the public join form']
|
||||
join_form_field_required jsonb [null, note: 'Per-field required config for the join form (JSONB map)']
|
||||
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
|
||||
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
|
||||
|
||||
|
|
@ -590,19 +608,123 @@ Table settings {
|
|||
'''
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// MEMBERSHIP DOMAIN — Groups
|
||||
// ============================================
|
||||
|
||||
Table groups {
|
||||
id uuid [pk, not null, default: `uuid_generate_v7()`, note: 'UUIDv7 primary key']
|
||||
name text [not null, note: 'Group name (unique case-insensitively via LOWER(name))']
|
||||
slug text [not null, unique, note: 'URL-friendly, immutable identifier auto-generated from name (shared GenerateSlug change)']
|
||||
description text [null, note: 'Optional description']
|
||||
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
|
||||
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
|
||||
|
||||
indexes {
|
||||
slug [unique, name: 'groups_unique_slug_index', note: 'Case-sensitive unique slug']
|
||||
name [unique, name: 'groups_unique_name_lower_index', note: 'UNIQUE on LOWER(name) - case-insensitive name uniqueness']
|
||||
}
|
||||
|
||||
Note: '''
|
||||
**Member Groups**
|
||||
|
||||
Flat groupings of members (no hierarchy in current schema). Many-to-many
|
||||
with members via member_groups. `slug` is generated by the shared
|
||||
Mv.Membership.Changes.GenerateSlug change (same as custom_fields) and is
|
||||
used for URL routing (/groups/:slug). Group names feed the member
|
||||
search_vector at weight B (see member_groups note).
|
||||
|
||||
**Future extension path (not yet in schema):**
|
||||
- parent_group_id (self-referential, nullable) + circular-ref guard + path calc for hierarchy
|
||||
- member_group_roles table linking MemberGroup to a Role (position within a group)
|
||||
'''
|
||||
}
|
||||
|
||||
Table member_groups {
|
||||
id uuid [pk, not null, default: `uuid_generate_v7()`, note: 'UUIDv7 primary key']
|
||||
member_id uuid [not null, note: 'FK to members']
|
||||
group_id uuid [not null, note: 'FK to groups']
|
||||
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
|
||||
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
|
||||
|
||||
indexes {
|
||||
(member_id, group_id) [unique, name: 'member_groups_unique_member_group_index', note: 'One association per member per group']
|
||||
member_id [name: 'member_groups_member_id_index']
|
||||
group_id [name: 'member_groups_group_id_index']
|
||||
}
|
||||
|
||||
Note: '''
|
||||
**Member ↔ Group Join Table**
|
||||
|
||||
CASCADE delete on BOTH foreign keys (the cascade lives only on the join
|
||||
table; members and groups themselves are never deleted by association
|
||||
removal). INSERT/UPDATE/DELETE here fires the trigger that refreshes the
|
||||
affected member's search_vector so group names (weight B) stay current.
|
||||
'''
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// MEMBERSHIP DOMAIN — Public Join Requests
|
||||
// ============================================
|
||||
|
||||
Table join_requests {
|
||||
id uuid [pk, not null, default: `uuid_generate_v7()`, note: 'UUIDv7 primary key']
|
||||
status text [not null, default: 'pending_confirmation', note: 'pending_confirmation → submitted (after email confirm) → approved/rejected']
|
||||
email text [not null, note: 'Applicant email']
|
||||
first_name text [null, note: 'Applicant first name']
|
||||
last_name text [null, note: 'Applicant last name']
|
||||
form_data jsonb [null, note: 'Submitted join-form field values (custom fields)']
|
||||
schema_version integer [null, note: 'Version of the join-form schema used at submission time']
|
||||
confirmation_token_hash text [null, note: 'Hash of the double-opt-in token (raw token never stored)']
|
||||
confirmation_token_expires_at timestamp [null, note: 'Token expiry (UTC)']
|
||||
confirmation_sent_at timestamp [null, note: 'When the confirmation email was sent (UTC)']
|
||||
submitted_at timestamp [null, note: 'When email was confirmed and request submitted (UTC)']
|
||||
approved_at timestamp [null, note: 'When an admin approved (UTC)']
|
||||
rejected_at timestamp [null, note: 'When an admin rejected (UTC)']
|
||||
reviewed_by_user_id uuid [null, note: 'User who reviewed (no FK constraint)']
|
||||
reviewed_by_display text [null, note: 'Reviewer display string, denormalized so the UI need not load the User']
|
||||
source text [null, note: 'Origin of the request (e.g., public form)']
|
||||
inserted_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Creation timestamp (UTC)']
|
||||
updated_at timestamp [not null, default: `now() AT TIME ZONE 'utc'`, note: 'Last update timestamp (UTC)']
|
||||
|
||||
indexes {
|
||||
confirmation_token_hash [unique, name: 'join_requests_confirmation_token_hash_unique', note: 'Partial unique WHERE confirmation_token_hash IS NOT NULL']
|
||||
email [name: 'join_requests_email_index']
|
||||
status [name: 'join_requests_status_index']
|
||||
}
|
||||
|
||||
Note: '''
|
||||
**Public Join Flow (Onboarding, Double Opt-In)**
|
||||
|
||||
Stores public join-form submissions. Double opt-in: the confirmation token
|
||||
is stored as a hash only; unconfirmed records have a ~24h retention and are
|
||||
removed by a scheduled cleanup job. `reviewed_by_user_id` is intentionally
|
||||
unconstrained (no FK); `reviewed_by_display` is denormalized so showing the
|
||||
reviewer does not require loading the User.
|
||||
'''
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// RELATIONSHIPS (Additional)
|
||||
// ============================================
|
||||
|
||||
// MemberGroup → Member (N:1)
|
||||
// - ON DELETE CASCADE (join table only): association removed, member preserved
|
||||
Ref: member_groups.member_id > members.id [delete: cascade]
|
||||
|
||||
// MemberGroup → Group (N:1)
|
||||
// - ON DELETE CASCADE (join table only): association removed, group preserved
|
||||
Ref: member_groups.group_id > groups.id [delete: cascade]
|
||||
|
||||
// User → Role (N:1)
|
||||
// - Many users can be assigned to one role
|
||||
// - ON DELETE RESTRICT: Cannot delete role if users are assigned
|
||||
Ref: users.role_id > roles.id [delete: restrict]
|
||||
|
||||
// Settings → MembershipFeeType (N:1, optional)
|
||||
// - Settings can reference a default membership fee type
|
||||
// - ON DELETE SET NULL: If fee type is deleted, setting is cleared
|
||||
Ref: settings.default_membership_fee_type_id > membership_fee_types.id [delete: set null]
|
||||
// Settings → MembershipFeeType (N:1, optional) — LOGICAL relationship only
|
||||
// - No DB foreign key (cross-domain dependency is deliberately avoided);
|
||||
// referential integrity is enforced in the app (Mv.Membership.Setting)
|
||||
Ref: settings.default_membership_fee_type_id > membership_fee_types.id
|
||||
|
||||
// ============================================
|
||||
// TABLE GROUPS (Updated)
|
||||
|
|
@ -624,12 +746,16 @@ TableGroup membership_domain {
|
|||
custom_field_values
|
||||
custom_fields
|
||||
settings
|
||||
groups
|
||||
member_groups
|
||||
join_requests
|
||||
|
||||
Note: '''
|
||||
**Membership Domain**
|
||||
|
||||
Core business logic for club membership management.
|
||||
Supports flexible, extensible member data model.
|
||||
Includes global application settings (singleton).
|
||||
Includes member groups (many-to-many), global application settings
|
||||
(singleton), and the public join-request flow.
|
||||
'''
|
||||
}
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue