183 lines
10 KiB
Markdown
183 lines
10 KiB
Markdown
# Database Schema Documentation
|
||
|
||
## Overview
|
||
|
||
This document provides a comprehensive overview of the Mila Membership Management System database schema.
|
||
|
||
- **DBML file:** [`database_schema.dbml`](./database_schema.dbml) — full per-column intent notes and relationship edges.
|
||
- **Search-vector performance:** see [`custom-fields-search-performance.md`](./custom-fields-search-performance.md) for trigger cost analysis and tuning.
|
||
|
||
The DBML is **hand-maintained** (not auto-generated); keep it in sync with `priv/repo/migrations/`.
|
||
|
||
## Schema Statistics
|
||
|
||
| Metric | Count |
|
||
|--------|-------|
|
||
| **Tables** | 12 |
|
||
| **Domains** | 4 (Accounts, Membership, MembershipFees, Authorization) |
|
||
| **Triggers** | 3 (member, custom_field_values, member_groups → member search-vector) |
|
||
|
||
## Tables Overview
|
||
|
||
### Accounts Domain
|
||
- **`users`** — authentication accounts. Dual auth (Password + OIDC), optional 1:1 link to a member; email is the source of truth when linked.
|
||
- **`tokens`** — JWT storage for AshAuthentication; multiple purposes, revocation by deletion.
|
||
|
||
OIDC account linking is recorded on the `users` table via the `oidc_id` column; there is no separate `user_identities` table.
|
||
|
||
### Membership Domain
|
||
- **`members`** — club member master data. Full-text + fuzzy search, bidirectional email sync with users, flexible address/contact data, `country`, optional `vereinfacht_contact_id` (external vereinfacht.de contact).
|
||
- **`custom_field_values`** — dynamic per-member attributes. Union-type value in JSONB; one value per custom field per member.
|
||
- **`custom_fields`** — schema definitions for custom field values (type, `required`/`show_in_overview` flags, optional `join_description`, auto-generated slug).
|
||
- **`settings`** — global application settings (singleton). Club name (also via `ASSOCIATION_NAME` env), member-field visibility/required maps, fee defaults, plus OIDC, SMTP/mail-from, vereinfacht.de, public join-form, `registration_enabled`, and `oidc_only` configuration. See [Settings configuration columns](#settings-configuration-columns).
|
||
- **`groups`** — member groupings. Case-insensitive-unique names, auto-generated immutable slugs, optional descriptions; many-to-many with members.
|
||
- **`member_groups`** — join table for members ↔ groups. Unique `(member_id, group_id)`, CASCADE delete on both sides (join table only).
|
||
- **`join_requests`** — public join flow (onboarding, double opt-in). Status machine `pending_confirmation → submitted → approved/rejected`; confirmation token stored as hash only, ~24h retention for unconfirmed records.
|
||
|
||
### Authorization Domain
|
||
- **`roles`** — RBAC. Links users to one of four hardcoded permission sets (`own_data`, `read_only`, `normal_user`, `admin`); system roles are deletion-protected.
|
||
|
||
### MembershipFees Domain
|
||
- **`membership_fee_types`** — fee types with immutable billing interval.
|
||
- **`membership_fee_cycles`** — per-member billing cycles with payment status.
|
||
|
||
## Settings configuration columns
|
||
|
||
The singleton `settings` row carries runtime configuration (all nullable unless noted). Grouped by area:
|
||
|
||
- **Member overview:** `member_field_visibility` (JSONB; absent key = visible), `member_field_required` (JSONB).
|
||
- **Membership fees:** `include_joining_cycle` (bool, NOT NULL, default true), `default_membership_fee_type_id` (FK → membership_fee_types, ON DELETE SET NULL).
|
||
- **Registration / login:** `registration_enabled` (bool, NOT NULL, default true), `oidc_only` (bool, NOT NULL, default false).
|
||
- **OIDC:** `oidc_client_id`, `oidc_client_secret`, `oidc_base_url`, `oidc_redirect_uri`, `oidc_admin_group_name`, `oidc_groups_claim`.
|
||
- **SMTP / mail-from:** `smtp_host`, `smtp_port` (bigint), `smtp_username`, `smtp_password`, `smtp_ssl`, `smtp_from_name`, `smtp_from_email`.
|
||
- **vereinfacht.de:** `vereinfacht_api_url`, `vereinfacht_api_key`, `vereinfacht_club_id`, `vereinfacht_app_url`.
|
||
- **Public join form:** `join_form_enabled` (bool, NOT NULL, default false), `join_form_field_ids` (text[]), `join_form_field_required` (JSONB).
|
||
|
||
## Key Relationships
|
||
|
||
```
|
||
User (0..1) ←→ (0..1) Member
|
||
↓ ↓
|
||
Tokens (N) CustomFieldValues (N)
|
||
↓ ↓
|
||
Role (N:1) CustomField (1)
|
||
|
||
Member (1) → (N) MembershipFeeCycles
|
||
↓
|
||
MembershipFeeType (1)
|
||
|
||
Member (N) ←→ (N) Group
|
||
↓ ↓
|
||
MemberGroups (N) MemberGroups (N)
|
||
|
||
Settings (1) → MembershipFeeType (0..1)
|
||
```
|
||
|
||
## Foreign Key On-Delete Behavior
|
||
|
||
| Relationship | On Delete | Rationale |
|
||
|--------------|-----------|-----------|
|
||
| `users.member_id → members.id` | SET NULL | Preserve user account when member deleted |
|
||
| `users.role_id → roles.id` | RESTRICT | Cannot delete a role that still has users |
|
||
| `custom_field_values.member_id → members.id` | CASCADE | Delete values with member |
|
||
| `custom_field_values.custom_field_id → custom_fields.id` | CASCADE | Delete values when the custom field is deleted |
|
||
| `members.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type assigned to members |
|
||
| `membership_fee_cycles.member_id → members.id` | CASCADE | Cycles deleted with member |
|
||
| `membership_fee_cycles.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type with cycles |
|
||
| `settings.default_membership_fee_type_id → membership_fee_types.id` | SET NULL | Clear default if fee type deleted |
|
||
| `member_groups.member_id → members.id` | CASCADE | Association removed; member preserved |
|
||
| `member_groups.group_id → groups.id` | CASCADE | Association removed; group preserved |
|
||
|
||
`join_requests.reviewed_by_user_id` is intentionally **unconstrained** (no FK); `reviewed_by_display` is denormalized so the UI need not load the reviewer User.
|
||
|
||
**User ↔ Member** is an optional 1:1 (both sides may be NULL; entities exist independently). **Member ↔ Group** is many-to-many through `member_groups` (CASCADE lives only on the join table).
|
||
|
||
## Important Business Rules
|
||
|
||
### Email Synchronization
|
||
- **User.email is the source of truth when linked.** On linking, `Member.email ← User.email` (overwrite). Afterwards changes sync bidirectionally. Validation prevents email conflicts with other unlinked users.
|
||
|
||
### Authentication Strategies
|
||
- **Password:** email + hashed_password. **OIDC:** email + oidc_id (Rauthy provider), the external identity recorded via the `oidc_id` column on `users`. At least one method required per user.
|
||
|
||
### Member Constraints
|
||
- `first_name` / `last_name`: optional, but if present min 1 char.
|
||
- `email`: unique, validated format (5–254 chars).
|
||
- `exit_date` must be after `join_date`.
|
||
- `postal_code`, `country`: optional, no format validation.
|
||
|
||
### CustomFieldValue System
|
||
- One value per custom field per member. Value stored as a union type in JSONB: `{type: "string|integer|boolean|date|email", value: <actual_value>}`. Custom fields can be marked `required` and toggled `show_in_overview`.
|
||
|
||
## Full-Text Search
|
||
|
||
### Implementation
|
||
- **Trigger** on `members` (INSERT/UPDATE): `update_search_vector` runs function `members_search_vector_trigger()`
|
||
- **Trigger** on `custom_field_values` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_custom_field_value_change` runs function `update_member_search_vector_from_custom_field_value()`
|
||
- **Trigger** on `member_groups` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_member_groups_change` runs function `update_member_search_vector_from_member_groups()`
|
||
- **Index Type:** GIN (Generalized Inverted Index)
|
||
|
||
### Weighted Fields
|
||
- **Weight A (highest):** first_name, last_name
|
||
- **Weight B:** email, notes, group names (from member_groups → groups)
|
||
- **Weight C:** city, street, house_number, postal_code, custom_field_values
|
||
- **Weight D (lowest):** join_date, exit_date
|
||
|
||
### Group Names in Search
|
||
Group names are included in the member search vector so that searching for a group name (e.g. "Vorstand") finds all members in that group:
|
||
- Group names are aggregated from `member_groups` joined with `groups` and receive weight 'B'
|
||
- The trigger `update_member_search_vector_on_member_groups_change` runs on INSERT/UPDATE/DELETE on `member_groups` and refreshes the affected member's `search_vector`
|
||
- See migration `20260217120000_add_group_names_to_member_search_vector.exs` (Issue #375)
|
||
|
||
### Custom Field Values in Search
|
||
Custom field values are automatically included in the search vector:
|
||
- All custom field values (string, integer, boolean, date, email) are aggregated and added to the search vector
|
||
- Values are converted to text format for indexing
|
||
- Custom field values receive weight 'C' (same as city, etc.)
|
||
- The search vector is automatically updated when custom field values are created, updated, or deleted via database triggers
|
||
|
||
### Usage Example
|
||
```sql
|
||
SELECT * FROM members
|
||
WHERE search_vector @@ to_tsquery('simple', 'john & doe');
|
||
```
|
||
|
||
## Fuzzy Search (Trigram-based)
|
||
|
||
- **Extension:** `pg_trgm`; GIN indexes with `gin_trgm_ops` on `first_name`, `last_name`, `email`, `city`, `street`, `notes`.
|
||
- **Similarity threshold:** 0.2 (default, configurable) — balances precision/recall.
|
||
- **Added:** November 2025 (PR #187, closes #162).
|
||
|
||
Fuzzy search combines several strategies (applied as an OR-chain alongside full-text and substring matching):
|
||
|
||
1. Full-text search — primary filter via tsvector.
|
||
2. Trigram similarity — `similarity(field, query) > threshold`.
|
||
3. Word similarity — `word_similarity(query, field) > threshold`.
|
||
4. Substring matching — `LIKE` / `ILIKE`.
|
||
5. `%` operator — quick trigram-similarity check.
|
||
|
||
For the Elixir search action and per-strategy filter functions, see `lib/membership/member.ex` and [`custom-fields-search-performance.md`](./custom-fields-search-performance.md).
|
||
|
||
## Database Extensions
|
||
|
||
Installed extensions are defined in `Mv.Repo.installed_extensions/0`:
|
||
|
||
| Extension | Purpose | Notes |
|
||
|-----------|---------|-------|
|
||
| `ash-functions` | Ash helper SQL functions | installed by Ash |
|
||
| `citext` | Case-insensitive text | `users.email` |
|
||
| `pg_trgm` | Trigram fuzzy search | added in `20251001141005_add_trigram_to_members.exs`; operators `%`, `similarity()`, `word_similarity()` |
|
||
|
||
`gen_random_uuid()` is built into PostgreSQL; `uuid_generate_v7()` is a custom SQL function defined in a migration (not provided by an extension).
|
||
|
||
## Sensitive Data (GDPR / logging)
|
||
|
||
- **Never log:** `users.hashed_password` (bcrypt), token fields (`jti`, `purpose`, `extra_data`), OIDC/SMTP/vereinfacht secrets in `settings`.
|
||
- **Personal data:** all member fields, user email, join-request applicant data.
|
||
|
||
---
|
||
|
||
**Last Updated:** 2026-06-15
|
||
**Schema Version:** 1.6 (12 tables)
|
||
**Database:** PostgreSQL 17.6 (dev) / 16 (prod)
|
||
|