# Database Schema Documentation

## Overview

This document provides a comprehensive overview of the Mila Membership Management System database schema.

- **DBML file:** [`database_schema.dbml`](./database_schema.dbml) — full per-column intent notes and relationship edges.
- **Search-vector performance:** see [`custom-fields-search-performance.md`](./custom-fields-search-performance.md) for trigger cost analysis and tuning.

The DBML is **hand-maintained** (not auto-generated); keep it in sync with `priv/repo/migrations/`.

## Schema Statistics

| Metric | Count |
|--------|-------|
| **Tables** | 12 |
| **Domains** | 4 (Accounts, Membership, MembershipFees, Authorization) |
| **Triggers** | 3 (member, custom_field_values, member_groups → member search-vector) |

## Tables Overview

### Accounts Domain
- **`users`** — authentication accounts. Dual auth (Password + OIDC), optional 1:1 link to a member; email is the source of truth when linked.
- **`tokens`** — JWT storage for AshAuthentication; multiple purposes, revocation by deletion.

OIDC account linking is recorded on the `users` table via the `oidc_id` column; there is no separate `user_identities` table.

### Membership Domain
- **`members`** — club member master data. Full-text + fuzzy search, bidirectional email sync with users, flexible address/contact data, `country`, optional `vereinfacht_contact_id` (external vereinfacht.de contact).
- **`custom_field_values`** — dynamic per-member attributes. Union-type value in JSONB; one value per custom field per member.
- **`custom_fields`** — schema definitions for custom field values (type, `required`/`show_in_overview` flags, optional `join_description`, auto-generated slug).
- **`settings`** — global application settings (singleton). Club name (also via `ASSOCIATION_NAME` env), member-field visibility/required maps, fee defaults, plus OIDC, SMTP/mail-from, vereinfacht.de, public join-form, `registration_enabled`, and `oidc_only` configuration. See [Settings configuration columns](#settings-configuration-columns).
- **`groups`** — member groupings. Case-insensitive-unique names, auto-generated immutable slugs, optional descriptions; many-to-many with members.
- **`member_groups`** — join table for members ↔ groups. Unique `(member_id, group_id)`, CASCADE delete on both sides (join table only).
- **`join_requests`** — public join flow (onboarding, double opt-in). Status machine `pending_confirmation → submitted → approved/rejected`; confirmation token stored as hash only, ~24h retention for unconfirmed records.

### Authorization Domain
- **`roles`** — RBAC. Links users to one of four hardcoded permission sets (`own_data`, `read_only`, `normal_user`, `admin`); system roles are deletion-protected.

### MembershipFees Domain
- **`membership_fee_types`** — fee types with immutable billing interval.
- **`membership_fee_cycles`** — per-member billing cycles with payment status.

## Settings configuration columns

The singleton `settings` row carries runtime configuration (all nullable unless noted). Grouped by area:

- **Member overview:** `member_field_visibility` (JSONB; absent key = visible), `member_field_required` (JSONB).
- **Membership fees:** `include_joining_cycle` (bool, NOT NULL, default true), `default_membership_fee_type_id` (FK → membership_fee_types, ON DELETE SET NULL).
- **Registration / login:** `registration_enabled` (bool, NOT NULL, default true), `oidc_only` (bool, NOT NULL, default false).
- **OIDC:** `oidc_client_id`, `oidc_client_secret`, `oidc_base_url`, `oidc_redirect_uri`, `oidc_admin_group_name`, `oidc_groups_claim`.
- **SMTP / mail-from:** `smtp_host`, `smtp_port` (bigint), `smtp_username`, `smtp_password`, `smtp_ssl`, `smtp_from_name`, `smtp_from_email`.
- **vereinfacht.de:** `vereinfacht_api_url`, `vereinfacht_api_key`, `vereinfacht_club_id`, `vereinfacht_app_url`.
- **Public join form:** `join_form_enabled` (bool, NOT NULL, default false), `join_form_field_ids` (text[]), `join_form_field_required` (JSONB).

## Key Relationships

```
User (0..1) ←→ (0..1) Member
       ↓              ↓
    Tokens (N)    CustomFieldValues (N)
       ↓                    ↓
    Role (N:1)        CustomField (1)

Member (1) → (N) MembershipFeeCycles
                    ↓
            MembershipFeeType (1)

Member (N) ←→ (N) Group
       ↓              ↓
  MemberGroups (N)  MemberGroups (N)

Settings (1) → MembershipFeeType (0..1)
```

## Foreign Key On-Delete Behavior

| Relationship | On Delete | Rationale |
|--------------|-----------|-----------|
| `users.member_id → members.id` | SET NULL | Preserve user account when member deleted |
| `users.role_id → roles.id` | RESTRICT | Cannot delete a role that still has users |
| `custom_field_values.member_id → members.id` | CASCADE | Delete values with member |
| `custom_field_values.custom_field_id → custom_fields.id` | CASCADE | Delete values when the custom field is deleted |
| `members.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type assigned to members |
| `membership_fee_cycles.member_id → members.id` | CASCADE | Cycles deleted with member |
| `membership_fee_cycles.membership_fee_type_id → membership_fee_types.id` | RESTRICT | Cannot delete a fee type with cycles |
| `settings.default_membership_fee_type_id → membership_fee_types.id` | SET NULL | Clear default if fee type deleted |
| `member_groups.member_id → members.id` | CASCADE | Association removed; member preserved |
| `member_groups.group_id → groups.id` | CASCADE | Association removed; group preserved |

`join_requests.reviewed_by_user_id` is intentionally **unconstrained** (no FK); `reviewed_by_display` is denormalized so the UI need not load the reviewer User.

**User ↔ Member** is an optional 1:1 (both sides may be NULL; entities exist independently). **Member ↔ Group** is many-to-many through `member_groups` (CASCADE lives only on the join table).

## Important Business Rules

### Email Synchronization
- **User.email is the source of truth when linked.** On linking, `Member.email ← User.email` (overwrite). Afterwards changes sync bidirectionally. Validation prevents email conflicts with other unlinked users.

### Authentication Strategies
- **Password:** email + hashed_password. **OIDC:** email + oidc_id (Rauthy provider), the external identity recorded via the `oidc_id` column on `users`. At least one method required per user.

### Member Constraints
- `first_name` / `last_name`: optional, but if present min 1 char.
- `email`: unique, validated format (5–254 chars).
- `exit_date` must be after `join_date`.
- `postal_code`, `country`: optional, no format validation.

### CustomFieldValue System
- One value per custom field per member. Value stored as a union type in JSONB: `{type: "string|integer|boolean|date|email", value: <actual_value>}`. Custom fields can be marked `required` and toggled `show_in_overview`.

## Full-Text Search

### Implementation
- **Trigger** on `members` (INSERT/UPDATE): `update_search_vector` runs function `members_search_vector_trigger()`
- **Trigger** on `custom_field_values` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_custom_field_value_change` runs function `update_member_search_vector_from_custom_field_value()`
- **Trigger** on `member_groups` (INSERT/UPDATE/DELETE): `update_member_search_vector_on_member_groups_change` runs function `update_member_search_vector_from_member_groups()`
- **Index Type:** GIN (Generalized Inverted Index)

### Weighted Fields
- **Weight A (highest):** first_name, last_name
- **Weight B:** email, notes, group names (from member_groups → groups)
- **Weight C:** city, street, house_number, postal_code, custom_field_values
- **Weight D (lowest):** join_date, exit_date

### Group Names in Search
Group names are included in the member search vector so that searching for a group name (e.g. "Vorstand") finds all members in that group:
- Group names are aggregated from `member_groups` joined with `groups` and receive weight 'B'
- The trigger `update_member_search_vector_on_member_groups_change` runs on INSERT/UPDATE/DELETE on `member_groups` and refreshes the affected member's `search_vector`
- See migration `20260217120000_add_group_names_to_member_search_vector.exs` (Issue #375)

### Custom Field Values in Search
Custom field values are automatically included in the search vector:
- All custom field values (string, integer, boolean, date, email) are aggregated and added to the search vector
- Values are converted to text format for indexing
- Custom field values receive weight 'C' (same as city, etc.)
- The search vector is automatically updated when custom field values are created, updated, or deleted via database triggers

### Usage Example
```sql
SELECT * FROM members
WHERE search_vector @@ to_tsquery('simple', 'john & doe');
```

## Fuzzy Search (Trigram-based)

- **Extension:** `pg_trgm`; GIN indexes with `gin_trgm_ops` on `first_name`, `last_name`, `email`, `city`, `street`, `notes`.
- **Similarity threshold:** 0.2 (default, configurable) — balances precision/recall.
- **Added:** November 2025 (PR #187, closes #162).

Fuzzy search combines several strategies (applied as an OR-chain alongside full-text and substring matching):

1. Full-text search — primary filter via tsvector.
2. Trigram similarity — `similarity(field, query) > threshold`.
3. Word similarity — `word_similarity(query, field) > threshold`.
4. Substring matching — `LIKE` / `ILIKE`.
5. `%` operator — quick trigram-similarity check.

For the Elixir search action and per-strategy filter functions, see `lib/membership/member.ex` and [`custom-fields-search-performance.md`](./custom-fields-search-performance.md).

## Database Extensions

Installed extensions are defined in `Mv.Repo.installed_extensions/0`:

| Extension | Purpose | Notes |
|-----------|---------|-------|
| `ash-functions` | Ash helper SQL functions | installed by Ash |
| `citext` | Case-insensitive text | `users.email` |
| `pg_trgm` | Trigram fuzzy search | added in `20251001141005_add_trigram_to_members.exs`; operators `%`, `similarity()`, `word_similarity()` |

`gen_random_uuid()` is built into PostgreSQL; `uuid_generate_v7()` is a custom SQL function defined in a migration (not provided by an extension).

## Sensitive Data (GDPR / logging)

- **Never log:** `users.hashed_password` (bcrypt), token fields (`jti`, `purpose`, `extra_data`), OIDC/SMTP/vereinfacht secrets in `settings`.
- **Personal data:** all member fields, user email, join-request applicant data.

---

**Last Updated:** 2026-06-15  
**Schema Version:** 1.6 (12 tables)  
**Database:** PostgreSQL 17.6 (dev) / 16 (prod)