Merge branch 'main' into feature/roles-and-permissions-concept
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
This commit is contained in:
commit
93916a09f9
56 changed files with 5738 additions and 1373 deletions
|
|
@ -52,21 +52,21 @@ This document provides a comprehensive overview of the Mila Membership Managemen
|
|||
- Bidirectional email sync with users
|
||||
- Flexible address and contact data
|
||||
|
||||
#### `properties`
|
||||
#### `custom_field_values`
|
||||
- **Purpose:** Dynamic custom member attributes
|
||||
- **Rows (Estimated):** Variable (N per member)
|
||||
- **Key Features:**
|
||||
- Union type value storage (JSONB)
|
||||
- Multiple data types supported
|
||||
- One property per type per member
|
||||
- One custom field value per custom field per member
|
||||
|
||||
#### `property_types`
|
||||
- **Purpose:** Schema definitions for custom properties
|
||||
#### `custom_fields`
|
||||
- **Purpose:** Schema definitions for custom_field_values
|
||||
- **Rows (Estimated):** Low (admin-defined)
|
||||
- **Key Features:**
|
||||
- Type definitions
|
||||
- Immutable and required flags
|
||||
- Centralized property management
|
||||
- Centralized custom field management
|
||||
|
||||
## Key Relationships
|
||||
|
||||
|
|
@ -77,7 +77,7 @@ User (0..1) ←→ (0..1) Member
|
|||
|
||||
Member (1) → (N) Properties
|
||||
↓
|
||||
PropertyType (1)
|
||||
CustomField (1)
|
||||
```
|
||||
|
||||
### Relationship Details
|
||||
|
|
@ -90,11 +90,11 @@ Member (1) → (N) Properties
|
|||
- `ON DELETE SET NULL` on user side (User preserved when Member deleted)
|
||||
|
||||
2. **Member → Properties (1:N)**
|
||||
- One member, many properties
|
||||
- `ON DELETE CASCADE` - properties deleted with member
|
||||
- Composite unique constraint (member_id, property_type_id)
|
||||
- One member, many custom_field_values
|
||||
- `ON DELETE CASCADE` - custom_field_values deleted with member
|
||||
- Composite unique constraint (member_id, custom_field_id)
|
||||
|
||||
3. **Property → PropertyType (N:1)**
|
||||
3. **CustomFieldValue → CustomField (N:1)**
|
||||
- Properties reference type definition
|
||||
- `ON DELETE RESTRICT` - cannot delete type if in use
|
||||
- Type defines data structure
|
||||
|
|
@ -121,8 +121,8 @@ Member (1) → (N) Properties
|
|||
- Phone: `+?[0-9\- ]{6,20}`
|
||||
- Postal code: 5 digits
|
||||
|
||||
### Property System
|
||||
- Maximum one property per type per member
|
||||
### CustomFieldValue System
|
||||
- Maximum one custom field value per custom field per member
|
||||
- Value stored as union type in JSONB
|
||||
- Supported types: string, integer, boolean, date, email
|
||||
- Types can be marked as immutable or required
|
||||
|
|
@ -132,16 +132,22 @@ Member (1) → (N) Properties
|
|||
### Performance Indexes
|
||||
|
||||
**members:**
|
||||
- `search_vector` (GIN) - Full-text search
|
||||
- `email` - Email lookups
|
||||
- `last_name` - Name sorting
|
||||
- `join_date` - Date filtering
|
||||
- `paid` (partial) - Payment status queries
|
||||
- `search_vector` (GIN) - Full-text search (tsvector)
|
||||
- `first_name` (GIN trgm) - Fuzzy search on first name
|
||||
- `last_name` (GIN trgm) - Fuzzy search on last name
|
||||
- `email` (GIN trgm) - Fuzzy search on email
|
||||
- `city` (GIN trgm) - Fuzzy search on city
|
||||
- `street` (GIN trgm) - Fuzzy search on street
|
||||
- `notes` (GIN trgm) - Fuzzy search on notes
|
||||
- `email` (B-tree) - Exact email lookups
|
||||
- `last_name` (B-tree) - Name sorting
|
||||
- `join_date` (B-tree) - Date filtering
|
||||
- `paid` (partial B-tree) - Payment status queries
|
||||
|
||||
**properties:**
|
||||
- `member_id` - Member property lookups
|
||||
- `property_type_id` - Type-based queries
|
||||
- Composite `(member_id, property_type_id)` - Uniqueness
|
||||
**custom_field_values:**
|
||||
- `member_id` - Member custom field value lookups
|
||||
- `custom_field_id` - Type-based queries
|
||||
- Composite `(member_id, custom_field_id)` - Uniqueness
|
||||
|
||||
**tokens:**
|
||||
- `subject` - User token lookups
|
||||
|
|
@ -172,6 +178,64 @@ SELECT * FROM members
|
|||
WHERE search_vector @@ to_tsquery('simple', 'john & doe');
|
||||
```
|
||||
|
||||
## Fuzzy Search (Trigram-based)
|
||||
|
||||
### Implementation
|
||||
- **Extension:** `pg_trgm` (PostgreSQL Trigram)
|
||||
- **Index Type:** GIN with `gin_trgm_ops` operator class
|
||||
- **Similarity Threshold:** 0.2 (default, configurable)
|
||||
- **Added:** November 2025 (PR #187, closes #162)
|
||||
|
||||
### How It Works
|
||||
Fuzzy search combines multiple search strategies:
|
||||
1. **Full-text search** - Primary filter using tsvector
|
||||
2. **Trigram similarity** - `similarity(field, query) > threshold`
|
||||
3. **Word similarity** - `word_similarity(query, field) > threshold`
|
||||
4. **Substring matching** - `LIKE` and `ILIKE` for exact substrings
|
||||
5. **Modulo operator** - `query % field` for quick similarity check
|
||||
|
||||
### Indexed Fields for Fuzzy Search
|
||||
- `first_name` - GIN trigram index
|
||||
- `last_name` - GIN trigram index
|
||||
- `email` - GIN trigram index
|
||||
- `city` - GIN trigram index
|
||||
- `street` - GIN trigram index
|
||||
- `notes` - GIN trigram index
|
||||
|
||||
### Usage Example (Ash Action)
|
||||
```elixir
|
||||
# In LiveView or context
|
||||
Member.fuzzy_search(Member, query: "john", similarity_threshold: 0.2)
|
||||
|
||||
# Or using Ash Query directly
|
||||
Member
|
||||
|> Ash.Query.for_read(:search, %{query: "john", similarity_threshold: 0.2})
|
||||
|> Mv.Membership.read!()
|
||||
```
|
||||
|
||||
### Usage Example (SQL)
|
||||
```sql
|
||||
-- Trigram similarity search
|
||||
SELECT * FROM members
|
||||
WHERE similarity(first_name, 'john') > 0.2
|
||||
OR similarity(last_name, 'doe') > 0.2
|
||||
ORDER BY similarity(first_name, 'john') DESC;
|
||||
|
||||
-- Word similarity (better for partial matches)
|
||||
SELECT * FROM members
|
||||
WHERE word_similarity('john', first_name) > 0.2;
|
||||
|
||||
-- Quick similarity check with % operator
|
||||
SELECT * FROM members
|
||||
WHERE 'john' % first_name;
|
||||
```
|
||||
|
||||
### Performance Considerations
|
||||
- **GIN indexes** speed up trigram operations significantly
|
||||
- **Similarity threshold** of 0.2 balances precision and recall
|
||||
- **Combined approach** (FTS + trigram) provides best results
|
||||
- Lower threshold = more results but less specific
|
||||
|
||||
## Database Extensions
|
||||
|
||||
### Required PostgreSQL Extensions
|
||||
|
|
@ -184,10 +248,17 @@ WHERE search_vector @@ to_tsquery('simple', 'john & doe');
|
|||
- Purpose: Case-insensitive text type
|
||||
- Used for: `users.email` (case-insensitive email matching)
|
||||
|
||||
3. **pg_trgm**
|
||||
- Purpose: Trigram-based fuzzy text search and similarity matching
|
||||
- Used for: Fuzzy member search with similarity scoring
|
||||
- Operators: `%` (similarity), `word_similarity()`, `similarity()`
|
||||
- Added in: Migration `20251001141005_add_trigram_to_members.exs`
|
||||
|
||||
### Installation
|
||||
```sql
|
||||
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
|
||||
CREATE EXTENSION IF NOT EXISTS "citext";
|
||||
CREATE EXTENSION IF NOT EXISTS "pg_trgm";
|
||||
```
|
||||
|
||||
## Migration Strategy
|
||||
|
|
@ -215,6 +286,7 @@ priv/repo/migrations/
|
|||
├── 20250620110850_add_accounts_domain.exs
|
||||
├── 20250912085235_AddSearchVectorToMembers.exs
|
||||
├── 20250926180341_add_unique_email_to_members.exs
|
||||
├── 20251001141005_add_trigram_to_members.exs
|
||||
└── 20251016130855_add_constraints_for_user_member_and_property.exs
|
||||
```
|
||||
|
||||
|
|
@ -225,8 +297,8 @@ priv/repo/migrations/
|
|||
| Relationship | On Delete | Rationale |
|
||||
|--------------|-----------|-----------|
|
||||
| `users.member_id → members.id` | SET NULL | Preserve user account when member deleted |
|
||||
| `properties.member_id → members.id` | CASCADE | Delete properties with member |
|
||||
| `properties.property_type_id → property_types.id` | RESTRICT | Prevent deletion of types in use |
|
||||
| `custom_field_values.member_id → members.id` | CASCADE | Delete custom_field_values with member |
|
||||
| `custom_field_values.custom_field_id → custom_fields.id` | RESTRICT | Prevent deletion of types in use |
|
||||
|
||||
### Validation Layers
|
||||
|
||||
|
|
@ -255,15 +327,15 @@ priv/repo/migrations/
|
|||
- Member search (uses GIN index on search_vector)
|
||||
- Member list with filters (uses indexes on join_date, paid)
|
||||
- User authentication (uses unique index on email/oidc_id)
|
||||
- Property lookups by member (uses index on member_id)
|
||||
- CustomFieldValue lookups by member (uses index on member_id)
|
||||
|
||||
**Medium Frequency:**
|
||||
- Member CRUD operations
|
||||
- Property updates
|
||||
- CustomFieldValue updates
|
||||
- Token validation
|
||||
|
||||
**Low Frequency:**
|
||||
- PropertyType management
|
||||
- CustomField management
|
||||
- User-Member linking
|
||||
- Bulk operations
|
||||
|
||||
|
|
@ -324,10 +396,10 @@ Install "DBML Language" extension to view/edit DBML files with:
|
|||
### Critical Tables (Priority 1)
|
||||
- `members` - Core business data
|
||||
- `users` - Authentication data
|
||||
- `property_types` - Schema definitions
|
||||
- `custom_fields` - Schema definitions
|
||||
|
||||
### Important Tables (Priority 2)
|
||||
- `properties` - Member custom data
|
||||
- `custom_field_values` - Member custom data
|
||||
- `tokens` - Can be regenerated but good to backup
|
||||
|
||||
### Backup Strategy
|
||||
|
|
@ -386,7 +458,7 @@ mix run priv/repo/seeds.exs
|
|||
|
||||
---
|
||||
|
||||
**Last Updated:** 2025-11-10
|
||||
**Schema Version:** 1.0
|
||||
**Last Updated:** 2025-11-13
|
||||
**Schema Version:** 1.1
|
||||
**Database:** PostgreSQL 17.6 (dev) / 16 (prod)
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,8 @@
|
|||
// - https://dbdocs.io
|
||||
// - VS Code Extensions: "DBML Language" or "dbdiagram.io"
|
||||
//
|
||||
// Version: 1.0
|
||||
// Last Updated: 2025-11-10
|
||||
// Version: 1.1
|
||||
// Last Updated: 2025-11-13
|
||||
|
||||
Project mila_membership_management {
|
||||
database_type: 'PostgreSQL'
|
||||
|
|
@ -17,15 +17,21 @@ Project mila_membership_management {
|
|||
A membership management application for small to mid-sized clubs.
|
||||
|
||||
## Key Features:
|
||||
- User authentication (OIDC + Password)
|
||||
- Member management with flexible custom properties
|
||||
- User authentication (OIDC + Password with secure account linking)
|
||||
- Member management with flexible custom fields
|
||||
- Bidirectional email synchronization between users and members
|
||||
- Full-text search capabilities
|
||||
- Full-text search capabilities (tsvector)
|
||||
- Fuzzy search with trigram matching (pg_trgm)
|
||||
- GDPR-compliant data management
|
||||
|
||||
## Domains:
|
||||
- **Accounts**: User authentication and session management
|
||||
- **Membership**: Club member data and custom properties
|
||||
- **Membership**: Club member data and custom fields
|
||||
|
||||
## Required PostgreSQL Extensions:
|
||||
- uuid-ossp (UUID generation)
|
||||
- citext (case-insensitive text)
|
||||
- pg_trgm (trigram-based fuzzy search)
|
||||
'''
|
||||
}
|
||||
|
||||
|
|
@ -130,10 +136,16 @@ Table members {
|
|||
|
||||
indexes {
|
||||
email [unique, name: 'members_unique_email_index']
|
||||
search_vector [type: gin, name: 'members_search_vector_idx', note: 'GIN index for full-text search']
|
||||
email [name: 'members_email_idx']
|
||||
last_name [name: 'members_last_name_idx', note: 'For name sorting']
|
||||
join_date [name: 'members_join_date_idx', note: 'For date filters']
|
||||
search_vector [type: gin, name: 'members_search_vector_idx', note: 'GIN index for full-text search (tsvector)']
|
||||
first_name [type: gin, name: 'members_first_name_trgm_idx', note: 'GIN trigram index for fuzzy search']
|
||||
last_name [type: gin, name: 'members_last_name_trgm_idx', note: 'GIN trigram index for fuzzy search']
|
||||
email [type: gin, name: 'members_email_trgm_idx', note: 'GIN trigram index for fuzzy search']
|
||||
city [type: gin, name: 'members_city_trgm_idx', note: 'GIN trigram index for fuzzy search']
|
||||
street [type: gin, name: 'members_street_trgm_idx', note: 'GIN trigram index for fuzzy search']
|
||||
notes [type: gin, name: 'members_notes_trgm_idx', note: 'GIN trigram index for fuzzy search']
|
||||
email [name: 'members_email_idx', note: 'B-tree index for exact lookups']
|
||||
last_name [name: 'members_last_name_idx', note: 'B-tree index for name sorting']
|
||||
join_date [name: 'members_join_date_idx', note: 'B-tree index for date filters']
|
||||
(paid) [name: 'members_paid_idx', type: btree, note: 'Partial index WHERE paid IS NOT NULL']
|
||||
}
|
||||
|
||||
|
|
@ -152,14 +164,21 @@ Table members {
|
|||
- Subsequent changes to either email sync bidirectionally
|
||||
- Validates that email is not already used by another unlinked user
|
||||
|
||||
**Full-Text Search:**
|
||||
- `search_vector` is auto-updated via trigger
|
||||
- Weighted fields: first_name (A), last_name (A), email (B), notes (B)
|
||||
- Supports flexible member search across multiple fields
|
||||
**Search Capabilities:**
|
||||
1. Full-Text Search (tsvector):
|
||||
- `search_vector` is auto-updated via trigger
|
||||
- Weighted fields: first_name (A), last_name (A), email (B), notes (B)
|
||||
- GIN index for fast text search
|
||||
|
||||
2. Fuzzy Search (pg_trgm):
|
||||
- Trigram-based similarity matching
|
||||
- 6 GIN trigram indexes on searchable fields
|
||||
- Configurable similarity threshold (default 0.2)
|
||||
- Supports typos and partial matches
|
||||
|
||||
**Relationships:**
|
||||
- Optional 1:1 with users (0..1 ↔ 0..1) - authentication account
|
||||
- 1:N with properties (custom dynamic fields)
|
||||
- 1:N with custom_field_values (custom dynamic fields)
|
||||
|
||||
**Validation Rules:**
|
||||
- first_name, last_name: min 1 character
|
||||
|
|
@ -172,20 +191,20 @@ Table members {
|
|||
'''
|
||||
}
|
||||
|
||||
Table properties {
|
||||
Table custom_field_values {
|
||||
id uuid [pk, not null, default: `gen_random_uuid()`, note: 'Primary identifier']
|
||||
value jsonb [null, note: 'Union type value storage (format: {type: "string", value: "example"})']
|
||||
member_id uuid [not null, note: 'Link to member']
|
||||
property_type_id uuid [not null, note: 'Link to property type definition']
|
||||
custom_field_id uuid [not null, note: 'Link to custom field definition']
|
||||
|
||||
indexes {
|
||||
(member_id, property_type_id) [unique, name: 'properties_unique_property_per_member_index', note: 'One property per type per member']
|
||||
member_id [name: 'properties_member_id_idx']
|
||||
property_type_id [name: 'properties_property_type_id_idx']
|
||||
(member_id, custom_field_id) [unique, name: 'custom_field_values_unique_custom_field_per_member_index', note: 'One custom field value per custom field per member']
|
||||
member_id [name: 'custom_field_values_member_id_idx']
|
||||
custom_field_id [name: 'custom_field_values_custom_field_id_idx']
|
||||
}
|
||||
|
||||
Note: '''
|
||||
**Dynamic Custom Member Properties**
|
||||
**Dynamic Custom Member Field Values**
|
||||
|
||||
Provides flexible, extensible attributes for members beyond the fixed schema.
|
||||
|
||||
|
|
@ -202,9 +221,9 @@ Table properties {
|
|||
- `email`: Validated email addresses
|
||||
|
||||
**Constraints:**
|
||||
- Each member can have only ONE property per property_type
|
||||
- Properties are deleted when member is deleted (CASCADE)
|
||||
- Property type cannot be deleted if properties exist (RESTRICT)
|
||||
- Each member can have only ONE custom field value per custom field
|
||||
- Custom field values are deleted when member is deleted (CASCADE)
|
||||
- Custom field cannot be deleted if custom field values exist (RESTRICT)
|
||||
|
||||
**Use Cases:**
|
||||
- Custom membership numbers
|
||||
|
|
@ -214,34 +233,34 @@ Table properties {
|
|||
'''
|
||||
}
|
||||
|
||||
Table property_types {
|
||||
Table custom_fields {
|
||||
id uuid [pk, not null, default: `gen_random_uuid()`, note: 'Primary identifier']
|
||||
name text [not null, unique, note: 'Property name/identifier (e.g., "membership_number")']
|
||||
name text [not null, unique, note: 'CustomFieldValue name/identifier (e.g., "membership_number")']
|
||||
value_type text [not null, note: 'Data type: string | integer | boolean | date | email']
|
||||
description text [null, note: 'Human-readable description']
|
||||
immutable boolean [not null, default: false, note: 'If true, value cannot be changed after creation']
|
||||
required boolean [not null, default: false, note: 'If true, all members must have this property']
|
||||
required boolean [not null, default: false, note: 'If true, all members must have this custom field']
|
||||
|
||||
indexes {
|
||||
name [unique, name: 'property_types_unique_name_index']
|
||||
name [unique, name: 'custom_fields_unique_name_index']
|
||||
}
|
||||
|
||||
Note: '''
|
||||
**Property Type Definitions**
|
||||
**CustomFieldValue Type Definitions**
|
||||
|
||||
Defines the schema and behavior for custom member properties.
|
||||
Defines the schema and behavior for custom member custom_field_values.
|
||||
|
||||
**Attributes:**
|
||||
- `name`: Unique identifier for the property type
|
||||
- `name`: Unique identifier for the custom field
|
||||
- `value_type`: Enforces data type consistency
|
||||
- `description`: Documentation for users/admins
|
||||
- `immutable`: Prevents changes after initial creation (e.g., membership numbers)
|
||||
- `required`: Enforces that all members must have this property
|
||||
- `required`: Enforces that all members must have this custom field
|
||||
|
||||
**Constraints:**
|
||||
- `value_type` must be one of: string, integer, boolean, date, email
|
||||
- `name` must be unique across all property types
|
||||
- Cannot be deleted if properties reference it (ON DELETE RESTRICT)
|
||||
- `name` must be unique across all custom fields
|
||||
- Cannot be deleted if custom_field_values reference it (ON DELETE RESTRICT)
|
||||
|
||||
**Examples:**
|
||||
- Membership Number (string, immutable, required)
|
||||
|
|
@ -264,25 +283,25 @@ Table property_types {
|
|||
Ref: users.member_id - members.id [delete: set null]
|
||||
|
||||
// Member → Properties (1:N)
|
||||
// - One member can have multiple properties
|
||||
// - Each property belongs to exactly one member
|
||||
// - One member can have multiple custom_field_values
|
||||
// - Each custom field value belongs to exactly one member
|
||||
// - ON DELETE CASCADE: Properties deleted when member deleted
|
||||
// - UNIQUE constraint: One property per type per member
|
||||
Ref: properties.member_id > members.id [delete: cascade]
|
||||
// - UNIQUE constraint: One custom field value per custom field per member
|
||||
Ref: custom_field_values.member_id > members.id [delete: cascade]
|
||||
|
||||
// Property → PropertyType (N:1)
|
||||
// - Many properties can reference one property type
|
||||
// - Property type defines the schema/behavior
|
||||
// - ON DELETE RESTRICT: Cannot delete type if properties exist
|
||||
Ref: properties.property_type_id > property_types.id [delete: restrict]
|
||||
// CustomFieldValue → CustomField (N:1)
|
||||
// - Many custom_field_values can reference one custom field
|
||||
// - CustomFieldValue type defines the schema/behavior
|
||||
// - ON DELETE RESTRICT: Cannot delete type if custom_field_values exist
|
||||
Ref: custom_field_values.custom_field_id > custom_fields.id [delete: restrict]
|
||||
|
||||
// ============================================
|
||||
// ENUMS
|
||||
// ============================================
|
||||
|
||||
// Valid data types for property values
|
||||
// Determines how Property.value is interpreted
|
||||
Enum property_value_type {
|
||||
// Valid data types for custom field values
|
||||
// Determines how CustomFieldValue.value is interpreted
|
||||
Enum custom_field_value_type {
|
||||
string [note: 'Text data']
|
||||
integer [note: 'Numeric data']
|
||||
boolean [note: 'True/False flags']
|
||||
|
|
@ -316,8 +335,8 @@ TableGroup accounts_domain {
|
|||
|
||||
TableGroup membership_domain {
|
||||
members
|
||||
properties
|
||||
property_types
|
||||
custom_field_values
|
||||
custom_fields
|
||||
|
||||
Note: '''
|
||||
**Membership Domain**
|
||||
|
|
|
|||
|
|
@ -131,11 +131,11 @@ Based on closed PRs from https://git.local-it.org/local-it/mitgliederverwaltung/
|
|||
|
||||
**Sprint 3 - 28.05 - 09.07**
|
||||
- Member CRUD operations
|
||||
- Basic property system
|
||||
- Basic custom field system
|
||||
- Initial UI with Tailwind CSS
|
||||
|
||||
**Sprint 4 - 09.07 - 30.07**
|
||||
- Property types implementation
|
||||
- CustomFieldValue types implementation
|
||||
- Data validation
|
||||
- Error handling improvements
|
||||
|
||||
|
|
@ -154,7 +154,7 @@ Based on closed PRs from https://git.local-it.org/local-it/mitgliederverwaltung/
|
|||
**PR #147:** *Add seed data for members*
|
||||
- Comprehensive seed data
|
||||
- Test users and members
|
||||
- Property type examples
|
||||
- CustomFieldValue type examples
|
||||
|
||||
#### Phase 3: Search & Navigation (Sprint 6)
|
||||
|
||||
|
|
@ -227,6 +227,108 @@ attribute :search_vector, AshPostgres.Tsvector,
|
|||
|
||||
---
|
||||
|
||||
#### Phase 6: Search Enhancement & OIDC Improvements (Sprint 9)
|
||||
|
||||
**Sprint 9 - 01.11 - 13.11 (finalized)**
|
||||
|
||||
**PR #187:** *Implement fuzzy search* (closes #162) 🔍
|
||||
- PostgreSQL `pg_trgm` extension for trigram-based fuzzy search
|
||||
- 6 new GIN trigram indexes on members table:
|
||||
- first_name, last_name, email, city, street, notes
|
||||
- Combined search strategy: Full-text (tsvector) + Trigram similarity
|
||||
- Configurable similarity threshold (default 0.2)
|
||||
- Migration: `20251001141005_add_trigram_to_members.exs`
|
||||
- 443 lines of comprehensive tests
|
||||
|
||||
**Key learnings:**
|
||||
- Trigram indexes significantly improve fuzzy matching
|
||||
- Combined FTS + trigram provides best user experience
|
||||
- word_similarity() better for partial word matching than similarity()
|
||||
- Similarity threshold of 0.2 balances precision and recall
|
||||
|
||||
**Implementation highlights:**
|
||||
```elixir
|
||||
# New Ash action: :search with fuzzy matching
|
||||
read :search do
|
||||
argument :query, :string, allow_nil?: true
|
||||
argument :similarity_threshold, :float, allow_nil?: true
|
||||
# Uses fragment() for pg_trgm operators: %, similarity(), word_similarity()
|
||||
end
|
||||
|
||||
# Public function for LiveView usage
|
||||
def fuzzy_search(query, opts) do
|
||||
Ash.Query.for_read(query, :search, %{query: query_string})
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**PR #192:** *OIDC handling and linking* (closes #171) 🔐
|
||||
- Secure OIDC account linking with password verification
|
||||
- Security fix: Filter OIDC sign-in by `oidc_id` instead of email
|
||||
- New custom error: `PasswordVerificationRequired`
|
||||
- New validation: `OidcEmailCollision` for email conflict detection
|
||||
- New LiveView: `LinkOidcAccountLive` for interactive linking
|
||||
- Automatic linking for passwordless users (no password prompt)
|
||||
- Password verification required for password-protected accounts
|
||||
- Comprehensive security logging for audit trail
|
||||
- Locale persistence via secure cookie (1 year TTL)
|
||||
- Documentation: `docs/oidc-account-linking.md`
|
||||
|
||||
**Security improvements:**
|
||||
- Prevents account takeover via OIDC email matching
|
||||
- Password verification before linking OIDC to password accounts
|
||||
- All linking attempts logged with appropriate severity
|
||||
- CSRF protection on linking forms
|
||||
- Secure cookie flags: `http_only`, `secure`, `same_site: "Lax"`
|
||||
|
||||
**Test coverage:**
|
||||
- 5 new comprehensive test files (1,793 lines total):
|
||||
- `user_authentication_test.exs` (265 lines)
|
||||
- `oidc_e2e_flow_test.exs` (415 lines)
|
||||
- `oidc_email_update_test.exs` (271 lines)
|
||||
- `oidc_password_linking_test.exs` (496 lines)
|
||||
- `oidc_passwordless_linking_test.exs` (210 lines)
|
||||
- Extended `oidc_integration_test.exs` (+136 lines)
|
||||
|
||||
**Key learnings:**
|
||||
- Account linking requires careful security considerations
|
||||
- Passwordless users should be auto-linked (better UX)
|
||||
- Audit logging essential for security-critical operations
|
||||
- Locale persistence improves user experience post-logout
|
||||
|
||||
---
|
||||
|
||||
**PR #193:** *Docs, Code Guidelines and Progress Log* 📚
|
||||
- Complete project documentation suite (5,554 lines)
|
||||
- New documentation files:
|
||||
- `CODE_GUIDELINES.md` (2,578 lines) - Comprehensive development guidelines
|
||||
- `docs/database-schema-readme.md` (392 lines) - Database documentation
|
||||
- `docs/database_schema.dbml` (329 lines) - DBML schema definition
|
||||
- `docs/development-progress-log.md` (1,227 lines) - This file
|
||||
- `docs/feature-roadmap.md` (743 lines) - Feature planning and roadmap
|
||||
- Reduced redundancy in README.md (links to detailed docs)
|
||||
- Cross-referenced documentation for easy navigation
|
||||
|
||||
---
|
||||
|
||||
**PR #201:** *Code documentation and refactoring* 🔧
|
||||
- @moduledoc for ALL modules (51 modules documented)
|
||||
- @doc for all public functions
|
||||
- Enabled Credo `ModuleDoc` check (enforces documentation standards)
|
||||
- Refactored complex functions:
|
||||
- `MemberLive.Index.handle_event/3` - Split sorting logic into smaller functions
|
||||
- `AuthController.handle_auth_failure/2` - Reduced cyclomatic complexity
|
||||
- Documentation coverage: 100% for core modules
|
||||
|
||||
**Key learnings:**
|
||||
- @moduledoc enforcement improves code maintainability
|
||||
- Refactoring complex functions improves readability
|
||||
- Documentation should explain "why" not just "what"
|
||||
- Credo helps maintain consistent code quality
|
||||
|
||||
---
|
||||
|
||||
## Implementation Decisions
|
||||
|
||||
### Architecture Patterns
|
||||
|
|
@ -277,21 +379,21 @@ attribute :search_vector, AshPostgres.Tsvector,
|
|||
|
||||
**Complete documentation:** See [`docs/email-sync.md`](email-sync.md) for decision tree and sync rules.
|
||||
|
||||
#### 4. Property System (EAV Pattern)
|
||||
#### 4. CustomFieldValue System (EAV Pattern)
|
||||
|
||||
**Implementation:** Entity-Attribute-Value pattern with union types
|
||||
|
||||
```elixir
|
||||
# Property Type defines schema
|
||||
defmodule Mv.Membership.PropertyType do
|
||||
# CustomFieldValue Type defines schema
|
||||
defmodule Mv.Membership.CustomField do
|
||||
attribute :name, :string # "Membership Number"
|
||||
attribute :value_type, :atom # :string, :integer, :boolean, :date, :email
|
||||
attribute :immutable, :boolean # Can't change after creation
|
||||
attribute :required, :boolean # All members must have this
|
||||
end
|
||||
|
||||
# Property stores values
|
||||
defmodule Mv.Membership.Property do
|
||||
# CustomFieldValue stores values
|
||||
defmodule Mv.Membership.CustomFieldValue do
|
||||
attribute :value, :union, # Polymorphic value storage
|
||||
constraints: [
|
||||
types: [
|
||||
|
|
@ -303,7 +405,7 @@ defmodule Mv.Membership.Property do
|
|||
]
|
||||
]
|
||||
belongs_to :member
|
||||
belongs_to :property_type
|
||||
belongs_to :custom_field
|
||||
end
|
||||
```
|
||||
|
||||
|
|
@ -311,12 +413,12 @@ end
|
|||
- Clubs need different custom fields
|
||||
- No schema migrations for new fields
|
||||
- Type safety with union types
|
||||
- Centralized property management
|
||||
- Centralized custom field management
|
||||
|
||||
**Constraints:**
|
||||
- One property per type per member (composite unique index)
|
||||
- One custom field value per custom field per member (composite unique index)
|
||||
- Properties deleted with member (CASCADE)
|
||||
- Property types protected if in use (RESTRICT)
|
||||
- CustomFieldValue types protected if in use (RESTRICT)
|
||||
|
||||
#### 5. Authentication Strategy
|
||||
|
||||
|
|
@ -369,9 +471,11 @@ end
|
|||
- ✅ Consistent styling
|
||||
- ✅ Mobile-responsive out of the box
|
||||
|
||||
#### 7. Full-Text Search Implementation
|
||||
#### 7. Search Implementation (Full-Text + Fuzzy)
|
||||
|
||||
**PostgreSQL tsvector + GIN Index**
|
||||
**Two-Tiered Search Strategy:**
|
||||
|
||||
**A) Full-Text Search (tsvector + GIN Index)**
|
||||
|
||||
```sql
|
||||
-- Auto-updating trigger
|
||||
|
|
@ -389,16 +493,40 @@ END
|
|||
$$ LANGUAGE plpgsql;
|
||||
```
|
||||
|
||||
**B) Fuzzy Search (pg_trgm + Trigram GIN Indexes)**
|
||||
|
||||
Added November 2025 (PR #187):
|
||||
|
||||
```elixir
|
||||
# Ash action combining FTS + trigram similarity
|
||||
read :search do
|
||||
argument :query, :string
|
||||
argument :similarity_threshold, :float
|
||||
|
||||
prepare fn query, _ctx ->
|
||||
# 1. Full-text search (tsvector)
|
||||
# 2. Trigram similarity (%, similarity(), word_similarity())
|
||||
# 3. Substring matching (contains, ilike)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**6 Trigram Indexes:**
|
||||
- first_name, last_name, email, city, street, notes
|
||||
- GIN index with `gin_trgm_ops` operator class
|
||||
|
||||
**Reasoning:**
|
||||
- Native PostgreSQL feature (no external service)
|
||||
- Fast with GIN index
|
||||
- Weighted fields (names more important than dates)
|
||||
- Native PostgreSQL features (no external service)
|
||||
- Combined approach handles typos + partial matches
|
||||
- Fast with GIN indexes
|
||||
- Simple lexer (no German stemming initially)
|
||||
- Similarity threshold configurable (default 0.2)
|
||||
|
||||
**Why not Elasticsearch/Meilisearch?**
|
||||
- Overkill for small to mid-sized clubs
|
||||
- Additional infrastructure complexity
|
||||
- PostgreSQL full-text sufficient for 10k+ members
|
||||
- PostgreSQL full-text + fuzzy sufficient for 10k+ members
|
||||
- Better integration with existing stack
|
||||
|
||||
### Deviations from Initial Plans
|
||||
|
||||
|
|
@ -465,12 +593,13 @@ end
|
|||
#### Database Migrations
|
||||
|
||||
**Key migrations in chronological order:**
|
||||
1. `20250528163901_initial_migration.exs` - Core tables (members, properties, property_types)
|
||||
1. `20250528163901_initial_migration.exs` - Core tables (members, custom_field_values, custom_fields)
|
||||
2. `20250617090641_member_fields.exs` - Member attributes expansion
|
||||
3. `20250620110850_add_accounts_domain.exs` - Users & tokens tables
|
||||
4. `20250912085235_AddSearchVectorToMembers.exs` - Full-text search (tsvector + GIN index)
|
||||
5. `20250926164519_member_relation.exs` - User-Member link (optional 1:1)
|
||||
6. `20251016130855_add_constraints_for_user_member_and_property.exs` - Email sync constraints
|
||||
6. `20251001141005_add_trigram_to_members.exs` - Fuzzy search (pg_trgm + 6 GIN trigram indexes)
|
||||
7. `20251016130855_add_constraints_for_user_member_and_property.exs` - Email sync constraints
|
||||
|
||||
**Learning:** Ash's code generation from resources ensures schema always matches code.
|
||||
|
||||
|
|
@ -643,7 +772,7 @@ end
|
|||
- Admin user: `admin@mv.local` / `testpassword`
|
||||
- Sample members: Hans Müller, Greta Schmidt, Friedrich Wagner
|
||||
- Linked accounts: Maria Weber, Thomas Klein
|
||||
- Property types: String, Date, Boolean, Email
|
||||
- CustomFieldValue types: String, Date, Boolean, Email
|
||||
|
||||
**Test Helpers:**
|
||||
```elixir
|
||||
|
|
@ -827,9 +956,9 @@ mix credo --strict
|
|||
mix credo suggest --format=oneline
|
||||
```
|
||||
|
||||
### 8. Property Value Type Mismatch
|
||||
### 8. CustomFieldValue Value Type Mismatch
|
||||
|
||||
**Issue:** Property value doesn't match property_type definition.
|
||||
**Issue:** CustomFieldValue value doesn't match custom_field definition.
|
||||
|
||||
**Error:**
|
||||
```
|
||||
|
|
@ -837,16 +966,16 @@ mix credo suggest --format=oneline
|
|||
```
|
||||
|
||||
**Solution:**
|
||||
Ensure property value matches property_type.value_type:
|
||||
Ensure custom field value matches custom_field.value_type:
|
||||
|
||||
```elixir
|
||||
# Property Type: value_type = :integer
|
||||
property_type = get_property_type("age")
|
||||
# CustomFieldValue Type: value_type = :integer
|
||||
custom_field = get_custom_field("age")
|
||||
|
||||
# Property Value: must be integer union type
|
||||
{:ok, property} = create_property(%{
|
||||
# CustomFieldValue Value: must be integer union type
|
||||
{:ok, custom_field_value} = create_custom_field_value(%{
|
||||
value: %{type: :integer, value: 25}, # Not "25" as string
|
||||
property_type_id: property_type.id
|
||||
custom_field_id: custom_field.id
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -1220,8 +1349,8 @@ This project demonstrates a modern Phoenix application built with:
|
|||
|
||||
---
|
||||
|
||||
**Document Version:** 1.0
|
||||
**Last Updated:** 2025-11-10
|
||||
**Document Version:** 1.1
|
||||
**Last Updated:** 2025-11-13
|
||||
**Maintainer:** Development Team
|
||||
**Status:** Living Document (update as project evolves)
|
||||
|
||||
|
|
|
|||
|
|
@ -26,9 +26,14 @@
|
|||
- ✅ Password-based authentication
|
||||
- ✅ User sessions and tokens
|
||||
- ✅ Basic authentication flows
|
||||
- ✅ **OIDC account linking with password verification** (PR #192, closes #171)
|
||||
- ✅ **Secure OIDC email collision handling** (PR #192)
|
||||
- ✅ **Automatic linking for passwordless users** (PR #192)
|
||||
|
||||
**Closed Issues:**
|
||||
- ✅ [#171](https://git.local-it.org/local-it/mitgliederverwaltung/issues/171) - OIDC handling and linking (closed 2025-11-13)
|
||||
|
||||
**Open Issues:**
|
||||
- [#171](https://git.local-it.org/local-it/mitgliederverwaltung/issues/171) - Ensure correct handling of Password login vs OIDC login (M)
|
||||
- [#146](https://git.local-it.org/local-it/mitgliederverwaltung/issues/146) - Translate "or" in the login screen (Low)
|
||||
- [#144](https://git.local-it.org/local-it/mitgliederverwaltung/issues/144) - Add language switch dropdown to login screen (Low)
|
||||
|
||||
|
|
@ -54,20 +59,24 @@
|
|||
- ✅ Address management
|
||||
- ✅ Membership status tracking
|
||||
- ✅ Full-text search (PostgreSQL tsvector)
|
||||
- ✅ **Fuzzy search with trigram matching** (PR #187, closes #162)
|
||||
- ✅ **Combined FTS + trigram search** (PR #187)
|
||||
- ✅ **6 GIN trigram indexes** for fuzzy matching (PR #187)
|
||||
- ✅ Sorting by basic fields
|
||||
- ✅ User-Member linking (optional 1:1)
|
||||
- ✅ Email synchronization between User and Member
|
||||
|
||||
**Closed Issues:**
|
||||
- ✅ [#162](https://git.local-it.org/local-it/mitgliederverwaltung/issues/162) - Fuzzy and substring search (closed 2025-11-12)
|
||||
|
||||
**Open Issues:**
|
||||
- [#169](https://git.local-it.org/local-it/mitgliederverwaltung/issues/169) - Allow combined creation of Users/Members (M, Low priority)
|
||||
- [#168](https://git.local-it.org/local-it/mitgliederverwaltung/issues/168) - Allow user-member association in edit/create views (M, High priority)
|
||||
- [#165](https://git.local-it.org/local-it/mitgliederverwaltung/issues/165) - Pagination for list of members (S, Low priority)
|
||||
- [#162](https://git.local-it.org/local-it/mitgliederverwaltung/issues/162) - Implement fuzzy and substring search (M, Medium priority)
|
||||
- [#160](https://git.local-it.org/local-it/mitgliederverwaltung/issues/160) - Implement clear icon in searchbar (S, Low priority)
|
||||
- [#154](https://git.local-it.org/local-it/mitgliederverwaltung/issues/154) - Concept advanced search (Low priority, needs refinement)
|
||||
|
||||
**Missing Features:**
|
||||
- ❌ Fuzzy search
|
||||
- ❌ Advanced filters (date ranges, multiple criteria)
|
||||
- ❌ Pagination (currently all members loaded)
|
||||
- ❌ Bulk operations (bulk delete, bulk update)
|
||||
|
|
@ -78,12 +87,12 @@
|
|||
|
||||
---
|
||||
|
||||
#### 3. **Custom Fields (Property System)** 🔧
|
||||
#### 3. **Custom Fields (CustomFieldValue System)** 🔧
|
||||
|
||||
**Current State:**
|
||||
- ✅ Property types (string, integer, boolean, date, email)
|
||||
- ✅ Property type management
|
||||
- ✅ Dynamic property assignment to members
|
||||
- ✅ CustomFieldValue types (string, integer, boolean, date, email)
|
||||
- ✅ CustomFieldValue type management
|
||||
- ✅ Dynamic custom field value assignment to members
|
||||
- ✅ Union type storage (JSONB)
|
||||
|
||||
**Open Issues:**
|
||||
|
|
@ -208,7 +217,7 @@
|
|||
- ❌ Global settings management
|
||||
- ❌ Club/Organization profile
|
||||
- ❌ Email templates configuration
|
||||
- ❌ Property type management UI (user-facing)
|
||||
- ❌ CustomFieldValue type management UI (user-facing)
|
||||
- ❌ Role and permission management UI
|
||||
- ❌ System health dashboard
|
||||
- ❌ Audit log viewer
|
||||
|
|
@ -367,8 +376,8 @@
|
|||
|
||||
| Feature Area | Current Status | Priority | Complexity |
|
||||
|--------------|----------------|----------|------------|
|
||||
| **Authentication & Authorization** | 40% complete | **High** | Medium |
|
||||
| **Member Management** | 70% complete | **High** | Low-Medium |
|
||||
| **Authentication & Authorization** | 60% complete | **High** | Medium |
|
||||
| **Member Management** | 85% complete | **High** | Low-Medium |
|
||||
| **Custom Fields** | 50% complete | **High** | Medium |
|
||||
| **User Management** | 60% complete | Medium | Low |
|
||||
| **Navigation & UX** | 50% complete | Medium | Low |
|
||||
|
|
@ -388,12 +397,12 @@
|
|||
### Open Milestones (From Issues)
|
||||
|
||||
1. ✅ **Ich kann einen neuen Kontakt anlegen** (Closed)
|
||||
2. 🔄 **I can search through the list of members - fulltext** (Open) - Related: #162, #154
|
||||
2. ✅ **I can search through the list of members - fulltext** (Closed) - #162 implemented (Fuzzy Search), #154 needs refinement
|
||||
3. 🔄 **I can sort the list of members for specific fields** (Open) - Related: #153
|
||||
4. 🔄 **We have a intuitive navigation structure** (Open)
|
||||
5. 🔄 **We have different roles and permissions** (Open) - Related: #191, #190, #151
|
||||
6. 🔄 **As Admin I can configure settings globally** (Open)
|
||||
7. 🔄 **Accounts & Logins** (Open) - Related: #171, #169, #168
|
||||
7. ✅ **Accounts & Logins** (Partially closed) - #171 implemented (OIDC linking), #169/#168 still open
|
||||
8. 🔄 **I can add custom fields** (Open) - Related: #194, #157, #161
|
||||
9. 🔄 **Import transactions via vereinfacht API** (Open) - Related: #156
|
||||
10. 🔄 **We have a staging environment** (Open)
|
||||
|
|
@ -472,9 +481,9 @@ Since this is a **Phoenix LiveView** application with **Ash Framework**, we have
|
|||
| Mount | Purpose | Auth | Query Params | Events |
|
||||
|-------|---------|------|--------------|--------|
|
||||
| `/members` | Member list with search/sort | 🔐 | `?search=&sort_by=&sort_dir=` | `search`, `sort`, `delete`, `select` |
|
||||
| `/members/new` | Create new member form | 🔐 | - | `save`, `cancel`, `add_property` |
|
||||
| `/members/new` | Create new member form | 🔐 | - | `save`, `cancel`, `add_custom_field_value` |
|
||||
| `/members/:id` | Member detail view | 🔐 | - | `edit`, `delete`, `link_user` |
|
||||
| `/members/:id/edit` | Edit member form | 🔐 | - | `save`, `cancel`, `add_property`, `remove_property` |
|
||||
| `/members/:id/edit` | Edit member form | 🔐 | - | `save`, `cancel`, `add_custom_field_value`, `remove_custom_field_value` |
|
||||
|
||||
#### LiveView Event Handlers
|
||||
|
||||
|
|
@ -486,8 +495,8 @@ Since this is a **Phoenix LiveView** application with **Ash Framework**, we have
|
|||
| `save` | Create/update member | `%{"member" => attrs}` | Redirect or show errors |
|
||||
| `link_user` | Link user to member | `%{"user_id" => id}` | Update member view |
|
||||
| `unlink_user` | Unlink user from member | - | Update member view |
|
||||
| `add_property` | Add custom property | `%{"property_type_id" => id, "value" => val}` | Update form |
|
||||
| `remove_property` | Remove custom property | `%{"property_id" => id}` | Update form |
|
||||
| `add_custom_field_value` | Add custom field value | `%{"custom_field_id" => id, "value" => val}` | Update form |
|
||||
| `remove_custom_field_value` | Remove custom field value | `%{"custom_field_value_id" => id}` | Update form |
|
||||
|
||||
#### Ash Resource Actions
|
||||
|
||||
|
|
@ -508,7 +517,7 @@ Since this is a **Phoenix LiveView** application with **Ash Framework**, we have
|
|||
| `Member` | `:fuzzy_search` | Fuzzy text search | 🔐 | `{query, threshold}` | `[%Member{}]` |
|
||||
| `Member` | `:advanced_search` | Multi-criteria search | 🔐 | `{filters: [{field, op, value}]}` | `[%Member{}]` |
|
||||
| `Member` | `:paginate` | Paginated member list | 🔐 | `{page, per_page, filters}` | `{members, total, page_info}` |
|
||||
| `Member` | `:sort_by_custom_field` | Sort by property | 🔐 | `{property_type_id, direction}` | `[%Member{}]` |
|
||||
| `Member` | `:sort_by_custom_field` | Sort by custom field | 🔐 | `{custom_field_id, direction}` | `[%Member{}]` |
|
||||
| `Member` | `:bulk_delete` | Delete multiple members | 🛡️ | `{ids: [id1, id2, ...]}` | `{:ok, count}` |
|
||||
| `Member` | `:bulk_update` | Update multiple members | 🛡️ | `{ids, attrs}` | `{:ok, count}` |
|
||||
| `Member` | `:export` | Export to CSV/Excel | 🔐 | `{format, filters}` | File download |
|
||||
|
|
@ -516,37 +525,37 @@ Since this is a **Phoenix LiveView** application with **Ash Framework**, we have
|
|||
|
||||
---
|
||||
|
||||
### 3. Custom Fields (Property System) Endpoints
|
||||
### 3. Custom Fields (CustomFieldValue System) Endpoints
|
||||
|
||||
#### LiveView Endpoints
|
||||
|
||||
| Mount | Purpose | Auth | Events |
|
||||
|-------|---------|------|--------|
|
||||
| `/property-types` | List property types | 🛡️ | `new`, `edit`, `delete` |
|
||||
| `/property-types/new` | Create property type | 🛡️ | `save`, `cancel` |
|
||||
| `/property-types/:id/edit` | Edit property type | 🛡️ | `save`, `cancel`, `delete` |
|
||||
| `/custom-fields` | List custom fields | 🛡️ | `new`, `edit`, `delete` |
|
||||
| `/custom-fields/new` | Create custom field | 🛡️ | `save`, `cancel` |
|
||||
| `/custom-fields/:id/edit` | Edit custom field | 🛡️ | `save`, `cancel`, `delete` |
|
||||
|
||||
#### Ash Resource Actions
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `PropertyType` | `:create` | Create property type | 🛡️ | `{name, value_type, description, ...}` | `{:ok, property_type}` |
|
||||
| `PropertyType` | `:read` | List property types | 🔐 | - | `[%PropertyType{}]` |
|
||||
| `PropertyType` | `:update` | Update property type | 🛡️ | `{id, attrs}` | `{:ok, property_type}` |
|
||||
| `PropertyType` | `:destroy` | Delete property type | 🛡️ | `{id}` | `{:ok, property_type}` |
|
||||
| `Property` | `:create` | Add property to member | 🔐 | `{member_id, property_type_id, value}` | `{:ok, property}` |
|
||||
| `Property` | `:update` | Update property value | 🔐 | `{id, value}` | `{:ok, property}` |
|
||||
| `Property` | `:destroy` | Remove property | 🔐 | `{id}` | `{:ok, property}` |
|
||||
| `CustomField` | `:create` | Create custom field | 🛡️ | `{name, value_type, description, ...}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:read` | List custom fields | 🔐 | - | `[%CustomField{}]` |
|
||||
| `CustomField` | `:update` | Update custom field | 🛡️ | `{id, attrs}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:destroy` | Delete custom field | 🛡️ | `{id}` | `{:ok, custom_field}` |
|
||||
| `CustomFieldValue` | `:create` | Add custom field value to member | 🔐 | `{member_id, custom_field_id, value}` | `{:ok, custom_field_value}` |
|
||||
| `CustomFieldValue` | `:update` | Update custom field value | 🔐 | `{id, value}` | `{:ok, custom_field_value}` |
|
||||
| `CustomFieldValue` | `:destroy` | Remove custom field value | 🔐 | `{id}` | `{:ok, custom_field_value}` |
|
||||
|
||||
#### **NEW: Enhanced Custom Fields** (Issue #194, #157, #161, #153)
|
||||
|
||||
| Resource | Action | Purpose | Auth | Input | Output |
|
||||
|----------|--------|---------|------|-------|--------|
|
||||
| `PropertyType` | `:set_default_visibility` | Show/hide by default | 🛡️ | `{id, visible}` | `{:ok, property_type}` |
|
||||
| `PropertyType` | `:set_required` | Mark as required | 🛡️ | `{id, required}` | `{:ok, property_type}` |
|
||||
| `PropertyType` | `:add_validation` | Add validation rule | 🛡️ | `{id, rule_type, params}` | `{:ok, property_type}` |
|
||||
| `PropertyType` | `:create_group` | Create field group | 🛡️ | `{name, property_type_ids}` | `{:ok, group}` |
|
||||
| `Property` | `:validate_value` | Validate property value | 🔐 | `{property_type_id, value}` | `{:ok, valid}` or `{:error, reason}` |
|
||||
| `CustomField` | `:set_default_visibility` | Show/hide by default | 🛡️ | `{id, visible}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:set_required` | Mark as required | 🛡️ | `{id, required}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:add_validation` | Add validation rule | 🛡️ | `{id, rule_type, params}` | `{:ok, custom_field}` |
|
||||
| `CustomField` | `:create_group` | Create field group | 🛡️ | `{name, custom_field_ids}` | `{:ok, group}` |
|
||||
| `CustomFieldValue` | `:validate_value` | Validate custom field value | 🔐 | `{custom_field_id, value}` | `{:ok, valid}` or `{:error, reason}` |
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
207
docs/oidc-account-linking.md
Normal file
207
docs/oidc-account-linking.md
Normal file
|
|
@ -0,0 +1,207 @@
|
|||
# OIDC Account Linking Implementation
|
||||
|
||||
## Overview
|
||||
|
||||
This feature implements secure account linking between password-based accounts and OIDC authentication. When a user attempts to log in via OIDC with an email that already exists as a password-only account, the system requires password verification before linking the accounts.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Key Components
|
||||
|
||||
#### 1. Security Fix: `lib/accounts/user.ex`
|
||||
|
||||
**Change**: The `sign_in_with_rauthy` action now filters by `oidc_id` instead of `email`.
|
||||
|
||||
```elixir
|
||||
read :sign_in_with_rauthy do
|
||||
argument :user_info, :map, allow_nil?: false
|
||||
argument :oauth_tokens, :map, allow_nil?: false
|
||||
prepare AshAuthentication.Strategy.OAuth2.SignInPreparation
|
||||
# SECURITY: Filter by oidc_id, NOT by email!
|
||||
filter expr(oidc_id == get_path(^arg(:user_info), [:sub]))
|
||||
end
|
||||
```
|
||||
|
||||
**Why**: Prevents OIDC users from bypassing password authentication and taking over existing accounts.
|
||||
|
||||
#### 2. Custom Error: `lib/accounts/user/errors/password_verification_required.ex`
|
||||
|
||||
Custom error raised when OIDC login conflicts with existing password account.
|
||||
|
||||
**Fields**:
|
||||
|
||||
- `user_id`: ID of the existing user
|
||||
- `oidc_user_info`: OIDC user information for account linking
|
||||
|
||||
#### 3. Validation: `lib/accounts/user/validations/oidc_email_collision.ex`
|
||||
|
||||
Validates email uniqueness during OIDC registration.
|
||||
|
||||
**Scenarios**:
|
||||
|
||||
1. **User exists with matching `oidc_id`**: Allow (upsert)
|
||||
2. **User exists without `oidc_id`** (password-protected OR passwordless): Raise `PasswordVerificationRequired`
|
||||
- The `LinkOidcAccountLive` will auto-link passwordless users without password prompt
|
||||
- Password-protected users must verify their password
|
||||
3. **User exists with different `oidc_id`**: Hard error (cannot link multiple OIDC providers)
|
||||
4. **No user exists**: Allow (new user creation)
|
||||
|
||||
#### 4. Account Linking Action: `lib/accounts/user.ex`
|
||||
|
||||
```elixir
|
||||
update :link_oidc_id do
|
||||
description "Links an OIDC ID to an existing user after password verification"
|
||||
accept []
|
||||
argument :oidc_id, :string, allow_nil?: false
|
||||
argument :oidc_user_info, :map, allow_nil?: false
|
||||
# ... implementation
|
||||
end
|
||||
```
|
||||
|
||||
**Features**:
|
||||
|
||||
- Links `oidc_id` to existing user
|
||||
- Updates email if it differs from OIDC provider
|
||||
- Syncs email changes to linked member
|
||||
|
||||
#### 5. Controller: `lib/mv_web/controllers/auth_controller.ex`
|
||||
|
||||
Refactored for better complexity and maintainability.
|
||||
|
||||
**Key improvements**:
|
||||
|
||||
- Reduced cyclomatic complexity from 11 to below 9
|
||||
- Better separation of concerns with helper functions
|
||||
- Comprehensive documentation
|
||||
|
||||
**Flow**:
|
||||
|
||||
1. Detects `PasswordVerificationRequired` error
|
||||
2. Stores OIDC info in session
|
||||
3. Redirects to account linking page
|
||||
|
||||
#### 6. LiveView: `lib/mv_web/live/auth/link_oidc_account_live.ex`
|
||||
|
||||
Interactive UI for password verification and account linking.
|
||||
|
||||
**Flow**:
|
||||
|
||||
1. Retrieves OIDC info from session
|
||||
2. **Auto-links passwordless users** immediately (no password prompt)
|
||||
3. Displays password verification form for password-protected users
|
||||
4. Verifies password using AshAuthentication
|
||||
5. Links OIDC account on success
|
||||
6. Redirects to complete OIDC login
|
||||
7. **Logs all security-relevant events** (successful/failed linking attempts)
|
||||
|
||||
### Locale Persistence
|
||||
|
||||
**Problem**: Locale was lost on logout (session cleared).
|
||||
|
||||
**Solution**: Store locale in persistent cookie (1 year TTL) with security flags.
|
||||
|
||||
**Changes**:
|
||||
|
||||
- `lib/mv_web/locale_controller.ex`: Sets locale cookie with `http_only` and `secure` flags
|
||||
- `lib/mv_web/router.ex`: Reads locale from cookie if session empty
|
||||
|
||||
**Security Features**:
|
||||
- `http_only: true` - Cookie not accessible via JavaScript (XSS protection)
|
||||
- `secure: true` - Cookie only transmitted over HTTPS in production
|
||||
- `same_site: "Lax"` - CSRF protection
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### 1. OIDC ID Matching
|
||||
|
||||
- **Before**: Matched by email (vulnerable to account takeover)
|
||||
- **After**: Matched by `oidc_id` (secure)
|
||||
|
||||
### 2. Account Linking Flow
|
||||
|
||||
- Password verification required before linking (for password-protected users)
|
||||
- Passwordless users are auto-linked immediately (secure, as they have no password)
|
||||
- OIDC info stored in session (not in URL/query params)
|
||||
- CSRF protection on all forms
|
||||
- All linking attempts logged for audit trail
|
||||
|
||||
### 3. Email Updates
|
||||
|
||||
- Email updates from OIDC provider are applied during linking
|
||||
- Email changes sync to linked member (if exists)
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- Internal errors are logged but not exposed to users (prevents information disclosure)
|
||||
- User-friendly error messages shown in UI
|
||||
- Security-relevant events logged with appropriate levels:
|
||||
- `Logger.info` for successful operations
|
||||
- `Logger.warning` for failed authentication attempts
|
||||
- `Logger.error` for system errors
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Scenario 1: New OIDC User
|
||||
|
||||
```elixir
|
||||
# User signs in with OIDC for the first time
|
||||
# → New user created with oidc_id
|
||||
```
|
||||
|
||||
### Scenario 2: Existing OIDC User
|
||||
|
||||
```elixir
|
||||
# User with oidc_id signs in via OIDC
|
||||
# → Matched by oidc_id, email updated if changed
|
||||
```
|
||||
|
||||
### Scenario 3: Password User + OIDC Login
|
||||
|
||||
```elixir
|
||||
# User with password account tries OIDC login
|
||||
# → PasswordVerificationRequired raised
|
||||
# → Redirected to /auth/link-oidc-account
|
||||
# → User enters password
|
||||
# → Password verified and logged
|
||||
# → oidc_id linked to account
|
||||
# → Successful linking logged
|
||||
# → Redirected to complete OIDC login
|
||||
```
|
||||
|
||||
### Scenario 4: Passwordless User + OIDC Login
|
||||
|
||||
```elixir
|
||||
# User without password (invited user) tries OIDC login
|
||||
# → PasswordVerificationRequired raised
|
||||
# → Redirected to /auth/link-oidc-account
|
||||
# → System detects passwordless user
|
||||
# → oidc_id automatically linked (no password prompt)
|
||||
# → Auto-linking logged
|
||||
# → Redirected to complete OIDC login
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
### Custom Actions
|
||||
|
||||
#### `link_oidc_id`
|
||||
|
||||
Links an OIDC ID to existing user after password verification.
|
||||
|
||||
**Arguments**:
|
||||
|
||||
- `oidc_id` (required): OIDC sub/id from provider
|
||||
- `oidc_user_info` (required): Full OIDC user info map
|
||||
|
||||
**Returns**: Updated user with linked `oidc_id`
|
||||
|
||||
**Side Effects**:
|
||||
|
||||
- Updates email if different from OIDC provider
|
||||
- Syncs email to linked member (if exists)
|
||||
|
||||
## References
|
||||
|
||||
- [AshAuthentication Documentation](https://hexdocs.pm/ash_authentication)
|
||||
- [OIDC Specification](https://openid.net/specs/openid-connect-core-1_0.html)
|
||||
- [Security Best Practices for Account Linking](https://cheatsheetseries.owasp.org/cheatsheets/Credential_Stuffing_Prevention_Cheat_Sheet.html)
|
||||
Loading…
Add table
Add a link
Reference in a new issue