local-it/mitgliederverwaltung

Fork 0

Moritz 47f18e9ef3

continuous-integration/drone/push Build is passing

Details

docs: update the docs

2025-11-13 16:56:41 +01:00

37 KiB

Raw Blame History

Development Progress Log

Project: Mila - Membership Management System
Repository: https://git.local-it.org/local-it/mitgliederverwaltung
License: AGPLv3
Status: Early Development (⚠️ Not Production Ready)

Project Overview
Setup and Foundation
Major Features Implementation
Implementation Decisions
Build and Deployment
Testing Strategy
Common Issues and Solutions
Future Improvements
Team Knowledge Base

Project Overview

Vision

Simple, usable, self-hostable membership management for small to mid-sized clubs.

Philosophy

"Software should help people spend less time on administration and more time on their community."

Core Principles

✅ Simple: Focused on essential club needs
✅ Usable: Clean, accessible UI for everyday volunteers
✅ Flexible: Customizable data fields, role-based permissions
✅ Open: 100% free and open source, no vendor lock-in
✅ Self-hostable: Full control over data and deployment

Target Users

Small to mid-sized clubs
Volunteer administrators (non-technical)
Club members (self-service access)

Setup and Foundation

Initial Project Setup

For current setup instructions, see README.md.

Historical context:

1. Phoenix Project Initialization (Sprint 0)

mix phx.new mv --no-ecto --no-mailer

Reasoning:

--no-ecto: Using Ash Framework with AshPostgres instead
--no-mailer: Added Swoosh later for better control

2. Technology Choices

For complete tech stack details, see CODE_GUIDELINES.md.

Key decisions:

Elixir 1.18.3 + OTP 27: Latest stable versions for performance
Ash Framework 3.0: Declarative resource layer, reduces boilerplate
Phoenix LiveView 1.1: Real-time UI without JavaScript complexity
Tailwind CSS 4.0: Utility-first styling with custom build
PostgreSQL 17: Advanced features (full-text search, JSONB, citext)
Bandit: Modern HTTP server, better than Cowboy for LiveView

3. Version Management (asdf)

Tool: asdf 0.16.5 for consistent environments across team

Versions pinned in .tool-versions:

Elixir 1.18.3-otp-27
Erlang 27.3.4
Just 1.43.0

4. Database Setup

PostgreSQL Extensions:

CREATE EXTENSION IF NOT EXISTS "uuid-ossp";  -- UUID generation
CREATE EXTENSION IF NOT EXISTS "citext";      -- Case-insensitive text

Migration Strategy:

mix ash.codegen --name <migration_name>  # Generate from Ash resources
mix ash.migrate                           # Apply migrations

Reasoning: Ash generates migrations from resource definitions, ensuring schema matches code.

5. Development Workflow (Just)

Chose Just over Makefile for:

Better error messages
Cleaner syntax
Cross-platform compatibility

Core commands: See README.md

Major Features Implementation

Sprint History & Key Pull Requests

Based on closed PRs from https://git.local-it.org/local-it/mitgliederverwaltung/pulls?state=closed:

Phase 1: Foundation (Sprint 0-2)

Sprint 0 - Vorarbeit

Initial project setup
Technology stack decisions
Repository structure

Sprint 2 - 07.05 - 28.05

Basic Phoenix setup
Initial database schema
Development environment configuration

Phase 2: Core Features (Sprint 3-5)

Sprint 3 - 28.05 - 09.07

Member CRUD operations
Basic property system
Initial UI with Tailwind CSS

Sprint 4 - 09.07 - 30.07

Property types implementation
Data validation
Error handling improvements

Sprint 5 - 31.07 - 11.09

PR #138: Customize login screen and members as landing page (closes #68, #137)

Custom login UI with DaisyUI
Members page as default landing
Improved navigation flow

PR #139: Added PR and issue templates (closes #129)

GitHub/GitLab issue templates
PR template for consistent reviews
Contribution guidelines

PR #147: Add seed data for members

Comprehensive seed data
Test users and members
Property type examples

Sprint 6 - 11.09 - 02.10

PR #163: Implement full-text search for members (closes #11) 🔍

PostgreSQL full-text search with tsvector
Weighted search fields (names: A, email/notes: B, contact: C)
GIN index for performance
Auto-updating trigger
Migration: 20250912085235_AddSearchVectorToMembers.exs

# Search implementation highlights
attribute :search_vector, AshPostgres.Tsvector,
  writable?: false,
  public?: false,
  select_by_default?: false

Key learnings:

Simple lexer used (no German stemming initially)
Weighted fields improve relevance
GIN index essential for performance

Phase 4: Sorting & User Management (Sprint 7)

Sprint 7 - 02.10 - 23.10

PR #166: Sorting header for members list (closes #152, #175)

Sortable table headers component
Multi-column sorting support
Visual indicators for sort direction
Accessibility improvements (ARIA labels)

PR #172: Create logical link between users and members (closes #164)

Optional 1:1 relationship (0..1 ↔ 0..1)
User belongs_to Member
Member has_one User
Foundation for email sync feature
Migration: 20250926164519_member_relation.exs

PR #148: Fix error when deleting members

Cascade delete handling
Proper foreign key constraints
Error message improvements

PR #173: Link to user data from profile button (closes #170)

Profile navigation improvements
User-member relationship display
Better UX for linked accounts

PR #178: Polish README (closes #158)

Updated documentation
Better onboarding instructions
Screenshots and examples

Phase 5: Email Synchronization (Sprint 8)

Sprint 8 - 23.10 - 13.11

PR #181: Sync email between user and member (closes #167) ✉️

Bidirectional email synchronization between User and Member
User.email as source of truth on linking
Custom Ash changes with conditional execution
Complex validation logic to prevent conflicts
Migration: 20251016130855_add_constraints_for_user_member_and_property.exs

See: docs/email-sync.md for complete sync rules and decision tree.

Phase 6: Search Enhancement & OIDC Improvements (Sprint 9)

Sprint 9 - 01.11 - 13.11 (finalized)

PR #187: Implement fuzzy search (closes #162) 🔍

PostgreSQL pg_trgm extension for trigram-based fuzzy search
6 new GIN trigram indexes on members table:
- first_name, last_name, email, city, street, notes
Combined search strategy: Full-text (tsvector) + Trigram similarity
Configurable similarity threshold (default 0.2)
Migration: 20251001141005_add_trigram_to_members.exs
443 lines of comprehensive tests

Key learnings:

Trigram indexes significantly improve fuzzy matching
Combined FTS + trigram provides best user experience
word_similarity() better for partial word matching than similarity()
Similarity threshold of 0.2 balances precision and recall

Implementation highlights:

# New Ash action: :search with fuzzy matching
read :search do
  argument :query, :string, allow_nil?: true
  argument :similarity_threshold, :float, allow_nil?: true
  # Uses fragment() for pg_trgm operators: %, similarity(), word_similarity()
end

# Public function for LiveView usage
def fuzzy_search(query, opts) do
  Ash.Query.for_read(query, :search, %{query: query_string})
end

PR #192: OIDC handling and linking (closes #171) 🔐

Secure OIDC account linking with password verification
Security fix: Filter OIDC sign-in by oidc_id instead of email
New custom error: PasswordVerificationRequired
New validation: OidcEmailCollision for email conflict detection
New LiveView: LinkOidcAccountLive for interactive linking
Automatic linking for passwordless users (no password prompt)
Password verification required for password-protected accounts
Comprehensive security logging for audit trail
Locale persistence via secure cookie (1 year TTL)
Documentation: docs/oidc-account-linking.md

Security improvements:

Prevents account takeover via OIDC email matching
Password verification before linking OIDC to password accounts
All linking attempts logged with appropriate severity
CSRF protection on linking forms
Secure cookie flags: http_only, secure, same_site: "Lax"

Test coverage:

5 new comprehensive test files (1,793 lines total):
- user_authentication_test.exs (265 lines)
- oidc_e2e_flow_test.exs (415 lines)
- oidc_email_update_test.exs (271 lines)
- oidc_password_linking_test.exs (496 lines)
- oidc_passwordless_linking_test.exs (210 lines)
Extended oidc_integration_test.exs (+136 lines)

Key learnings:

Account linking requires careful security considerations
Passwordless users should be auto-linked (better UX)
Audit logging essential for security-critical operations
Locale persistence improves user experience post-logout

PR #193: Docs, Code Guidelines and Progress Log 📚

Complete project documentation suite (5,554 lines)
New documentation files:
- CODE_GUIDELINES.md (2,578 lines) - Comprehensive development guidelines
- docs/database-schema-readme.md (392 lines) - Database documentation
- docs/database_schema.dbml (329 lines) - DBML schema definition
- docs/development-progress-log.md (1,227 lines) - This file
- docs/feature-roadmap.md (743 lines) - Feature planning and roadmap
Reduced redundancy in README.md (links to detailed docs)
Cross-referenced documentation for easy navigation

PR #201: Code documentation and refactoring 🔧

@moduledoc for ALL modules (51 modules documented)
@doc for all public functions
Enabled Credo ModuleDoc check (enforces documentation standards)
Refactored complex functions:
- MemberLive.Index.handle_event/3 - Split sorting logic into smaller functions
- AuthController.handle_auth_failure/2 - Reduced cyclomatic complexity
Documentation coverage: 100% for core modules

Key learnings:

@moduledoc enforcement improves code maintainability
Refactoring complex functions improves readability
Documentation should explain "why" not just "what"
Credo helps maintain consistent code quality

Implementation Decisions

Architecture Patterns

1. Ash Framework Over Traditional Phoenix

Decision: Use Ash Framework as the primary data layer instead of traditional Ecto contexts.

Reasoning:

Declarative resource definitions reduce boilerplate
Built-in authorization with policies
Type safety with calculations and aggregates
Code generation for migrations

Trade-offs:

Steeper learning curve
Less common in Phoenix community
Newer ecosystem (fewer resources)
More opinionated structure

Outcome:

✅ Faster feature development
✅ Consistent API across resources
⚠️ Requires team training

2. Domain-Driven Design

Decision: Organize by business domains (Accounts, Membership) rather than technical layers.

Reasoning:

Clear separation of concerns
Business logic separate from web layer
Scalable for future domains (payments, communications)

For detailed project structure, see CODE_GUIDELINES.md.

3. Bidirectional Email Sync

Problem: Users and Members can exist independently, but when linked, emails must stay synchronized.

Solution: Custom Ash changes with conditional execution

Why not simpler approaches?

❌ Single email table: Too restrictive (members without users need emails)
❌ Always sync: Performance concerns, unnecessary for unlinked entities
❌ Manual sync: Error-prone, inconsistent
✅ Conditional sync with validations: Flexible, safe, performant

Complete documentation: See docs/email-sync.md for decision tree and sync rules.

4. Property System (EAV Pattern)

Implementation: Entity-Attribute-Value pattern with union types

# Property Type defines schema
defmodule Mv.Membership.PropertyType do
  attribute :name, :string           # "Membership Number"
  attribute :value_type, :atom       # :string, :integer, :boolean, :date, :email
  attribute :immutable, :boolean     # Can't change after creation
  attribute :required, :boolean      # All members must have this
end

# Property stores values
defmodule Mv.Membership.Property do
  attribute :value, :union,          # Polymorphic value storage
    constraints: [
      types: [
        string: [type: :string],
        integer: [type: :integer],
        boolean: [type: :boolean],
        date: [type: :date],
        email: [type: Mv.Membership.Email]
      ]
    ]
  belongs_to :member
  belongs_to :property_type
end

Reasoning:

Clubs need different custom fields
No schema migrations for new fields
Type safety with union types
Centralized property management

Constraints:

One property per type per member (composite unique index)
Properties deleted with member (CASCADE)
Property types protected if in use (RESTRICT)

5. Authentication Strategy

Multi-Strategy Authentication:

authentication do
  strategies do
    # Password-based
    password :password do
      identity_field :email
      hash_provider AshAuthentication.BcryptProvider
    end
    
    # OIDC (Rauthy)
    oidc :rauthy do
      client_id Mv.Secrets
      base_url Mv.Secrets
      client_secret Mv.Secrets
    end
  end
end

Reasoning:

Flexibility: Clubs choose authentication method
Self-hosting: OIDC with Rauthy (open source)
Fallback: Password auth for non-SSO users
Security: bcrypt for password hashing

Token Management:

Store all tokens (store_all_tokens? true)
JWT-based sessions
Token revocation support

6. UI Framework Choice

Tailwind CSS + DaisyUI

Reasoning:

Tailwind: Utility-first, no custom CSS
DaisyUI: Pre-built components, consistent design
Heroicons: Icon library, inline SVG
Phoenix LiveView: Server-rendered, minimal JavaScript

Trade-offs:

Larger HTML (utility classes)
Learning curve for utility-first CSS
✅ Faster development
✅ Consistent styling
✅ Mobile-responsive out of the box

7. Search Implementation (Full-Text + Fuzzy)

Two-Tiered Search Strategy:

A) Full-Text Search (tsvector + GIN Index)

-- Auto-updating trigger
CREATE FUNCTION members_search_vector_trigger() RETURNS trigger AS $$
BEGIN
  NEW.search_vector :=
    setweight(to_tsvector('simple', coalesce(NEW.first_name, '')), 'A') ||
    setweight(to_tsvector('simple', coalesce(NEW.last_name, '')), 'A') ||
    setweight(to_tsvector('simple', coalesce(NEW.email, '')), 'B') ||
    setweight(to_tsvector('simple', coalesce(NEW.notes, '')), 'B') ||
    setweight(to_tsvector('simple', coalesce(NEW.city, '')), 'C') ||
    -- ... more fields
  RETURN NEW;
END
$$ LANGUAGE plpgsql;

B) Fuzzy Search (pg_trgm + Trigram GIN Indexes)

Added November 2025 (PR #187):

# Ash action combining FTS + trigram similarity
read :search do
  argument :query, :string
  argument :similarity_threshold, :float

  prepare fn query, _ctx ->
    # 1. Full-text search (tsvector)
    # 2. Trigram similarity (%, similarity(), word_similarity())
    # 3. Substring matching (contains, ilike)
  end
end

6 Trigram Indexes:

first_name, last_name, email, city, street, notes
GIN index with gin_trgm_ops operator class

Reasoning:

Native PostgreSQL features (no external service)
Combined approach handles typos + partial matches
Fast with GIN indexes
Simple lexer (no German stemming initially)
Similarity threshold configurable (default 0.2)

Why not Elasticsearch/Meilisearch?

Overkill for small to mid-sized clubs
Additional infrastructure complexity
PostgreSQL full-text + fuzzy sufficient for 10k+ members
Better integration with existing stack

Deviations from Initial Plans

1. No Ecto Schemas

Original Plan: Traditional Phoenix with Ecto
Actual: Ash Resources with AshPostgres

Why: Ash provides more features with less code (policies, admin, code generation)

2. Bidirectional Email Sync

Original Plan: Single email, always linked
Actual: Optional link with conditional sync

Why: Members can exist without user accounts (flexibility requirement)

3. UUIDv7 for Members

Original Plan: Standard UUIDv4
Actual: UUIDv7 for members, v4 for others

# Member uses UUIDv7 (sortable by creation time)
uuid_v7_primary_key :id

# Users use standard UUID
uuid_primary_key :id

Why: Better database performance, chronological ordering

4. No Default Create Action for Users

Decision: Intentionally exclude default :create action

actions do
  # Explicitly NO default :create
  defaults [:read, :destroy]
  
  # Use specific create actions instead
  create :create_user
  create :register_with_password
  create :register_with_rauthy
end

Why: Bypass email sync if default create used (safety measure)

Build and Deployment

Development Workflow

For current setup instructions, see README.md.

Key workflow decisions:

Just as task runner: Simplifies common tasks, better than raw mix commands
Docker Compose for services: Consistent environments, easy local OIDC testing
Seed data included: Realistic test data for development

Database Migrations

Key migrations in chronological order:

20250528163901_initial_migration.exs - Core tables (members, properties, property_types)
20250617090641_member_fields.exs - Member attributes expansion
20250620110850_add_accounts_domain.exs - Users & tokens tables
20250912085235_AddSearchVectorToMembers.exs - Full-text search (tsvector + GIN index)
20250926164519_member_relation.exs - User-Member link (optional 1:1)
20251001141005_add_trigram_to_members.exs - Fuzzy search (pg_trgm + 6 GIN trigram indexes)
20251016130855_add_constraints_for_user_member_and_property.exs - Email sync constraints

Learning: Ash's code generation from resources ensures schema always matches code.

Environment Variables & Secrets

Key environment variables:

SECRET_KEY_BASE - Phoenix session encryption
TOKEN_SIGNING_SECRET - JWT token signing
OIDC_CLIENT_SECRET - Rauthy OAuth2 client secret
DATABASE_URL - PostgreSQL connection (production only)

Secret management approach:

Development: .env file (gitignored)
Production: config/runtime.exs reads from environment
Generation: mix phx.gen.secret

For complete setup, see README.md and README.md - Testing SSO.

Testing

Key testing decisions:

Ecto Sandbox: Isolated, concurrent tests
ExUnit: Built-in testing framework (no external dependencies)
Test structure: Mirrors application structure (accounts/, membership/, mv_web/)

Important test patterns:

Email sync edge cases (see test/accounts/email_sync_edge_cases_test.exs)
User-Member relationship tests (see test/accounts/user_member_relationship_test.exs)
LiveView integration tests

For testing guidelines, see CODE_GUIDELINES.md - Testing Standards.

Code Quality

Tools in use:

Credo ~> 1.7: Static code analysis
Sobelow ~> 0.14: Security analysis
mix_audit ~> 2.1: Dependency vulnerability scanning
mix format: Auto-formatting (2-space indentation, 120 char line length)

CI/CD: Drone CI runs linting, formatting checks, tests, and security scans on every push.

Build Status:

For detailed guidelines, see CODE_GUIDELINES.md.

Docker Deployment

Deployment strategy:

Multi-stage build: Builder stage (Debian + Elixir) → Runtime stage (Debian slim)
Assets: Compiled during build with mix assets.deploy
Releases: Mix release for production (smaller image, faster startup)
Migrations: Run via Mv.Release.migrate module

Key decisions:

Bandit instead of Cowboy: Better LiveView performance
Postgres 16 in production: Stable, well-tested
Separate dev/prod compose files: Different needs (dev has Rauthy, Mailcrab)
Release module (Mv.Release): Handles migrations and seeding in production

For complete deployment instructions, see README.md - Production Deployment.

Automated Dependency Updates

Tool: Renovate (via Drone CI)

Configuration: renovate_backend_config.js

Key decisions:

Schedule: First week of each month (reduces PR noise)
Grouping: Mix dependencies, asdf tools, postgres updates grouped
Disabled: Elixir/Erlang auto-updates (manual version management via asdf)

Why disabled for Elixir/Erlang?

OTP version coupling requires careful testing
Version compatibility with dependencies
Manual control preferred for core runtime

For details, see CODE_GUIDELINES.md - Dependency Management.

Testing Strategy

Test Coverage Areas

1. Unit Tests (Domain Logic)

Example: Member Email Validation

defmodule Mv.Membership.MemberTest do
  use Mv.DataCase, async: true

  describe "email validation" do
    test "accepts valid email" do
      assert {:ok, member} = create_member(%{email: "valid@example.com"})
    end
    
    test "rejects invalid email" do
      assert {:error, _} = create_member(%{email: "invalid"})
    end
  end
end

2. Integration Tests (Cross-Domain)

Example: User-Member Relationship

defmodule Mv.Accounts.UserMemberRelationshipTest do
  use Mv.DataCase, async: true

  test "linking user to member syncs emails" do
    {:ok, user} = create_user(%{email: "user@example.com"})
    {:ok, member} = create_member(%{email: "member@example.com"})
    
    # Link user to member
    {:ok, updated_member} = link_user_to_member(user, member)
    
    # Member email should match user email
    assert updated_member.email == "user@example.com"
  end
end

3. LiveView Tests

Example: Member List Sorting

defmodule MvWeb.MemberLive.IndexTest do
  use MvWeb.ConnCase, async: true
  import Phoenix.LiveViewTest

  test "sorting members by last name", %{conn: conn} do
    {:ok, view, _html} = live(conn, ~p"/members")
    
    # Click sort header
    view
    |> element("th[phx-click='sort']")
    |> render_click()
    
    # Verify sorted order in view
    assert has_element?(view, "#member-1")
  end
end

4. Component Tests

Example: Search Bar

defmodule MvWeb.Components.SearchBarTest do
  use MvWeb.ConnCase, async: true
  import Phoenix.LiveViewTest

  test "renders search input" do
    assigns = %{search_query: "", id: "search"}
    
    html = render_component(&search_bar/1, assigns)
    
    assert html =~ "input"
    assert html =~ ~s(type="search")
  end
end

Test Data Management

Seed Data:

Admin user: admin@mv.local / testpassword
Sample members: Hans Müller, Greta Schmidt, Friedrich Wagner
Linked accounts: Maria Weber, Thomas Klein
Property types: String, Date, Boolean, Email

Test Helpers:

# test/support/fixtures.ex
def member_fixture(attrs \\ %{}) do
  default_attrs = %{
    first_name: "Test",
    last_name: "User",
    email: "test#{System.unique_integer()}@example.com"
  }

  {:ok, member} =
    default_attrs
    |> Map.merge(attrs)
    |> Mv.Membership.create_member()

  member
end

Testing best practices applied:

Async by default with Ecto Sandbox
Descriptive test names explaining behavior
Arrange-Act-Assert pattern
One assertion per test
Fixtures for test data setup

For complete guidelines, see CODE_GUIDELINES.md - Testing Standards.

Common Issues and Solutions

1. Email Synchronization Conflicts

Issue: Creating user/member with email that exists in other table (unlinked).

Error:

"Email already used by another (unlinked) member"

Root Cause: Custom validation prevents cross-table email conflicts for linked entities.

Solution:

Link existing entities first
Or use different email
Validation only applies to linked entities

Documentation: docs/email-sync.md

2. Ash Migration Conflicts

Issue: Migrations out of sync with resource definitions.

Symptoms:

Migration fails
Columns don't match resource attributes
Foreign keys missing

Solution:

# Rollback conflicting migrations
mix ash_postgres.rollback -n 1

# Delete migration files
rm priv/repo/migrations/<timestamp>_*.exs
rm priv/resource_snapshots/repo/<resource>_*.json

# Regenerate
mix ash.codegen --name <migration_name>

# Or use Just helper
just regen-migrations <name>

3. OIDC Authentication Not Working

Issue: OIDC login fails with redirect error.

Symptoms:

"Invalid redirect_uri"
"Client not found"

Checklist:

✅ Rauthy running: docker compose ps
✅ Client created in Rauthy admin panel
✅ Redirect URI matches exactly: http://localhost:4000/auth/user/rauthy/callback
✅ OIDC_CLIENT_SECRET in .env
✅ App restarted after .env update

Debug:

# Check Rauthy logs
docker compose logs rauthy

# Check app logs for OIDC errors
mix phx.server

4. Full-Text Search Not Working

Issue: Search returns no results.

Symptoms:

Empty search results
tsvector not updated

Solution:

-- Check if trigger exists
SELECT tgname FROM pg_trigger WHERE tgrelid = 'members'::regclass;

-- Manually update search_vector (if trigger missing)
UPDATE members SET search_vector = 
  setweight(to_tsvector('simple', first_name), 'A') ||
  setweight(to_tsvector('simple', last_name), 'A');

-- Or recreate trigger
psql mv_dev < priv/repo/migrations/20250912085235_AddSearchVectorToMembers.exs

5. Docker Build Fails

Issue: Production Docker build fails.

Common causes:

Mix dependencies compilation errors
Asset compilation fails
Missing environment variables

Solution:

# Clean build cache
docker builder prune

# Build with no cache
docker build --no-cache -t mila:latest .

# Check build logs for specific error
docker build -t mila:latest . 2>&1 | tee build.log

6. Test Failures After Migration

Issue: Tests fail after running new migration.

Symptoms:

column does not exist
relation does not exist

Solution:

# Reset test database
MIX_ENV=test mix ash.reset

# Or manually
MIX_ENV=test mix ecto.drop
MIX_ENV=test mix ash.setup

# Run tests again
mix test

7. Credo/Formatter Conflicts

Issue: CI fails with formatting/style issues.

Solution:

# Format all files
mix format

# Check what would change
mix format --check-formatted --dry-run

# Run Credo
mix credo --strict

# Auto-fix some issues
mix credo suggest --format=oneline

8. Property Value Type Mismatch

Issue: Property value doesn't match property_type definition.

Error:

"Expected type :integer, got :string"

Solution: Ensure property value matches property_type.value_type:

# Property Type: value_type = :integer
property_type = get_property_type("age")

# Property Value: must be integer union type
{:ok, property} = create_property(%{
  value: %{type: :integer, value: 25},  # Not "25" as string
  property_type_id: property_type.id
})

Future Improvements

Planned Features (Roadmap)

Based on open milestones: https://git.local-it.org/local-it/mitgliederverwaltung/pulls

High Priority

Roles & Permissions 🔐
- Admin, Treasurer, Member roles
- Resource-level permissions
- Ash policies for authorization
Payment Tracking 💰
- Payment history
- Fee calculations
- Due dates and reminders
- Import from vereinfacht API
Intuitive Navigation 🧭
- Breadcrumbs
- Better menu structure
- Search in navigation

Medium Priority

Email Communication 📧
- Send emails to members
- Email templates
- Bulk email (with consent)
Member Self-Service 👤
- Members update own data
- Online application
- Profile management
Advanced Filtering 🔍
- Multi-field filters
- Saved filter presets
- Export filtered results
Accessibility Improvements ♿
- WCAG 2.1 AA compliance
- Screen reader optimization
- Keyboard navigation
- High contrast mode

Low Priority

Document Management 📄
- Attach files to members
- Document templates
- Digital signatures
Reporting & Analytics 📊
- Membership statistics
- Payment reports
- Custom reports
Staging Environment 🔧
- Separate staging server
- Automated deployments
- Preview branches

Technical Debt

German Stemming for Search
- Current: Simple lexer
- Needed: German language support in full-text search
- Library: ts_german or Snowball
Performance Optimization
- Add more indexes based on query patterns
- Optimize N+1 queries (use Ash preloading)
- Lazy loading for large datasets
Error Handling Improvements
- Better user-facing error messages
- Error tracking (Sentry integration?)
- Graceful degradation
Test Coverage
- Current: ~70% (estimated)
- Goal: >85%
- Focus: Email sync edge cases, validation logic
Documentation
- User manual
- Admin guide
- API documentation (if needed)
- Video tutorials

Infrastructure Improvements

Monitoring
- Application metrics (Prometheus?)
- Error tracking
- Performance monitoring
Backup Strategy
- Automated database backups
- Point-in-time recovery
- Backup testing
Scalability
- Database connection pooling
- Caching strategy (ETS, Redis?)
- CDN for assets
Security Hardening
- Rate limiting
- CSRF protection (already enabled)
- Security headers
- Regular security audits

Team Knowledge Base

Key Contacts & Resources

Repository: https://git.local-it.org/local-it/mitgliederverwaltung
CI/CD: https://drone.dev.local-it.cloud/local-it/mitgliederverwaltung
Issues: https://git.local-it.org/local-it/mitgliederverwaltung/-/issues
Pull Requests: https://git.local-it.org/local-it/mitgliederverwaltung/pulls

Development Conventions

Commit Messages

Follow conventional commits:

<type>: <subject>

<body>

<footer>

Types:

feat: New feature
fix: Bug fix
docs: Documentation
style: Formatting
refactor: Code refactoring
test: Tests
chore: Maintenance

Example:

feat: add email synchronization for user-member links

Implement bidirectional email sync between users and members.
User.email is source of truth on initial link, then syncs both ways.

Closes #167

Branch Naming

feature/<issue-number>-<short-description>
bugfix/<issue-number>-<short-description>
chore/<description>

Examples:

feature/167-email-sync
bugfix/152-sorting-header
chore/update-dependencies

Pull Request Template

## Description
[Brief description of changes]

## Related Issue
Closes #[issue number]

## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update

## Testing
- [ ] Tests pass locally
- [ ] New tests added
- [ ] Manual testing completed

## Checklist
- [ ] Code follows style guidelines
- [ ] Self-review completed
- [ ] Documentation updated
- [ ] No new warnings

Useful Commands Cheat Sheet

# Development
just run                    # Start everything
just test                   # Run tests
just lint                   # Check code quality
just audit                  # Security checks
just format                 # Format code

# Database
just reset-database         # Reset DB
just seed-database          # Load seeds
just regen-migrations NAME  # Regenerate migrations

# Ash Framework
mix ash.codegen --name NAME          # Generate migration
mix ash.setup                        # Setup database
mix ash.reset                        # Reset database
mix ash_postgres.rollback -n N       # Rollback N migrations

# Testing
mix test                             # All tests
mix test --cover                     # With coverage
mix test test/path/to/test.exs       # Specific file
mix test test/path/to/test.exs:42    # Specific line

# Code Quality
mix format                           # Format all
mix format --check-formatted         # Check format
mix credo                            # Linting
mix credo --strict                   # Strict mode
mix sobelow --config                 # Security scan
mix deps.audit                       # Dependency audit

# Docker
docker compose up -d                 # Start services
docker compose logs -f app           # Follow logs
docker compose down                  # Stop services
docker compose exec app bash         # Shell into container

# Production
docker compose -f docker-compose.prod.yml up -d
docker compose -f docker-compose.prod.yml exec app /app/bin/mv eval "Mv.Release.migrate"

Debugging Tips

1. IEx (Interactive Elixir):

# Start with IEx
iex -S mix phx.server

# In IEx:
iex> import Ecto.Query
iex> alias Mv.Repo
iex> Repo.all(Mv.Membership.Member) |> IO.inspect()

2. LiveView Debugging:

# In LiveView module
def handle_event("some_event", params, socket) do
  IO.inspect(params, label: "DEBUG PARAMS")
  IO.inspect(socket.assigns, label: "DEBUG ASSIGNS")
  # ...
end

3. Query Debugging:

# Enable query logging
config :logger, level: :debug

# In code
import Ecto.Query
Mv.Repo.all(from m in Mv.Membership.Member, where: m.paid == true)
|> IO.inspect(label: "QUERY RESULT")

4. Ash Debugging:

# Enable Ash debug logging
config :ash, :log_level, :debug

# Inspect changeset errors
case Mv.Membership.create_member(attrs) do
  {:error, error} -> IO.inspect(error, label: "ASH ERROR")
  result -> result
end

Common Gotchas

Ash Actions Must Be Defined
- Can't use Ecto directly on Ash resources
- Always use Ash actions: Ash.create, Ash.update, etc.
Email Sync Only for Linked Entities
- Unlinked users/members don't validate cross-table emails
- Validation kicks in only when linking
Migrations Must Be Run in Order
- Ash migrations depend on resource snapshots
- Don't skip migrations
LiveView Assigns Are Immutable
- Must return new socket: {:noreply, assign(socket, key: value)}
- Can't mutate: socket.assigns.key = value ❌
Test Database Must Be Reset After Schema Changes
- MIX_ENV=test mix ash.reset after migrations
Docker Compose Networks
- Dev uses network_mode: host for Rauthy access
- Prod should use proper Docker networks
Secrets in runtime.exs, Not config.exs
- config.exs is compile-time
- runtime.exs is runtime (for env vars)

Learning Resources

Ash Framework:

Official Docs: https://hexdocs.pm/ash/
AshPostgres: https://hexdocs.pm/ash_postgres/
AshAuthentication: https://hexdocs.pm/ash_authentication/

Phoenix Framework:

Phoenix Guides: https://hexdocs.pm/phoenix/overview.html
LiveView Docs: https://hexdocs.pm/phoenix_live_view/

Tailwind CSS:

Tailwind Docs: https://tailwindcss.com/docs
DaisyUI Components: https://daisyui.com/components/

PostgreSQL:

Full-Text Search: https://www.postgresql.org/docs/current/textsearch.html
Constraints: https://www.postgresql.org/docs/current/ddl-constraints.html

Conclusion

This project demonstrates a modern Phoenix application built with:

✅ Ash Framework for declarative resources and policies
✅ Phoenix LiveView for real-time, server-rendered UI
✅ Tailwind CSS + DaisyUI for rapid UI development
✅ PostgreSQL with advanced features (full-text search, UUIDv7)
✅ Multi-strategy authentication (Password + OIDC)
✅ Complex business logic (bidirectional email sync)
✅ Flexible data model (EAV pattern with union types)

Key Achievements:

🎯 8 sprints completed
🚀 82 pull requests merged
✅ Core features implemented (CRUD, search, auth, sync)
📚 Comprehensive documentation
🔒 Security-focused (audits, validations, policies)
🐳 Docker-ready for self-hosting

Next Steps:

Implement roles & permissions
Add payment tracking
Improve accessibility (WCAG 2.1 AA)
Member self-service portal
Email communication features

Document Version: 1.1
Last Updated: 2025-11-13
Maintainer: Development Team
Status: Living Document (update as project evolves)

37 KiB Raw Blame History

Development Progress Log

Table of Contents

Project Overview

Vision

Philosophy

Core Principles

Target Users

Setup and Foundation

Initial Project Setup

1. Phoenix Project Initialization (Sprint 0)

2. Technology Choices

3. Version Management (asdf)

4. Database Setup

5. Development Workflow (Just)

Major Features Implementation

Sprint History & Key Pull Requests

Phase 1: Foundation (Sprint 0-2)

Phase 2: Core Features (Sprint 3-5)

Phase 3: Search & Navigation (Sprint 6)

Phase 4: Sorting & User Management (Sprint 7)

Phase 5: Email Synchronization (Sprint 8)

Phase 6: Search Enhancement & OIDC Improvements (Sprint 9)

Implementation Decisions

Architecture Patterns

1. Ash Framework Over Traditional Phoenix

2. Domain-Driven Design

3. Bidirectional Email Sync

4. Property System (EAV Pattern)

5. Authentication Strategy

6. UI Framework Choice

7. Search Implementation (Full-Text + Fuzzy)

Deviations from Initial Plans

1. No Ecto Schemas

2. Bidirectional Email Sync

3. UUIDv7 for Members

4. No Default Create Action for Users

Build and Deployment

Development Workflow

Database Migrations

Environment Variables & Secrets

Testing

Code Quality

Docker Deployment

Automated Dependency Updates

Testing Strategy

Test Coverage Areas

1. Unit Tests (Domain Logic)

2. Integration Tests (Cross-Domain)

3. LiveView Tests

4. Component Tests

Test Data Management

Common Issues and Solutions

1. Email Synchronization Conflicts

2. Ash Migration Conflicts

3. OIDC Authentication Not Working

4. Full-Text Search Not Working

5. Docker Build Fails

6. Test Failures After Migration

7. Credo/Formatter Conflicts

8. Property Value Type Mismatch

Future Improvements

Planned Features (Roadmap)

High Priority

Medium Priority

Low Priority

Technical Debt

Infrastructure Improvements

Team Knowledge Base

Key Contacts & Resources

Development Conventions

Commit Messages

Branch Naming

Pull Request Template

Useful Commands Cheat Sheet

Debugging Tips

Common Gotchas

Learning Resources

Conclusion

37 KiB

Raw Blame History