local-it/mitgliederverwaltung
local-it/mitgliederverwaltung
2
3
Fork 0
Code Issues 57 Pull requests 3 Projects 2 Releases 5 Packages 1 Activity Actions
mitgliederverwaltung/docs/development-progress-log.md

12 KiB
Raw Blame History

Development Progress Log

Project: Mila — Membership Management System (AGPLv3)
Status: Early Development (⚠️ Not Production Ready)

A coarse history of what was built and, more importantly, why. Per-feature status lives in feature-roadmap.md; setup/workflow/conventions live in README.md and CODE_GUIDELINES.md. This file keeps the chronological narrative, the architecture decisions and deviations, the migration index, and the hard-won gotchas.

Chronological narrative

Sprint dates and per-feature ✅/open/missing status are in feature-roadmap.md; this is the coarse arc.

  • Sprint 0–2 (foundation): mix phx.new mv --no-ecto --no-mailer (Ash instead of Ecto; Swoosh added later). Stack pinned via asdf (.tool-versions): Elixir 1.18.3-otp-27, Erlang 27.3.4, Just 1.46.0. PostgreSQL extensions (see Mv.Repo.installed_extensions/0): ash-functions, citext, pg_trgm. UUIDv7 comes from a custom uuid_generate_v7() SQL function defined in the extensions migration, not from an extension.
  • Sprint 3–5 (core): Member CRUD, EAV custom-field system, Tailwind/DaisyUI UI, custom login + members landing page, seed data.
  • Sprint 6 (search): Full-text search — weighted tsvector (names A, email/notes B, contact C), GIN index, auto-updating trigger, simple lexer (no German stemming yet).
  • Sprint 7 (sorting & links): Sortable table headers (ARIA); optional 1:1 User↔Member link (User belongs_to Member, Member has_one User) as foundation for email sync.
  • Sprint 8 (email sync): Bidirectional User↔Member email sync; User.email is source of truth on linking. Rules + decision tree in email-sync.md.
  • Sprint 9 (search + OIDC): Fuzzy search (pg_trgm, 6 trigram GIN indexes, combined FTS+trigram, threshold 0.2, word_similarity). Secure OIDC account linking with password verification — see oidc-account-linking.md. Documentation suite + 100% @moduledoc coverage with Credo ModuleDoc enforcement.
  • Late 2025 → 2026: Membership-fee system (types, calendar cycles, settings, status tracking); custom-field search + required-field validation; field-visibility settings; bulk email copy; sidebar navigation (DaisyUI drawer, WCAG 2.1 AA); CSV import (chunked, configurable limits, custom fields); groups (many-to-many, search-integrated); RBAC (4 permission sets, database-backed roles, resource policies, page-permission plug, system-actor pattern); onboarding/join requests (issue #308, TDD); statistics page MVP; SMTP configuration. See feature-roadmap.md and the dedicated docs per area.

Architecture Decisions & Deviations

Ash Framework over Ecto contexts

Declarative resources, built-in policy authorization, calculations/aggregates, migration code-gen. Trade-off: steeper learning curve, smaller ecosystem, more opinionated. Original plan was traditional Phoenix + Ecto; switched because Ash delivers more (policies, admin, codegen) with less code. Detailed structure in CODE_GUIDELINES.md.

Domain-Driven structure

Organized by business domain (Accounts, Membership, MembershipFees) rather than technical layer, to keep business logic out of the web layer and scale to future domains.

UUIDv7 for Members only

Members use uuid_v7_primary_key :id (sortable by creation time → better index locality, chronological ordering); Users and other resources keep standard uuid_primary_key :id (v4). The split is deliberate.

No default :create action on User

User defines defaults [:read], declares :destroy as a separate explicit named action, and exposes explicit creates (create_user, register_with_password, register_with_oidc) — never a default :create. Rationale: a default create would bypass the email-sync changes, so it is excluded as a safety measure.

Bidirectional email sync — conditional, not always-on

Users and Members can exist independently; emails must stay in sync only when linked. Rejected alternatives: single shared email table (too restrictive — members without users still need emails); always-sync (needless work for unlinked entities); manual sync (error-prone). Chosen: conditional sync via custom Ash changes + validations — flexible, safe, performant. Source of truth is User.email on linking. Full rules + decision tree in email-sync.md.

Custom fields — EAV with union types

CustomField defines the schema (name, value_type ∈ {string, integer, boolean, date, email}, required, show_in_overview); CustomFieldValue stores the polymorphic value via Ash :union type and belongs_to :member + belongs_to :custom_field. Chosen so clubs can add fields without schema migrations, with type safety. Constraints: one value per (member, custom_field) (composite unique index); values CASCADE-deleted with the member; custom-field types RESTRICT-protected while in use.

Search — native PostgreSQL, not Elasticsearch/Meilisearch

Two tiers: weighted tsvector full-text (auto-updating trigger) + pg_trgm fuzzy (6 trigram GIN indexes, similarity/word_similarity, configurable threshold 0.2). Rejected Elasticsearch/Meilisearch: overkill for small/mid clubs, extra infra, and PostgreSQL FTS+fuzzy is sufficient for 10k+ members with better stack integration. German stemming remains a known gap (simple lexer).

Authentication — multi-strategy

Password (bcrypt) + OIDC (Rauthy, self-hostable). store_all_tokens? true, JWT sessions, token revocation. Clubs pick the method; password is the non-SSO fallback.

Deployment

Multi-stage Docker (Debian builder → slim runtime), assets via mix assets.deploy, Mix release, migrations via Mv.Release.migrate. Bandit over Cowboy (LiveView performance). Renovate runs the first week of each month (grouped mix/asdf/postgres); Elixir/Erlang auto-updates are disabled — OTP coupling and dependency compatibility require manual asdf control.

Migration Index (26, chronological)

Timestamp → intent. Ash generates migrations from resource snapshots, so order matters and they must not be skipped.

  1. 20250421101957_initialize_extensions_1 — PostgreSQL extensions (ash-functions, citext, pg_trgm) plus the custom uuid_generate_v7() SQL function (source of UUIDv7; not an extension)
  2. 20250528163901_initial_migration — core tables (members, custom_field_values, custom_fields — originally property_types/properties)
  3. 20250617090641_member_fields — member attributes expansion
  4. 20250617132424_member_delete — member deletion constraints
  5. 20250620110849_add_accounts_domain_extensions — accounts domain extensions
  6. 20250620110850_add_accounts_domain — users & tokens tables
  7. 20250912085235_AddSearchVectorToMembers — full-text search (tsvector + GIN index)
  8. 20250926164519_member_relation — User–Member link (optional 1:1)
  9. 20250926180341_add_unique_email_to_members — unique email constraint on members
  10. 20251001141005_add_trigram_to_members — fuzzy search (pg_trgm + 6 GIN trigram indexes)
  11. 20251016130855_add_constraints_for_user_member_and_property — email-sync constraints
  12. 20251113163600_rename_properties_to_custom_fields_extensions_1 — rename properties extensions
  13. 20251113163602_rename_properties_to_custom_fields — rename property_types → custom_fields, properties → custom_field_values
  14. 20251113180429_add_slug_to_custom_fields — add slug to custom fields
  15. 20251113183538_change_custom_field_delete_cascade — change delete cascade behavior
  16. 20251119160509_add_show_in_overview_to_custom_fields — add show_in_overview flag
  17. 20251127134451_add_settings_table — create settings table (singleton)
  18. 20251201115939_add_member_field_visibility_to_settings — add member_field_visibility JSONB to settings
  19. 20251202145404_remove_birth_date_from_members — remove birth_date field
  20. 20251204123714_add_custom_field_values_to_search_vector — include custom field values in search vector
  21. 20251211151449_add_membership_fees_tables — create membership_fee_types and membership_fee_cycles tables
  22. 20251211172549_remove_immutable_from_custom_fields — remove immutable flag from custom fields
  23. 20251211195058_add_membership_fee_settings — add membership fee settings to settings table
  24. 20251218113900_remove_paid_from_members — remove paid boolean from members (replaced by cycle status)
  25. 20260102155350_remove_phone_number_and_make_fields_optional — remove phone_number; make first_name/last_name optional
  26. 20260106161215_add_authorization_domain — create roles table and add role_id to users

(Later migrations exist for join requests 20260309141437_add_join_requests, join-form settings 20260310114701, and groups 20260127141620_add_groups_and_member_groups — added after this index was first cut; extend as needed.)

Gotchas & Hard-won Lessons

Ash manage_relationship validation (drove a real bug fix)

During validation (while manage_relationship is processing), the linked record lives in changeset.relationships, not changeset.attributes — member_id is still nil until the action completes:

# during validation:
changeset.relationships.member = [{[%{id: "uuid"}], opts}]
changeset.attributes.member_id = nil  # still nil!
# after action:
changeset.attributes.member_id = "uuid"

So a validation needing the linked id must read both sources:

defp get_member_id_from_changeset(changeset) do
  case Map.get(changeset.relationships, :member) do
    [{[%{id: id}], _opts}] -> id                               # new link
    _ -> Ash.Changeset.get_attribute(changeset, :member_id)    # existing
  end
end

This fixed email-validation false positives when linking a user and member that share an email.

test_helper.exs clears Vereinfacht/OIDC ENV at startup

test/test_helper.exs clears Vereinfacht- and OIDC-related environment variables on boot. Mv.Config prefers ENV over database settings; without clearing, a developer's loaded .env would make OIDC sign-in redirect tests depend on the shell and go flaky (and risk hitting real APIs). Tests that need specific OIDC ENV set them in setup and restore via on_exit.

LiveView + JavaScript: when to reach for JS

push_event/3 → window.addEventListener("phx:...") is the idiomatic escape hatch. Use JS only for: direct DOM manipulation (autocomplete input values — LiveView DOM patching races on rapid state changes), browser APIs (clipboard), third-party libs, and preventing browser defaults (form submit on Enter). Do not use JS for form submissions, show/hide, server data fetching, or keyboard-navigation logic. Keyboard nav was implemented server-side (phx-window-keydown, ~45 lines) with a minimal hook (~13 lines) only to preventDefault() on Enter — ~20–50 ms roundtrip is imperceptible (perception threshold ~100 ms), versus ~80 lines for a full client-side solution. Don't prematurely optimize for latency.

Common gotchas

  1. Ash actions are mandatory — never use Ecto directly on Ash resources; always Ash.create/Ash.update/etc.
  2. Email sync only for linked entities — cross-table email validation kicks in only on linking; unlinked users/members are not checked.
  3. Migrations run in order — they depend on resource snapshots; don't skip.
  4. LiveView assigns are immutable — return {:noreply, assign(socket, ...)}; never mutate socket.assigns.x.
  5. Reset test DB after schema changes — MIX_ENV=test mix ash.reset.
  6. Docker networks — dev uses network_mode: host for Rauthy; prod should use proper Docker networks.
  7. Secrets in runtime.exs, not config.exs — config.exs is compile-time; runtime.exs reads env vars at runtime.

Bulk email copy (#230)

Format First Last <email> with semicolon separator (compatible with all major email clients). Button shows the count of visible selected members (respects search/filter), not total selected. Server formats via push_event/3; client handles clipboard with an API + older-browser fallback.