diff --git a/docs/csv-member-import-v1.md b/docs/csv-member-import-v1.md index ed5618b..1a717c6 100644 --- a/docs/csv-member-import-v1.md +++ b/docs/csv-member-import-v1.md @@ -48,7 +48,7 @@ A **basic CSV member import feature** that allows administrators to upload a CSV - Upload CSV file via LiveView file upload - Parse CSV with bilingual header support for core member fields (English/German) - Auto-detect delimiter (`;` or `,`) using header recognition -- Map CSV columns to core member fields (`first_name`, `last_name`, `email`, `street`, `postal_code`, `city`) +- Map CSV columns to core member fields (`first_name`, `last_name`, `email`, `street`, `postal_code`, `city`, `country`) - **Import custom field values** - Map CSV columns to existing custom fields by name (unknown custom field columns will be ignored with a warning) - Validate each row (required field: `email`) - Create members via Ash resource (one-by-one, **no background jobs**, processed in chunks of 200 rows via LiveView messages) @@ -149,13 +149,26 @@ A **basic CSV member import feature** that allows administrators to upload a CSV **v1 Supported Fields:** -**Core Member Fields:** +**Core Member Fields (all importable):** +- `email` / `E-Mail` (required) - `first_name` / `Vorname` (optional) - `last_name` / `Nachname` (optional) -- `email` / `E-Mail` (required) -- `street` / `Straße` (optional) -- `postal_code` / `PLZ` / `Postleitzahl` (optional) +- `join_date` / `Beitrittsdatum` (optional, ISO-8601 date) +- `exit_date` / `Austrittsdatum` (optional, ISO-8601 date) +- `notes` / `Notizen` (optional) +- `country` / `Land` / `Staat` (optional) - `city` / `Stadt` (optional) +- `street` / `Straße` (optional) +- `house_number` / `Hausnummer` / `Nr.` (optional) +- `postal_code` / `PLZ` / `Postleitzahl` (optional) +- `membership_fee_start_date` / `Beitragsbeginn` (optional, ISO-8601 date) + +Address column order in import/export matches the members overview: country, city, street, house number, postal code. + +**Not supported for import (by design):** +- **membership_fee_status** – Computed field (from fee cycles). Not stored; export-only. +- **groups** – Many-to-many relationship. Would require resolving group names to IDs; not in current scope. +- **membership_fee_type_id** – Foreign key; could be added later (e.g. resolve type name to ID). **Custom Fields:** - Any custom field column using the custom field's **name** as the header (e.g., `membership_number`, `birth_date`) @@ -176,9 +189,15 @@ A **basic CSV member import feature** that allows administrators to upload a CSV | `first_name` | `first_name`, `firstname` | `Vorname`, `vorname` | | `last_name` | `last_name`, `lastname`, `surname` | `Nachname`, `nachname`, `Familienname` | | `email` | `email`, `e-mail`, `e_mail` | `E-Mail`, `e-mail`, `e_mail` | +| `join_date` | `join date`, `join_date` | `Beitrittsdatum`, `beitritts-datum` | +| `exit_date` | `exit date`, `exit_date` | `Austrittsdatum`, `austritts-datum` | +| `notes` | `notes` | `Notizen`, `bemerkungen` | | `street` | `street`, `address` | `Straße`, `strasse`, `Strasse` | +| `house_number` | `house number`, `house_number`, `house no` | `Hausnummer`, `Nr`, `Nr.`, `Nummer` | | `postal_code` | `postal_code`, `zip`, `postcode` | `PLZ`, `plz`, `Postleitzahl`, `postleitzahl` | | `city` | `city`, `town` | `Stadt`, `stadt`, `Ort` | +| `country` | `country` | `Land`, `land`, `Staat`, `staat` | +| `membership_fee_start_date` | `membership fee start date`, `membership_fee_start_date`, `fee start` | `Beitragsbeginn`, `beitrags-beginn` | **Header Normalization (used consistently for both input headers AND mapping variants):** - Trim whitespace diff --git a/docs/database-schema-readme.md b/docs/database-schema-readme.md index 6e444a5..f58cbea 100644 --- a/docs/database-schema-readme.md +++ b/docs/database-schema-readme.md @@ -191,7 +191,8 @@ Settings (1) → MembershipFeeType (0..1) - Join date cannot be in future - Exit date must be after join date - Phone: `+?[0-9\- ]{6,20}` -- Postal code: 5 digits +- Postal code: optional (no format validation) +- Country: optional ### CustomFieldValue System - Maximum one custom field value per custom field per member @@ -240,7 +241,7 @@ Settings (1) → MembershipFeeType (0..1) ### Weighted Fields - **Weight A (highest):** first_name, last_name - **Weight B:** email, notes, group names (from member_groups → groups) -- **Weight C:** city, street, house_number, postal_code, custom_field_values +- **Weight C:** city, street, house_number, postal_code, country, custom_field_values - **Weight D (lowest):** join_date, exit_date ### Group Names in Search diff --git a/docs/database_schema.dbml b/docs/database_schema.dbml index 23605bf..61da063 100644 --- a/docs/database_schema.dbml +++ b/docs/database_schema.dbml @@ -131,6 +131,7 @@ Table members { street text [null, note: 'Street name'] house_number text [null, note: 'House number'] postal_code text [null, note: '5-digit German postal code'] + country text [null, note: 'Country of residence'] search_vector tsvector [null, note: 'Full-text search index (auto-generated)'] membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - assigned fee type'] membership_fee_start_date date [null, note: 'Date from which membership fees should be calculated'] @@ -188,7 +189,8 @@ Table members { - email: 5-254 characters, valid email format (required) - join_date: cannot be in future - exit_date: must be after join_date (if both present) - - postal_code: exactly 5 digits (if present) + - postal_code: optional (no format validation) + - country: optional ''' } diff --git a/lib/membership/member.ex b/lib/membership/member.ex index 3af514e..8f24595 100644 --- a/lib/membership/member.ex +++ b/lib/membership/member.ex @@ -22,7 +22,6 @@ defmodule Mv.Membership.Member do ## Validations - Required: email (all other fields are optional) - Email format validation (using EctoCommons.EmailValidator) - - Postal code format: exactly 5 digits (German format) - Date validations: join_date not in future, exit_date after join_date - Email uniqueness: prevents conflicts with unlinked users - Linked member email change: only admins or the linked user may change a linked member's email (see `Mv.Membership.Member.Validations.EmailChangePermission`) @@ -477,11 +476,6 @@ defmodule Mv.Membership.Member do where: [present([:join_date, :exit_date])], message: "cannot be before join date" - # Postal code format (only if set) - validate match(:postal_code, ~r/^\d{5}$/), - where: [present(:postal_code)], - message: "must consist of 5 digits" - # Email validation with EctoCommons.EmailValidator validate fn changeset, _ -> email = Ash.Changeset.get_attribute(changeset, :email) @@ -648,6 +642,10 @@ defmodule Mv.Membership.Member do allow_nil? true end + attribute :country, :string do + allow_nil? true + end + attribute :search_vector, AshPostgres.Tsvector, writable?: false, public?: false, @@ -1249,7 +1247,8 @@ defmodule Mv.Membership.Member do contains(postal_code, ^query) or contains(house_number, ^query) or contains(email, ^query) or - contains(city, ^query) + contains(city, ^query) or + contains(country, ^query) ) end diff --git a/lib/mv/constants.ex b/lib/mv/constants.ex index de429e8..3a01fa9 100644 --- a/lib/mv/constants.ex +++ b/lib/mv/constants.ex @@ -10,6 +10,7 @@ defmodule Mv.Constants do :join_date, :exit_date, :notes, + :country, :city, :street, :house_number, diff --git a/lib/mv/membership/import/header_mapper.ex b/lib/mv/membership/import/header_mapper.ex index 709e156..d96d96e 100644 --- a/lib/mv/membership/import/header_mapper.ex +++ b/lib/mv/membership/import/header_mapper.ex @@ -17,15 +17,24 @@ defmodule Mv.Membership.Import.HeaderMapper do ## Member Field Mapping - Maps CSV headers to canonical member fields: - - `email` (required) - - `first_name` (optional) - - `last_name` (optional) - - `street` (optional) - - `postal_code` (optional) - - `city` (optional) + Maps CSV headers to canonical member fields (same as `Mv.Constants.member_fields()` for + importable attributes). All DB-backed member attributes can be imported. - Supports both English and German variants (e.g., "Email" / "E-Mail", "First Name" / "Vorname"). + - `email` (required) + - `first_name`, `last_name` (optional) + - `join_date`, `exit_date` (optional, ISO-8601 date) + - `notes` (optional) + - `country`, `city`, `street`, `house_number`, `postal_code` (optional) + - `membership_fee_start_date` (optional, ISO-8601 date) + + Supports English and German header variants (e.g. "Email" / "E-Mail", "Join Date" / "Beitrittsdatum"). + + ## Fields not supported for import + + - **membership_fee_status** – Computed (calculation from membership fee cycles). Not stored; + cannot be set via CSV. Export can include it. + - **groups** – Many-to-many relationship (through member_groups). Import would require + resolving group names/slugs to IDs and creating associations; not in current import scope. ## Custom Field Detection @@ -75,11 +84,37 @@ defmodule Mv.Membership.Import.HeaderMapper do "nachname", "familienname" ], + join_date: [ + "join date", + "join_date", + "beitrittsdatum", + "beitritts-datum" + ], + exit_date: [ + "exit date", + "exit_date", + "austrittsdatum", + "austritts-datum" + ], + notes: [ + "notes", + "notizen", + "bemerkungen" + ], street: [ "street", "address", "strasse" ], + house_number: [ + "house number", + "house_number", + "house no", + "hausnummer", + "nr", + "nr.", + "nummer" + ], postal_code: [ "postal code", "postal_code", @@ -93,6 +128,18 @@ defmodule Mv.Membership.Import.HeaderMapper do "town", "stadt", "ort" + ], + country: [ + "country", + "land", + "staat" + ], + membership_fee_start_date: [ + "membership fee start date", + "membership_fee_start_date", + "fee start", + "beitragsbeginn", + "beitrags-beginn" ] } diff --git a/lib/mv/membership/import/member_csv.ex b/lib/mv/membership/import/member_csv.ex index c967bf5..23e0d93 100644 --- a/lib/mv/membership/import/member_csv.ex +++ b/lib/mv/membership/import/member_csv.ex @@ -549,9 +549,12 @@ defmodule Mv.Membership.Import.MemberCSV do line_number, actor ) do + # Convert empty strings to nil for date fields so Ash accepts them + member_attrs = sanitize_date_fields(trimmed_member_attrs) + # Create member with custom field values member_attrs_with_cf = - trimmed_member_attrs + member_attrs |> Map.put(:custom_field_values, custom_field_values) # Only include custom_field_values if not empty @@ -793,6 +796,23 @@ defmodule Mv.Membership.Import.MemberCSV do end) end + # Converts empty strings to nil for date fields so Ash can accept them + @date_fields [:join_date, :exit_date, :membership_fee_start_date] + + defp sanitize_date_fields(attrs) when is_map(attrs) do + Enum.reduce(@date_fields, attrs, fn field, acc -> + put_date_field(acc, field, Map.get(acc, field)) + end) + end + + defp put_date_field(acc, field, ""), do: Map.put(acc, field, nil) + + defp put_date_field(acc, field, val) when is_binary(val) do + if String.trim(val) == "", do: Map.put(acc, field, nil), else: acc + end + + defp put_date_field(acc, _field, _), do: acc + # Formats Ash errors into MemberCSV.Error structs defp format_ash_error(%Ash.Error.Invalid{errors: errors}, line_number, email) do # Try to find email-related errors first (for better error messages) diff --git a/lib/mv_web/live/import_live.ex b/lib/mv_web/live/import_live.ex index e97ecd7..4e172ed 100644 --- a/lib/mv_web/live/import_live.ex +++ b/lib/mv_web/live/import_live.ex @@ -92,14 +92,22 @@ defmodule MvWeb.ImportLive do <%= if Authorization.can?(@current_user, :create, Mv.Membership.Member) do %> <%!-- CSV Import Section --%> - <.form_section title={gettext("Import Members (CSV)")}> - - - - <%= if @import_status == :running or @import_status == :done or @import_status == :error do %> - - <% end %> - +
+ <.header> + {gettext("Import Members")} + <:subtitle> + {gettext("Import members from CSV files.")} + + + <.form_section title={gettext("Choose CSV file")}> + + + + <%= if @import_status == :running or @import_status == :done or @import_status == :error do %> + + <% end %> + +
<% else %>