Merge pull request 'Refinex CSV import and PDf export closes #299 and #433' (#446) from feat/299_plz into main
All checks were successful
continuous-integration/drone/push Build is passing

Reviewed-on: #446
This commit is contained in:
carla 2026-02-24 16:32:31 +01:00
commit d614ad2219
27 changed files with 926 additions and 131 deletions

View file

@ -48,7 +48,7 @@ A **basic CSV member import feature** that allows administrators to upload a CSV
- Upload CSV file via LiveView file upload - Upload CSV file via LiveView file upload
- Parse CSV with bilingual header support for core member fields (English/German) - Parse CSV with bilingual header support for core member fields (English/German)
- Auto-detect delimiter (`;` or `,`) using header recognition - Auto-detect delimiter (`;` or `,`) using header recognition
- Map CSV columns to core member fields (`first_name`, `last_name`, `email`, `street`, `postal_code`, `city`) - Map CSV columns to core member fields (`first_name`, `last_name`, `email`, `street`, `postal_code`, `city`, `country`)
- **Import custom field values** - Map CSV columns to existing custom fields by name (unknown custom field columns will be ignored with a warning) - **Import custom field values** - Map CSV columns to existing custom fields by name (unknown custom field columns will be ignored with a warning)
- Validate each row (required field: `email`) - Validate each row (required field: `email`)
- Create members via Ash resource (one-by-one, **no background jobs**, processed in chunks of 200 rows via LiveView messages) - Create members via Ash resource (one-by-one, **no background jobs**, processed in chunks of 200 rows via LiveView messages)
@ -149,13 +149,26 @@ A **basic CSV member import feature** that allows administrators to upload a CSV
**v1 Supported Fields:** **v1 Supported Fields:**
**Core Member Fields:** **Core Member Fields (all importable):**
- `email` / `E-Mail` (required)
- `first_name` / `Vorname` (optional) - `first_name` / `Vorname` (optional)
- `last_name` / `Nachname` (optional) - `last_name` / `Nachname` (optional)
- `email` / `E-Mail` (required) - `join_date` / `Beitrittsdatum` (optional, ISO-8601 date)
- `street` / `Straße` (optional) - `exit_date` / `Austrittsdatum` (optional, ISO-8601 date)
- `postal_code` / `PLZ` / `Postleitzahl` (optional) - `notes` / `Notizen` (optional)
- `country` / `Land` / `Staat` (optional)
- `city` / `Stadt` (optional) - `city` / `Stadt` (optional)
- `street` / `Straße` (optional)
- `house_number` / `Hausnummer` / `Nr.` (optional)
- `postal_code` / `PLZ` / `Postleitzahl` (optional)
- `membership_fee_start_date` / `Beitragsbeginn` (optional, ISO-8601 date)
Address column order in import/export matches the members overview: country, city, street, house number, postal code.
**Not supported for import (by design):**
- **membership_fee_status** Computed field (from fee cycles). Not stored; export-only.
- **groups** Many-to-many relationship. Would require resolving group names to IDs; not in current scope.
- **membership_fee_type_id** Foreign key; could be added later (e.g. resolve type name to ID).
**Custom Fields:** **Custom Fields:**
- Any custom field column using the custom field's **name** as the header (e.g., `membership_number`, `birth_date`) - Any custom field column using the custom field's **name** as the header (e.g., `membership_number`, `birth_date`)
@ -176,9 +189,15 @@ A **basic CSV member import feature** that allows administrators to upload a CSV
| `first_name` | `first_name`, `firstname` | `Vorname`, `vorname` | | `first_name` | `first_name`, `firstname` | `Vorname`, `vorname` |
| `last_name` | `last_name`, `lastname`, `surname` | `Nachname`, `nachname`, `Familienname` | | `last_name` | `last_name`, `lastname`, `surname` | `Nachname`, `nachname`, `Familienname` |
| `email` | `email`, `e-mail`, `e_mail` | `E-Mail`, `e-mail`, `e_mail` | | `email` | `email`, `e-mail`, `e_mail` | `E-Mail`, `e-mail`, `e_mail` |
| `join_date` | `join date`, `join_date` | `Beitrittsdatum`, `beitritts-datum` |
| `exit_date` | `exit date`, `exit_date` | `Austrittsdatum`, `austritts-datum` |
| `notes` | `notes` | `Notizen`, `bemerkungen` |
| `street` | `street`, `address` | `Straße`, `strasse`, `Strasse` | | `street` | `street`, `address` | `Straße`, `strasse`, `Strasse` |
| `house_number` | `house number`, `house_number`, `house no` | `Hausnummer`, `Nr`, `Nr.`, `Nummer` |
| `postal_code` | `postal_code`, `zip`, `postcode` | `PLZ`, `plz`, `Postleitzahl`, `postleitzahl` | | `postal_code` | `postal_code`, `zip`, `postcode` | `PLZ`, `plz`, `Postleitzahl`, `postleitzahl` |
| `city` | `city`, `town` | `Stadt`, `stadt`, `Ort` | | `city` | `city`, `town` | `Stadt`, `stadt`, `Ort` |
| `country` | `country` | `Land`, `land`, `Staat`, `staat` |
| `membership_fee_start_date` | `membership fee start date`, `membership_fee_start_date`, `fee start` | `Beitragsbeginn`, `beitrags-beginn` |
**Header Normalization (used consistently for both input headers AND mapping variants):** **Header Normalization (used consistently for both input headers AND mapping variants):**
- Trim whitespace - Trim whitespace

View file

@ -191,7 +191,8 @@ Settings (1) → MembershipFeeType (0..1)
- Join date cannot be in future - Join date cannot be in future
- Exit date must be after join date - Exit date must be after join date
- Phone: `+?[0-9\- ]{6,20}` - Phone: `+?[0-9\- ]{6,20}`
- Postal code: 5 digits - Postal code: optional (no format validation)
- Country: optional
### CustomFieldValue System ### CustomFieldValue System
- Maximum one custom field value per custom field per member - Maximum one custom field value per custom field per member
@ -240,7 +241,7 @@ Settings (1) → MembershipFeeType (0..1)
### Weighted Fields ### Weighted Fields
- **Weight A (highest):** first_name, last_name - **Weight A (highest):** first_name, last_name
- **Weight B:** email, notes, group names (from member_groups → groups) - **Weight B:** email, notes, group names (from member_groups → groups)
- **Weight C:** city, street, house_number, postal_code, custom_field_values - **Weight C:** city, street, house_number, postal_code, country, custom_field_values
- **Weight D (lowest):** join_date, exit_date - **Weight D (lowest):** join_date, exit_date
### Group Names in Search ### Group Names in Search

View file

@ -131,6 +131,7 @@ Table members {
street text [null, note: 'Street name'] street text [null, note: 'Street name']
house_number text [null, note: 'House number'] house_number text [null, note: 'House number']
postal_code text [null, note: '5-digit German postal code'] postal_code text [null, note: '5-digit German postal code']
country text [null, note: 'Country of residence']
search_vector tsvector [null, note: 'Full-text search index (auto-generated)'] search_vector tsvector [null, note: 'Full-text search index (auto-generated)']
membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - assigned fee type'] membership_fee_type_id uuid [null, note: 'FK to membership_fee_types - assigned fee type']
membership_fee_start_date date [null, note: 'Date from which membership fees should be calculated'] membership_fee_start_date date [null, note: 'Date from which membership fees should be calculated']
@ -188,7 +189,8 @@ Table members {
- email: 5-254 characters, valid email format (required) - email: 5-254 characters, valid email format (required)
- join_date: cannot be in future - join_date: cannot be in future
- exit_date: must be after join_date (if both present) - exit_date: must be after join_date (if both present)
- postal_code: exactly 5 digits (if present) - postal_code: optional (no format validation)
- country: optional
''' '''
} }

View file

@ -22,7 +22,6 @@ defmodule Mv.Membership.Member do
## Validations ## Validations
- Required: email (all other fields are optional) - Required: email (all other fields are optional)
- Email format validation (using EctoCommons.EmailValidator) - Email format validation (using EctoCommons.EmailValidator)
- Postal code format: exactly 5 digits (German format)
- Date validations: join_date not in future, exit_date after join_date - Date validations: join_date not in future, exit_date after join_date
- Email uniqueness: prevents conflicts with unlinked users - Email uniqueness: prevents conflicts with unlinked users
- Linked member email change: only admins or the linked user may change a linked member's email (see `Mv.Membership.Member.Validations.EmailChangePermission`) - Linked member email change: only admins or the linked user may change a linked member's email (see `Mv.Membership.Member.Validations.EmailChangePermission`)
@ -477,11 +476,6 @@ defmodule Mv.Membership.Member do
where: [present([:join_date, :exit_date])], where: [present([:join_date, :exit_date])],
message: "cannot be before join date" message: "cannot be before join date"
# Postal code format (only if set)
validate match(:postal_code, ~r/^\d{5}$/),
where: [present(:postal_code)],
message: "must consist of 5 digits"
# Email validation with EctoCommons.EmailValidator # Email validation with EctoCommons.EmailValidator
validate fn changeset, _ -> validate fn changeset, _ ->
email = Ash.Changeset.get_attribute(changeset, :email) email = Ash.Changeset.get_attribute(changeset, :email)
@ -648,6 +642,10 @@ defmodule Mv.Membership.Member do
allow_nil? true allow_nil? true
end end
attribute :country, :string do
allow_nil? true
end
attribute :search_vector, AshPostgres.Tsvector, attribute :search_vector, AshPostgres.Tsvector,
writable?: false, writable?: false,
public?: false, public?: false,
@ -1249,7 +1247,8 @@ defmodule Mv.Membership.Member do
contains(postal_code, ^query) or contains(postal_code, ^query) or
contains(house_number, ^query) or contains(house_number, ^query) or
contains(email, ^query) or contains(email, ^query) or
contains(city, ^query) contains(city, ^query) or
contains(country, ^query)
) )
end end

View file

@ -10,6 +10,7 @@ defmodule Mv.Constants do
:join_date, :join_date,
:exit_date, :exit_date,
:notes, :notes,
:country,
:city, :city,
:street, :street,
:house_number, :house_number,

View file

@ -17,15 +17,24 @@ defmodule Mv.Membership.Import.HeaderMapper do
## Member Field Mapping ## Member Field Mapping
Maps CSV headers to canonical member fields: Maps CSV headers to canonical member fields (same as `Mv.Constants.member_fields()` for
- `email` (required) importable attributes). All DB-backed member attributes can be imported.
- `first_name` (optional)
- `last_name` (optional)
- `street` (optional)
- `postal_code` (optional)
- `city` (optional)
Supports both English and German variants (e.g., "Email" / "E-Mail", "First Name" / "Vorname"). - `email` (required)
- `first_name`, `last_name` (optional)
- `join_date`, `exit_date` (optional, ISO-8601 date)
- `notes` (optional)
- `country`, `city`, `street`, `house_number`, `postal_code` (optional)
- `membership_fee_start_date` (optional, ISO-8601 date)
Supports English and German header variants (e.g. "Email" / "E-Mail", "Join Date" / "Beitrittsdatum").
## Fields not supported for import
- **membership_fee_status** Computed (calculation from membership fee cycles). Not stored;
cannot be set via CSV. Export can include it.
- **groups** Many-to-many relationship (through member_groups). Import would require
resolving group names/slugs to IDs and creating associations; not in current import scope.
## Custom Field Detection ## Custom Field Detection
@ -75,11 +84,37 @@ defmodule Mv.Membership.Import.HeaderMapper do
"nachname", "nachname",
"familienname" "familienname"
], ],
join_date: [
"join date",
"join_date",
"beitrittsdatum",
"beitritts-datum"
],
exit_date: [
"exit date",
"exit_date",
"austrittsdatum",
"austritts-datum"
],
notes: [
"notes",
"notizen",
"bemerkungen"
],
street: [ street: [
"street", "street",
"address", "address",
"strasse" "strasse"
], ],
house_number: [
"house number",
"house_number",
"house no",
"hausnummer",
"nr",
"nr.",
"nummer"
],
postal_code: [ postal_code: [
"postal code", "postal code",
"postal_code", "postal_code",
@ -93,6 +128,18 @@ defmodule Mv.Membership.Import.HeaderMapper do
"town", "town",
"stadt", "stadt",
"ort" "ort"
],
country: [
"country",
"land",
"staat"
],
membership_fee_start_date: [
"membership fee start date",
"membership_fee_start_date",
"fee start",
"beitragsbeginn",
"beitrags-beginn"
] ]
} }

View file

@ -549,9 +549,12 @@ defmodule Mv.Membership.Import.MemberCSV do
line_number, line_number,
actor actor
) do ) do
# Convert empty strings to nil for date fields so Ash accepts them
member_attrs = sanitize_date_fields(trimmed_member_attrs)
# Create member with custom field values # Create member with custom field values
member_attrs_with_cf = member_attrs_with_cf =
trimmed_member_attrs member_attrs
|> Map.put(:custom_field_values, custom_field_values) |> Map.put(:custom_field_values, custom_field_values)
# Only include custom_field_values if not empty # Only include custom_field_values if not empty
@ -793,6 +796,23 @@ defmodule Mv.Membership.Import.MemberCSV do
end) end)
end end
# Converts empty strings to nil for date fields so Ash can accept them
@date_fields [:join_date, :exit_date, :membership_fee_start_date]
defp sanitize_date_fields(attrs) when is_map(attrs) do
Enum.reduce(@date_fields, attrs, fn field, acc ->
put_date_field(acc, field, Map.get(acc, field))
end)
end
defp put_date_field(acc, field, ""), do: Map.put(acc, field, nil)
defp put_date_field(acc, field, val) when is_binary(val) do
if String.trim(val) == "", do: Map.put(acc, field, nil), else: acc
end
defp put_date_field(acc, _field, _), do: acc
# Formats Ash errors into MemberCSV.Error structs # Formats Ash errors into MemberCSV.Error structs
defp format_ash_error(%Ash.Error.Invalid{errors: errors}, line_number, email) do defp format_ash_error(%Ash.Error.Invalid{errors: errors}, line_number, email) do
# Try to find email-related errors first (for better error messages) # Try to find email-related errors first (for better error messages)

View file

@ -92,14 +92,22 @@ defmodule MvWeb.ImportLive do
<Layouts.app flash={@flash} current_user={@current_user} club_name={@club_name}> <Layouts.app flash={@flash} current_user={@current_user} club_name={@club_name}>
<%= if Authorization.can?(@current_user, :create, Mv.Membership.Member) do %> <%= if Authorization.can?(@current_user, :create, Mv.Membership.Member) do %>
<%!-- CSV Import Section --%> <%!-- CSV Import Section --%>
<.form_section title={gettext("Import Members (CSV)")}> <div data-testid="import-page">
<Components.custom_fields_notice {assigns} /> <.header>
<Components.template_links {assigns} /> {gettext("Import Members")}
<Components.import_form {assigns} /> <:subtitle>
<%= if @import_status == :running or @import_status == :done or @import_status == :error do %> {gettext("Import members from CSV files.")}
<Components.import_progress {assigns} /> </:subtitle>
<% end %> </.header>
</.form_section> <.form_section title={gettext("Choose CSV file")}>
<Components.custom_fields_notice {assigns} />
<Components.template_links {assigns} />
<Components.import_form {assigns} />
<%= if @import_status == :running or @import_status == :done or @import_status == :error do %>
<Components.import_progress {assigns} />
<% end %>
</.form_section>
</div>
<% else %> <% else %>
<div role="alert" class="alert alert-error"> <div role="alert" class="alert alert-error">
<.icon name="hero-exclamation-circle" class="size-5" aria-hidden="true" /> <.icon name="hero-exclamation-circle" class="size-5" aria-hidden="true" />

View file

@ -20,12 +20,12 @@ defmodule MvWeb.ImportLive.Components do
""" """
def custom_fields_notice(assigns) do def custom_fields_notice(assigns) do
~H""" ~H"""
<div role="note" class="alert alert-info mb-4"> <div role="note" class="alert alert-info mb-4 w-xl">
<.icon name="hero-information-circle" class="size-5" aria-hidden="true" /> <.icon name="hero-information-circle" class="size-5" aria-hidden="true" />
<div> <div>
<p class="text-sm mb-2"> <p class="text-sm mb-2">
{gettext( {gettext(
"Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, so they must be listed in the list of member data (like e-mail or first name). Unknown data field columns will be ignored with a warning." "Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, because unknown data field columns will be ignored. Groups and membership fees are not supported for import."
)} )}
</p> </p>
<p class="text-sm"> <p class="text-sm">
@ -48,7 +48,7 @@ defmodule MvWeb.ImportLive.Components do
def template_links(assigns) do def template_links(assigns) do
~H""" ~H"""
<div class="mb-4"> <div class="mb-4">
<p class="text-sm text-base-content/70 mb-2"> <p class="mb-2">
{gettext("Download CSV templates:")} {gettext("Download CSV templates:")}
</p> </p>
<ul class="list-disc list-inside space-y-1"> <ul class="list-disc list-inside space-y-1">
@ -88,22 +88,20 @@ defmodule MvWeb.ImportLive.Components do
phx-submit="start_import" phx-submit="start_import"
data-testid="csv-upload-form" data-testid="csv-upload-form"
> >
<div class="form-control"> <fieldset class="mb-2 fieldset w-md">
<label for="csv_file" class="label"> <label for="csv_file">
<span class="label-text"> <span class="mb-1 label">{gettext("CSV File")}</span>
{gettext("CSV File")}
</span>
</label> </label>
<.live_file_input <.live_file_input
upload={@uploads.csv_file} upload={@uploads.csv_file}
id="csv_file" id="csv_file"
class="file-input file-input-bordered w-full" class="file-input file-input-bordered"
aria-describedby="csv_file_help" aria-describedby="csv_file_help"
/> />
<p class="label-text-alt mt-1" id="csv_file_help"> <p class="text-sm text-base-content/60 mt-2" id="csv_file_help">
{gettext("CSV files only, maximum %{size} MB", size: @csv_import_max_file_size_mb)} {gettext("CSV files only, maximum %{size} MB", size: @csv_import_max_file_size_mb)}
</p> </p>
</div> </fieldset>
<.button <.button
type="submit" type="submit"

View file

@ -101,21 +101,10 @@ defmodule MvWeb.MemberLive.Form do
</div> </div>
</div> </div>
<%!-- Address Row --%> <%!-- Address: Country, Postal Code, City in one row --%>
<div class="flex gap-4"> <div class="flex gap-4">
<div class="flex-1"> <div class="w-48">
<.input <.input field={@form[:country]} label={gettext("Country")} />
field={@form[:street]}
label={gettext("Street")}
required={@member_field_required_map[:street]}
/>
</div>
<div class="w-16">
<.input
field={@form[:house_number]}
label={gettext("Nr.")}
required={@member_field_required_map[:house_number]}
/>
</div> </div>
<div class="w-24"> <div class="w-24">
<.input <.input
@ -124,17 +113,23 @@ defmodule MvWeb.MemberLive.Form do
required={@member_field_required_map[:postal_code]} required={@member_field_required_map[:postal_code]}
/> />
</div> </div>
<div class="w-32"> <div class="w-48">
<.input <.input field={@form[:city]} label={gettext("City")} />
field={@form[:city]}
label={gettext("City")}
required={@member_field_required_map[:city]}
/>
</div> </div>
</div> </div>
<%!-- Email (always required) --%> <%!-- Street and Nr. below --%>
<div> <div class="flex gap-4">
<div class="w-64">
<.input field={@form[:street]} label={gettext("Street")} />
</div>
<div class="w-24">
<.input field={@form[:house_number]} label={gettext("Nr.")} />
</div>
</div>
<%!-- Email --%>
<div class="w-64">
<.input field={@form[:email]} label={gettext("Email")} required type="email" /> <.input field={@form[:email]} label={gettext("Email")} required type="email" />
</div> </div>
@ -708,6 +703,7 @@ defmodule MvWeb.MemberLive.Form do
|> extract_form_value(form, :house_number, &to_string/1) |> extract_form_value(form, :house_number, &to_string/1)
|> extract_form_value(form, :postal_code, &to_string/1) |> extract_form_value(form, :postal_code, &to_string/1)
|> extract_form_value(form, :city, &to_string/1) |> extract_form_value(form, :city, &to_string/1)
|> extract_form_value(form, :country, &to_string/1)
|> extract_form_value(form, :join_date, &format_date_value/1) |> extract_form_value(form, :join_date, &format_date_value/1)
|> extract_form_value(form, :exit_date, &format_date_value/1) |> extract_form_value(form, :exit_date, &format_date_value/1)
|> extract_form_value(form, :notes, &to_string/1) |> extract_form_value(form, :notes, &to_string/1)

View file

@ -223,6 +223,24 @@
> >
{member.notes} {member.notes}
</:col> </:col>
<:col
:let={member}
:if={:country in @member_fields_visible}
label={
~H"""
<.live_component
module={MvWeb.Components.SortHeaderComponent}
id={:sort_country}
field={:country}
label={gettext("Country")}
sort_field={@sort_field}
sort_order={@sort_order}
/>
"""
}
>
{member.country}
</:col>
<:col <:col
:let={member} :let={member}
:if={:city in @member_fields_visible} :if={:city in @member_fields_visible}

View file

@ -451,8 +451,8 @@ defmodule MvWeb.MemberLive.Show do
|> Enum.filter(&(&1 && &1 != "")) |> Enum.filter(&(&1 && &1 != ""))
|> Enum.join(" ") |> Enum.join(" ")
[street_part, city_part] [member.country, street_part, city_part]
|> Enum.filter(&(&1 != "")) |> Enum.filter(&(&1 && &1 != ""))
|> Enum.join(", ") |> Enum.join(", ")
|> case do |> case do
"" -> nil "" -> nil

View file

@ -27,6 +27,7 @@ defmodule MvWeb.Translations.MemberFields do
def label(:street), do: gettext("Street") def label(:street), do: gettext("Street")
def label(:house_number), do: gettext("House Number") def label(:house_number), do: gettext("House Number")
def label(:postal_code), do: gettext("Postal Code") def label(:postal_code), do: gettext("Postal Code")
def label(:country), do: gettext("Country")
def label(:membership_fee_start_date), do: gettext("Membership Fee Start Date") def label(:membership_fee_start_date), do: gettext("Membership Fee Start Date")
def label(:membership_fee_status), do: gettext("Membership Fee Status") def label(:membership_fee_status), do: gettext("Membership Fee Status")
def label(:membership_fee_type), do: gettext("Fee Type") def label(:membership_fee_type), do: gettext("Fee Type")

View file

@ -2050,11 +2050,6 @@ msgstr "Fehlgeschlagen: %{count} Zeile(n)"
msgid "German Template" msgid "German Template"
msgstr "Deutsche Vorlage" msgstr "Deutsche Vorlage"
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Import Members (CSV)"
msgstr "Mitglieder importieren (CSV)"
#: lib/mv_web/live/import_live/components.ex #: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Import Results" msgid "Import Results"
@ -2383,11 +2378,6 @@ msgstr "Du hast keine Berechtigung, auf diese Seite zuzugreifen."
msgid "Manage Member Data" msgid "Manage Member Data"
msgstr "Mitgliederdaten verwalten" msgstr "Mitgliederdaten verwalten"
#: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format, fuzzy
msgid "Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, so they must be listed in the list of member data (like e-mail or first name). Unknown data field columns will be ignored with a warning."
msgstr "Verwende die Namen der Datenfelder als Spaltennamen in der CSV Datei. Datenfelder müssen in Mila bereits angelegt sein, damit sie importiert werden können. sie müssen in der Liste der Mitgliederdaten als Datenfeld enthalten sein (z.B. E-Mail). Spalten mit unbekannten Spaltenüberschriften werden mit einer Warnung ignoriert."
#: lib/mv_web/components/export_dropdown.ex #: lib/mv_web/components/export_dropdown.ex
#, elixir-autogen, elixir-format, fuzzy #, elixir-autogen, elixir-format, fuzzy
msgid "Export members to CSV" msgid "Export members to CSV"
@ -2576,7 +2566,7 @@ msgstr "Erstellt am:"
#: lib/mv_web/components/export_dropdown.ex #: lib/mv_web/components/export_dropdown.ex
#, elixir-autogen, elixir-format, fuzzy #, elixir-autogen, elixir-format, fuzzy
msgid "Export" msgid "Export"
msgstr "Nach CSV exportieren" msgstr "Export"
#: lib/mv_web/controllers/member_pdf_export_controller.ex #: lib/mv_web/controllers/member_pdf_export_controller.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
@ -2619,6 +2609,13 @@ msgstr "Import"
msgid "Value type cannot be changed after creation" msgid "Value type cannot be changed after creation"
msgstr "Der Wertetyp kann nach dem Erstellen nicht mehr geändert werden." msgstr "Der Wertetyp kann nach dem Erstellen nicht mehr geändert werden."
#: lib/mv_web/live/member_live/form.ex
#: lib/mv_web/live/member_live/index.html.heex
#: lib/mv_web/translations/member_fields.ex
#, elixir-autogen, elixir-format
msgid "Country"
msgstr "Land"
#: lib/mv_web/live/group_live/show.ex #: lib/mv_web/live/group_live/show.ex
#, elixir-autogen, elixir-format, fuzzy #, elixir-autogen, elixir-format, fuzzy
msgid "Could not load member list. Please try again." msgid "Could not load member list. Please try again."
@ -2978,6 +2975,31 @@ msgstr "Für die Vereinfacht-Integration erforderlich und kann nicht deaktiviert
msgid "Fee Type" msgid "Fee Type"
msgstr "Beitragsart" msgstr "Beitragsart"
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Import members from CSV files."
msgstr "Miglieder aus CSV Dateien importieren."
#: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format, fuzzy
msgid "Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, because unknown data field columns will be ignored. Groups and membership fees are not supported for import."
msgstr "Verwende die Namen der Datenfelder als Spaltennamen in der CSV Datei. Datenfelder müssen in Mila bereits angelegt sein, da unbekannte Spaltennamen ignoriert werden. Gruppen und der Beitragsstatus kann nicht importiert werden."
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Choose CSV file"
msgstr "CSV Datei auswählen"
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format, fuzzy
msgid "Import Members"
msgstr "Mitglieder importieren (CSV)"
#~ #: lib/mv_web/live/import_live.ex
#~ #, elixir-autogen, elixir-format
#~ msgid "Datei auswählen"
#~ msgstr ""
#: lib/mv_web/live/global_settings_live.ex #: lib/mv_web/live/global_settings_live.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Admin group name" msgid "Admin group name"

View file

@ -2051,11 +2051,6 @@ msgstr ""
msgid "German Template" msgid "German Template"
msgstr "" msgstr ""
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Import Members (CSV)"
msgstr ""
#: lib/mv_web/live/import_live/components.ex #: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Import Results" msgid "Import Results"
@ -2384,11 +2379,6 @@ msgstr ""
msgid "Manage Member Data" msgid "Manage Member Data"
msgstr "" msgstr ""
#: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format
msgid "Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, so they must be listed in the list of member data (like e-mail or first name). Unknown data field columns will be ignored with a warning."
msgstr ""
#: lib/mv_web/components/export_dropdown.ex #: lib/mv_web/components/export_dropdown.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Export members to CSV" msgid "Export members to CSV"
@ -2620,6 +2610,13 @@ msgstr ""
msgid "Value type cannot be changed after creation" msgid "Value type cannot be changed after creation"
msgstr "" msgstr ""
#: lib/mv_web/live/member_live/form.ex
#: lib/mv_web/live/member_live/index.html.heex
#: lib/mv_web/translations/member_fields.ex
#, elixir-autogen, elixir-format
msgid "Country"
msgstr ""
#: lib/mv_web/live/group_live/show.ex #: lib/mv_web/live/group_live/show.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Could not load member list. Please try again." msgid "Could not load member list. Please try again."
@ -2978,6 +2975,26 @@ msgstr ""
msgid "Fee Type" msgid "Fee Type"
msgstr "" msgstr ""
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Import members from CSV files."
msgstr ""
#: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format
msgid "Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, because unknown data field columns will be ignored. Groups and membership fees are not supported for import."
msgstr ""
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Choose CSV file"
msgstr ""
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Import Members"
msgstr ""
#: lib/mv_web/live/global_settings_live.ex #: lib/mv_web/live/global_settings_live.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Admin group name" msgid "Admin group name"

View file

@ -2051,11 +2051,6 @@ msgstr ""
msgid "German Template" msgid "German Template"
msgstr "" msgstr ""
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Import Members (CSV)"
msgstr ""
#: lib/mv_web/live/import_live/components.ex #: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Import Results" msgid "Import Results"
@ -2384,11 +2379,6 @@ msgstr ""
msgid "Manage Member Data" msgid "Manage Member Data"
msgstr "" msgstr ""
#: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format, fuzzy
msgid "Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, so they must be listed in the list of member data (like e-mail or first name). Unknown data field columns will be ignored with a warning."
msgstr ""
#: lib/mv_web/components/export_dropdown.ex #: lib/mv_web/components/export_dropdown.ex
#, elixir-autogen, elixir-format, fuzzy #, elixir-autogen, elixir-format, fuzzy
msgid "Export members to CSV" msgid "Export members to CSV"
@ -2620,6 +2610,13 @@ msgstr ""
msgid "Value type cannot be changed after creation" msgid "Value type cannot be changed after creation"
msgstr "" msgstr ""
#: lib/mv_web/live/member_live/form.ex
#: lib/mv_web/live/member_live/index.html.heex
#: lib/mv_web/translations/member_fields.ex
#, elixir-autogen, elixir-format
msgid "Country"
msgstr ""
#: lib/mv_web/live/group_live/show.ex #: lib/mv_web/live/group_live/show.ex
#, elixir-autogen, elixir-format, fuzzy #, elixir-autogen, elixir-format, fuzzy
msgid "Could not load member list. Please try again." msgid "Could not load member list. Please try again."
@ -2978,6 +2975,26 @@ msgstr "Required for Vereinfacht integration and cannot be disabled."
msgid "Fee Type" msgid "Fee Type"
msgstr "Fee Type" msgstr "Fee Type"
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Import members from CSV files."
msgstr ""
#: lib/mv_web/live/import_live/components.ex
#, elixir-autogen, elixir-format, fuzzy
msgid "Use the data field name as the CSV column header in your file. Data fields must exist in Mila before importing, because unknown data field columns will be ignored. Groups and membership fees are not supported for import."
msgstr ""
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format
msgid "Choose CSV file"
msgstr ""
#: lib/mv_web/live/import_live.ex
#, elixir-autogen, elixir-format, fuzzy
msgid "Import Members"
msgstr ""
#: lib/mv_web/live/global_settings_live.ex #: lib/mv_web/live/global_settings_live.ex
#, elixir-autogen, elixir-format #, elixir-autogen, elixir-format
msgid "Admin group name" msgid "Admin group name"

View file

@ -9,7 +9,13 @@
#set page( #set page(
paper: "a4", paper: "a4",
flipped: true, flipped: true,
margin: (top: 1.2cm, bottom: 1.2cm, left: 1.0cm, right: 1.0cm) margin: (top: 1.2cm, bottom: 1.2cm, left: 1.0cm, right: 1.0cm),
numbering: "1",
footer: context [
#set text(size: 8pt)
#set align(center)
#counter(page).display("1 / 1", both: true)
]
) )
#set text(size: 9pt, hyphenate: true) #set text(size: 9pt, hyphenate: true)
@ -58,7 +64,6 @@
#let start = fixed_count + chunk_index * max_dynamic_cols #let start = fixed_count + chunk_index * max_dynamic_cols
#let page_cols = fixed_cols + dyn_cols_chunk #let page_cols = fixed_cols + dyn_cols_chunk
#let headers = page_cols.map(c => c.at("label", default: ""))
// widths: first two columns fixed (32mm, 42mm), rest distributed as 1fr // widths: first two columns fixed (32mm, 42mm), rest distributed as 1fr
#let widths = ( #let widths = (
@ -67,9 +72,9 @@
..((1fr,) * dyn_count) ..((1fr,) * dyn_count)
) )
#let header_cells = headers.map(h => text(weight: "bold", size: 9pt)[#h]) #let header_cells = page_cols.map(c => text(weight: "bold", size: 9pt)[#c.at("label", default: "")])
// Body cells (row-major), nur die Spalten dieses Chunks // Body cells (row-major), only columns of this chunk
#let body_cells = ( #let body_cells = (
rows rows
.map(row => row.slice(0, fixed_count) + row.slice(start, start + dyn_count)) .map(row => row.slice(0, fixed_count) + row.slice(start, start + dyn_count))
@ -77,8 +82,27 @@
.flatten() .flatten()
) )
// Thinner grid for body; thicker vertical line between column 2 and 3; header and outer borders thick
#let thin_stroke = 0.3pt + black
#let thick_sep = 1.5pt + black
#let thick_stroke = 1pt + black
#let last_x = fixed_count + dyn_count - 1
#let last_y = rows.len()
#let stroke_fn = (x, y) => {
let top = if y == 0 or y == 1 { thick_stroke } else { thin_stroke }
let bottom = if y == 0 or y == last_y { thick_stroke } else { thin_stroke }
let left = if x == 0 { thick_stroke } else if x == 2 { thick_sep } else { thin_stroke }
let right = if x == last_x { thick_stroke } else { thin_stroke }
(top: top, bottom: bottom, left: left, right: right)
}
// Light gray background for first two columns (first_name, last_name)
#let fill_fn = (x, y) => if x < 2 { rgb("f2f2f2") } else { none }
#table( #table(
columns: widths, columns: widths,
stroke: stroke_fn,
fill: fill_fn,
table.header(..header_cells), table.header(..header_cells),
..body_cells, ..body_cells,
) )

View file

@ -0,0 +1,577 @@
defmodule Mv.Repo.Migrations.AddCountryToMembers do
@moduledoc """
Adds country as an optional member field and includes it in full-text search.
- Adds :country column to members table (text, nullable)
- Updates members_search_vector_trigger() to include country (weight C)
- Updates update_member_search_vector_from_custom_field_value() to include country
- Updates update_member_search_vector_from_member_groups() to include country
- Backfills existing members' search_vector with country
"""
use Ecto.Migration
def up do
alter table(:members) do
add :country, :text
end
# 1. Main trigger on members: add country to search_vector
execute("""
CREATE OR REPLACE FUNCTION members_search_vector_trigger() RETURNS trigger AS $$
DECLARE
custom_values_text text;
groups_text text;
BEGIN
SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
INTO custom_values_text
FROM custom_field_values
WHERE member_id = NEW.id AND value IS NOT NULL;
SELECT string_agg(g.name, ' ')
INTO groups_text
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = NEW.id;
NEW.search_vector :=
setweight(to_tsvector('simple', coalesce(NEW.first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(NEW.last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(NEW.email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(NEW.join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(NEW.exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(NEW.notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(NEW.city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(NEW.street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(NEW.house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(NEW.postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(NEW.country, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(custom_values_text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(groups_text, '')), 'B');
RETURN NEW;
END
$$ LANGUAGE plpgsql;
""")
# 2. Custom field trigger: include country in recomputed search_vector
execute("""
CREATE OR REPLACE FUNCTION update_member_search_vector_from_custom_field_value() RETURNS trigger AS $$
DECLARE
member_id_val uuid;
member_first_name text;
member_last_name text;
member_email text;
member_join_date date;
member_exit_date date;
member_notes text;
member_city text;
member_street text;
member_house_number text;
member_postal_code text;
member_country text;
custom_values_text text;
groups_text text;
old_value_text text;
new_value_text text;
BEGIN
member_id_val := COALESCE(NEW.member_id, OLD.member_id);
IF TG_OP = 'UPDATE' THEN
old_value_text := COALESCE(
NULLIF(OLD.value->>'_union_value', ''),
NULLIF(OLD.value->>'value', ''),
''
);
new_value_text := COALESCE(
NULLIF(NEW.value->>'_union_value', ''),
NULLIF(NEW.value->>'value', ''),
''
);
IF (old_value_text IS NOT DISTINCT FROM new_value_text) AND
(OLD.member_id IS NOT DISTINCT FROM NEW.member_id) AND
(OLD.custom_field_id IS NOT DISTINCT FROM NEW.custom_field_id) THEN
RETURN COALESCE(NEW, OLD);
END IF;
END IF;
SELECT
first_name,
last_name,
email,
join_date,
exit_date,
notes,
city,
street,
house_number,
postal_code,
country
INTO
member_first_name,
member_last_name,
member_email,
member_join_date,
member_exit_date,
member_notes,
member_city,
member_street,
member_house_number,
member_postal_code,
member_country
FROM members
WHERE id = member_id_val;
SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
INTO custom_values_text
FROM custom_field_values
WHERE member_id = member_id_val AND value IS NOT NULL;
SELECT string_agg(g.name, ' ')
INTO groups_text
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = member_id_val;
UPDATE members
SET search_vector =
setweight(to_tsvector('simple', coalesce(member_first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_country, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(custom_values_text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(groups_text, '')), 'B')
WHERE id = member_id_val;
RETURN COALESCE(NEW, OLD);
END
$$ LANGUAGE plpgsql;
""")
# 3. Member groups trigger: include country when refreshing search_vector
execute("""
CREATE OR REPLACE FUNCTION update_member_search_vector_from_member_groups() RETURNS trigger AS $$
DECLARE
member_id_val uuid;
member_first_name text;
member_last_name text;
member_email text;
member_join_date date;
member_exit_date date;
member_notes text;
member_city text;
member_street text;
member_house_number text;
member_postal_code text;
member_country text;
custom_values_text text;
groups_text text;
BEGIN
FOR member_id_val IN
SELECT COALESCE(NEW.member_id, OLD.member_id)
UNION ALL
SELECT OLD.member_id
WHERE TG_OP = 'UPDATE' AND OLD.member_id IS DISTINCT FROM NEW.member_id
LOOP
SELECT
first_name,
last_name,
email,
join_date,
exit_date,
notes,
city,
street,
house_number,
postal_code,
country
INTO
member_first_name,
member_last_name,
member_email,
member_join_date,
member_exit_date,
member_notes,
member_city,
member_street,
member_house_number,
member_postal_code,
member_country
FROM members
WHERE id = member_id_val;
SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
INTO custom_values_text
FROM custom_field_values
WHERE member_id = member_id_val AND value IS NOT NULL;
SELECT string_agg(g.name, ' ')
INTO groups_text
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = member_id_val;
UPDATE members
SET search_vector =
setweight(to_tsvector('simple', coalesce(member_first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_country, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(custom_values_text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(groups_text, '')), 'B')
WHERE id = member_id_val;
END LOOP;
RETURN COALESCE(NEW, OLD);
END
$$ LANGUAGE plpgsql;
""")
# 4. Backfill: update all members' search_vector to include country
execute("""
UPDATE members m
SET search_vector =
setweight(to_tsvector('simple', coalesce(m.first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(m.last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(m.email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(m.join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(m.exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(m.notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(m.city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(m.street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(m.house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(m.postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(m.country, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(
(SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
FROM custom_field_values
WHERE member_id = m.id AND value IS NOT NULL),
''
)), 'C') ||
setweight(to_tsvector('simple', coalesce(
(SELECT string_agg(g.name, ' ')
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = m.id),
''
)), 'B')
""")
end
def down do
# Restore trigger functions without country (revert to previous version from AddGroupNamesToMemberSearchVector)
execute("""
CREATE OR REPLACE FUNCTION members_search_vector_trigger() RETURNS trigger AS $$
DECLARE
custom_values_text text;
groups_text text;
BEGIN
SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
INTO custom_values_text
FROM custom_field_values
WHERE member_id = NEW.id AND value IS NOT NULL;
SELECT string_agg(g.name, ' ')
INTO groups_text
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = NEW.id;
NEW.search_vector :=
setweight(to_tsvector('simple', coalesce(NEW.first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(NEW.last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(NEW.email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(NEW.join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(NEW.exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(NEW.notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(NEW.city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(NEW.street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(NEW.house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(NEW.postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(custom_values_text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(groups_text, '')), 'B');
RETURN NEW;
END
$$ LANGUAGE plpgsql;
""")
execute("""
CREATE OR REPLACE FUNCTION update_member_search_vector_from_custom_field_value() RETURNS trigger AS $$
DECLARE
member_id_val uuid;
member_first_name text;
member_last_name text;
member_email text;
member_join_date date;
member_exit_date date;
member_notes text;
member_city text;
member_street text;
member_house_number text;
member_postal_code text;
custom_values_text text;
groups_text text;
old_value_text text;
new_value_text text;
BEGIN
member_id_val := COALESCE(NEW.member_id, OLD.member_id);
IF TG_OP = 'UPDATE' THEN
old_value_text := COALESCE(
NULLIF(OLD.value->>'_union_value', ''),
NULLIF(OLD.value->>'value', ''),
''
);
new_value_text := COALESCE(
NULLIF(NEW.value->>'_union_value', ''),
NULLIF(NEW.value->>'value', ''),
''
);
IF (old_value_text IS NOT DISTINCT FROM new_value_text) AND
(OLD.member_id IS NOT DISTINCT FROM NEW.member_id) AND
(OLD.custom_field_id IS NOT DISTINCT FROM NEW.custom_field_id) THEN
RETURN COALESCE(NEW, OLD);
END IF;
END IF;
SELECT
first_name,
last_name,
email,
join_date,
exit_date,
notes,
city,
street,
house_number,
postal_code
INTO
member_first_name,
member_last_name,
member_email,
member_join_date,
member_exit_date,
member_notes,
member_city,
member_street,
member_house_number,
member_postal_code
FROM members
WHERE id = member_id_val;
SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
INTO custom_values_text
FROM custom_field_values
WHERE member_id = member_id_val AND value IS NOT NULL;
SELECT string_agg(g.name, ' ')
INTO groups_text
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = member_id_val;
UPDATE members
SET search_vector =
setweight(to_tsvector('simple', coalesce(member_first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(custom_values_text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(groups_text, '')), 'B')
WHERE id = member_id_val;
RETURN COALESCE(NEW, OLD);
END
$$ LANGUAGE plpgsql;
""")
execute("""
CREATE OR REPLACE FUNCTION update_member_search_vector_from_member_groups() RETURNS trigger AS $$
DECLARE
member_id_val uuid;
member_first_name text;
member_last_name text;
member_email text;
member_join_date date;
member_exit_date date;
member_notes text;
member_city text;
member_street text;
member_house_number text;
member_postal_code text;
custom_values_text text;
groups_text text;
BEGIN
FOR member_id_val IN
SELECT COALESCE(NEW.member_id, OLD.member_id)
UNION ALL
SELECT OLD.member_id
WHERE TG_OP = 'UPDATE' AND OLD.member_id IS DISTINCT FROM NEW.member_id
LOOP
SELECT
first_name,
last_name,
email,
join_date,
exit_date,
notes,
city,
street,
house_number,
postal_code
INTO
member_first_name,
member_last_name,
member_email,
member_join_date,
member_exit_date,
member_notes,
member_city,
member_street,
member_house_number,
member_postal_code
FROM members
WHERE id = member_id_val;
SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
INTO custom_values_text
FROM custom_field_values
WHERE member_id = member_id_val AND value IS NOT NULL;
SELECT string_agg(g.name, ' ')
INTO groups_text
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = member_id_val;
UPDATE members
SET search_vector =
setweight(to_tsvector('simple', coalesce(member_first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(member_email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(member_notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(member_city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(member_postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(custom_values_text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(groups_text, '')), 'B')
WHERE id = member_id_val;
END LOOP;
RETURN COALESCE(NEW, OLD);
END
$$ LANGUAGE plpgsql;
""")
# Backfill without country
execute("""
UPDATE members m
SET search_vector =
setweight(to_tsvector('simple', coalesce(m.first_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(m.last_name, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(m.email, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(m.join_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(m.exit_date::text, '')), 'D') ||
setweight(to_tsvector('simple', coalesce(m.notes, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(m.city, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(m.street, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(m.house_number::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(m.postal_code::text, '')), 'C') ||
setweight(to_tsvector('simple', coalesce(
(SELECT string_agg(
CASE
WHEN value ? '_union_value' THEN value->>'_union_value'
WHEN value ? 'value' THEN value->>'value'
ELSE ''
END,
' '
)
FROM custom_field_values
WHERE member_id = m.id AND value IS NOT NULL),
''
)), 'C') ||
setweight(to_tsvector('simple', coalesce(
(SELECT string_agg(g.name, ' ')
FROM member_groups mg
JOIN groups g ON g.id = mg.group_id
WHERE mg.member_id = m.id),
''
)), 'B')
""")
alter table(:members) do
remove :country
end
end
end

View file

@ -1,2 +1,2 @@
Vorname;Nachname;E-Mail;Straße;PLZ;Stadt Vorname;Nachname;E-Mail;Land;Stadt;Straße;Hausnummer;PLZ;Beitrittsdatum;Austrittsdatum;Notizen;Beitragsbeginn
Max;Mustermann;max.mustermann@example.com;Hauptstraße;10115;Berlin Max;Mustermann;max.mustermann@example.com;Deutschland;Berlin;Hauptstraße;12;10115;2020-01-15;;;

1 Vorname Nachname E-Mail Land Stadt Straße Hausnummer PLZ Beitrittsdatum Austrittsdatum Notizen Beitragsbeginn
2 Max Mustermann max.mustermann@example.com Deutschland Berlin Hauptstraße 12 10115 2020-01-15

View file

@ -1,2 +1,2 @@
first_name;last_name;email;street;postal_code;city first_name;last_name;email;country;city;street;house_number;postal_code;join_date;exit_date;notes;membership_fee_start_date
John;Doe;john.doe@example.com;Main Street;12345;Berlin John;Doe;john.doe@example.com;Germany;Berlin;Main Street;1a;12345;2020-01-15;;;

1 first_name last_name email country city street house_number postal_code join_date exit_date notes membership_fee_start_date
2 John Doe john.doe@example.com Germany Berlin Main Street 1a 12345 2020-01-15

View file

@ -80,15 +80,9 @@ defmodule Mv.Membership.MemberTest do
assert {:ok, _member} = Membership.create_member(attrs, actor: actor) assert {:ok, _member} = Membership.create_member(attrs, actor: actor)
end end
test "Postal code is optional but must have 5 digits if specified", %{actor: actor} do test "Postal code is optional", %{actor: actor} do
attrs = Map.put(@valid_attrs, :postal_code, "1234") attrs = Map.delete(@valid_attrs, :postal_code)
assert {:ok, _member} = Membership.create_member(attrs, actor: actor)
assert {:error, %Ash.Error.Invalid{errors: errors}} =
Membership.create_member(attrs, actor: actor)
assert error_message(errors, :postal_code) =~ "must consist of 5 digits"
attrs2 = Map.delete(@valid_attrs, :postal_code)
assert {:ok, _member} = Membership.create_member(attrs2, actor: actor)
end end
end end

View file

@ -207,9 +207,15 @@ defmodule Mv.Membership.Import.HeaderMapperTest do
"Email", "Email",
"First Name", "First Name",
"Last Name", "Last Name",
"Join Date",
"Exit Date",
"Notes",
"Country",
"City",
"Street", "Street",
"House Number",
"Postal Code", "Postal Code",
"City" "Membership Fee Start Date"
] ]
assert {:ok, %{member: member_map, custom: custom_map, unknown: unknown}} = assert {:ok, %{member: member_map, custom: custom_map, unknown: unknown}} =
@ -218,15 +224,34 @@ defmodule Mv.Membership.Import.HeaderMapperTest do
assert member_map[:email] == 0 assert member_map[:email] == 0
assert member_map[:first_name] == 1 assert member_map[:first_name] == 1
assert member_map[:last_name] == 2 assert member_map[:last_name] == 2
assert member_map[:street] == 3 assert member_map[:join_date] == 3
assert member_map[:postal_code] == 4 assert member_map[:exit_date] == 4
assert member_map[:city] == 5 assert member_map[:notes] == 5
assert member_map[:country] == 6
assert member_map[:city] == 7
assert member_map[:street] == 8
assert member_map[:house_number] == 9
assert member_map[:postal_code] == 10
assert member_map[:membership_fee_start_date] == 11
assert custom_map == %{} assert custom_map == %{}
assert unknown == [] assert unknown == []
end end
test "maps German member field variants" do test "maps German member field variants" do
headers = ["E-Mail", "Vorname", "Nachname", "Straße", "PLZ", "Stadt"] headers = [
"E-Mail",
"Vorname",
"Nachname",
"Beitrittsdatum",
"Austrittsdatum",
"Notizen",
"Land",
"Stadt",
"Straße",
"Hausnummer",
"PLZ",
"Beitragsbeginn"
]
assert {:ok, %{member: member_map, custom: custom_map, unknown: unknown}} = assert {:ok, %{member: member_map, custom: custom_map, unknown: unknown}} =
HeaderMapper.build_maps(headers, []) HeaderMapper.build_maps(headers, [])
@ -234,9 +259,15 @@ defmodule Mv.Membership.Import.HeaderMapperTest do
assert member_map[:email] == 0 assert member_map[:email] == 0
assert member_map[:first_name] == 1 assert member_map[:first_name] == 1
assert member_map[:last_name] == 2 assert member_map[:last_name] == 2
assert member_map[:street] == 3 assert member_map[:join_date] == 3
assert member_map[:postal_code] == 4 assert member_map[:exit_date] == 4
assert member_map[:city] == 5 assert member_map[:notes] == 5
assert member_map[:country] == 6
assert member_map[:city] == 7
assert member_map[:street] == 8
assert member_map[:house_number] == 9
assert member_map[:postal_code] == 10
assert member_map[:membership_fee_start_date] == 11
assert custom_map == %{} assert custom_map == %{}
assert unknown == [] assert unknown == []
end end

View file

@ -24,6 +24,7 @@ defmodule MvWeb.Components.SortHeaderComponentTest do
:house_number, :house_number,
:postal_code, :postal_code,
:city, :city,
:country,
:join_date :join_date
] ]
@ -100,6 +101,7 @@ defmodule MvWeb.Components.SortHeaderComponentTest do
assert has_element?(view, "[data-testid='street'] .opacity-40") assert has_element?(view, "[data-testid='street'] .opacity-40")
assert has_element?(view, "[data-testid='house_number'] .opacity-40") assert has_element?(view, "[data-testid='house_number'] .opacity-40")
assert has_element?(view, "[data-testid='postal_code'] .opacity-40") assert has_element?(view, "[data-testid='postal_code'] .opacity-40")
assert has_element?(view, "[data-testid='country'] .opacity-40")
assert has_element?(view, "[data-testid='join_date'] .opacity-40") assert has_element?(view, "[data-testid='join_date'] .opacity-40")
end end

View file

@ -45,11 +45,11 @@ defmodule MvWeb.GlobalSettingsLiveConfigTest do
{:ok, view, _html} = live(conn, ~p"/admin/import") {:ok, view, _html} = live(conn, ~p"/admin/import")
# Generate CSV with 501 rows (exceeding custom limit of 500) # Generate CSV with 501 rows (exceeding custom limit of 500)
header = "first_name;last_name;email;street;postal_code;city\n" header = "first_name;last_name;email;country;city;street;postal_code\n"
rows = rows =
for i <- 1..501 do for i <- 1..501 do
"Row#{i};Last#{i};email#{i}@example.com;Street#{i};12345;City#{i}\n" "Row#{i};Last#{i};email#{i}@example.com;Country#{i};City#{i};Street#{i};12345\n"
end end
large_csv = header <> Enum.join(rows) large_csv = header <> Enum.join(rows)

View file

@ -136,10 +136,10 @@ defmodule MvWeb.ImportLiveTest do
test "error list is capped and truncation message is shown", %{conn: conn} do test "error list is capped and truncation message is shown", %{conn: conn} do
{:ok, view, _html} = live(conn, ~p"/admin/import") {:ok, view, _html} = live(conn, ~p"/admin/import")
header = "first_name;last_name;email;street;postal_code;city\n" header = "first_name;last_name;email;country;city;street;postal_code\n"
invalid_rows = invalid_rows =
for i <- 1..100, do: "Row#{i};Last#{i};;Street#{i};12345;City#{i}\n" for i <- 1..100, do: "Row#{i};Last#{i};;Country#{i};City#{i};Street#{i};12345\n"
upload_csv_file(view, header <> Enum.join(invalid_rows), "large_invalid.csv") upload_csv_file(view, header <> Enum.join(invalid_rows), "large_invalid.csv")
submit_import(view) submit_import(view)
@ -154,11 +154,11 @@ defmodule MvWeb.ImportLiveTest do
test "row limit is enforced (1001 rows rejected)", %{conn: conn} do test "row limit is enforced (1001 rows rejected)", %{conn: conn} do
{:ok, view, _html} = live(conn, ~p"/admin/import") {:ok, view, _html} = live(conn, ~p"/admin/import")
header = "first_name;last_name;email;street;postal_code;city\n" header = "first_name;last_name;email;country;city;street;postal_code\n"
rows = rows =
for i <- 1..1001 do for i <- 1..1001 do
"Row#{i};Last#{i};email#{i}@example.com;Street#{i};12345;City#{i}\n" "Row#{i};Last#{i};email#{i}@example.com;Country#{i};City#{i};Street#{i};12345\n"
end end
upload_csv_file(view, header <> Enum.join(rows), "too_many_rows.csv") upload_csv_file(view, header <> Enum.join(rows), "too_many_rows.csv")
@ -235,11 +235,10 @@ defmodule MvWeb.ImportLiveTest do
test "page loads and shows import form", %{conn: conn} do test "page loads and shows import form", %{conn: conn} do
{:ok, view, _html} = live(conn, ~p"/admin/import") {:ok, view, _html} = live(conn, ~p"/admin/import")
assert has_element?(view, "[data-testid='import-page']")
assert has_element?(view, "[data-testid='csv-upload-form']") assert has_element?(view, "[data-testid='csv-upload-form']")
assert has_element?(view, "[data-testid='start-import-button']") assert has_element?(view, "[data-testid='start-import-button']")
assert has_element?(view, "[data-testid='custom-fields-link']") assert has_element?(view, "[data-testid='custom-fields-link']")
html = render(view)
assert html =~ "Import Members (CSV)"
end end
test "template links and file input are present", %{conn: conn} do test "template links and file input are present", %{conn: conn} do

View file

@ -16,6 +16,7 @@ defmodule MvWeb.MemberLive.IndexMemberFieldsDisplayTest do
house_number: "123", house_number: "123",
postal_code: "12345", postal_code: "12345",
city: "Berlin", city: "Berlin",
country: "Germany",
join_date: ~D[2020-01-15] join_date: ~D[2020-01-15]
}, },
actor: system_actor actor: system_actor

View file

@ -162,6 +162,7 @@ defmodule MvWeb.MemberLive.IndexTest do
:house_number, :house_number,
:postal_code, :postal_code,
:city, :city,
:country,
:join_date :join_date
] ]