Import/Export consolidation closes #536

## Description of the implemented changes The changes were: - [x] Bugfixing - [x] New Feature - [ ] Breaking Change - [x] Refactoring Import/Export consolidation: the CSV import god module is split into focused modules with narrower exception handling, the three parallel member export pipelines are collapsed into one that loads members through the member overview's own query (so CSV and PDF exports contain exactly the members, in exactly the order, the overview shows), and the fee type interval rule becomes a named, translated member validation. ## What has been changed? - Pin every import error and warning message exactly - Extract custom field value coercion into its own module - Extract import error and warning texts into a formatter - Let unexpected exceptions fail the chunk instead of a row - Rescue only expected failures when reading custom fields and parsing CSV - Describe the split import modules and row failure rules - Validate custom-field emails with the rule used when saving them - Pin the CSV export format and both controllers' error responses - Encode prepared export rows as CSV - Share the export column labels between CSV and PDF - Derive overview query options from plain assigns - Carry the complete overview filter state in the export payload - Accept text boolean filters and the legacy fee status column name - Load export members through the overview query - Export PDF members exactly as the overview lists them - Export CSV members exactly as the overview lists them - Remove the export's own sorting, filtering and member-based CSV path - Enforce the fee type interval rule as a member validation - Translate the fee type interval rule messages - Drop the unreachable client-side fee type interval checks - Locate the fee type interval rule in the member validations - Share the overview's sortable field list with the export - Remove the unused in-memory boolean custom field filter - Derive the label member keys from the export field allowlist - Align moduledocs with the export scope and fee type validation - Rely on golden files and shared generators instead of duplicate examples - Resolve the fee status column at the overview's reference date - Drop substring export tests the golden and scope tests subsume - Record the import and export consolidation under Unreleased ## Definition of Done ### Code Quality - [ ] No new technical depths - [x] Linting passed - [x] Documentation is added were needed ### Accessibility - [ ] New elements are properly defined with html-tags - [ ] Colour contrast follows WCAG criteria - [ ] Aria labels are added when needed - [ ] Everything is accessible by keyboard - [ ] Tab-Order is comprehensible - [ ] All interactive elements have a visible focus ### Testing - [x] Tests for new code are written - [x] All tests pass - [ ] axe-core dev tools show no critical or major issues ## Additional Notes - Scope widened during planning: the export now mirrors the member overview (all filters, sort incl. tie-breakers, selection rule of the bulk actions) instead of keeping the export's own legacy sort/filter code byte-identical. CSV and PDF output change accordingly (see CHANGELOG); cell format, headers and formula-injection protection are pinned byte-exact by golden files. - The CSV import now validates e-mail custom fields with the same rule used when saving them: values like `a@b` get a clear error instead of "is invalid", and values the storage type accepts but the old import check rejected (e.g. `a@-.de`) are now imported. - The review loop hit its 4-iteration cap; the remaining state was accepted, and one follow-up fix (custom-field e-mail check) was added afterwards with green CI and Dialyzer. - Larger commits (more than 5 files or 150 lines): the import module extraction, the export golden tests, the payload/overview-query/PDF/CSV export switch, the legacy export pipeline removal, the interval validation move, and the test consolidation. - Follow-up: moving the overview query from `MvWeb.MemberLive.Index` into the Membership domain is deliberately deferred to a separate issue.

simon commented

2026-10-15 16:02:46 +02:00

Owner

Description of the implemented changes

The changes were:

Bugfixing
New Feature
Breaking Change
Refactoring

Import/Export consolidation: the CSV import god module is split into focused modules with narrower exception handling, the three parallel member export pipelines are collapsed into one that loads members through the member overview's own query (so CSV and PDF exports contain exactly the members, in exactly the order, the overview shows), and the fee type interval rule becomes a named, translated member validation.

What has been changed?

Pin every import error and warning message exactly
Extract custom field value coercion into its own module
Extract import error and warning texts into a formatter
Let unexpected exceptions fail the chunk instead of a row
Rescue only expected failures when reading custom fields and parsing CSV
Describe the split import modules and row failure rules
Validate custom-field emails with the rule used when saving them
Pin the CSV export format and both controllers' error responses
Encode prepared export rows as CSV
Share the export column labels between CSV and PDF
Derive overview query options from plain assigns
Carry the complete overview filter state in the export payload
Accept text boolean filters and the legacy fee status column name
Load export members through the overview query
Export PDF members exactly as the overview lists them
Export CSV members exactly as the overview lists them
Remove the export's own sorting, filtering and member-based CSV path
Enforce the fee type interval rule as a member validation
Translate the fee type interval rule messages
Drop the unreachable client-side fee type interval checks
Locate the fee type interval rule in the member validations
Share the overview's sortable field list with the export
Remove the unused in-memory boolean custom field filter
Derive the label member keys from the export field allowlist
Align moduledocs with the export scope and fee type validation
Rely on golden files and shared generators instead of duplicate examples
Resolve the fee status column at the overview's reference date
Drop substring export tests the golden and scope tests subsume
Record the import and export consolidation under Unreleased

Definition of Done

Code Quality

No new technical depths
Linting passed
Documentation is added were needed

Accessibility

New elements are properly defined with html-tags
Colour contrast follows WCAG criteria
Aria labels are added when needed
Everything is accessible by keyboard
Tab-Order is comprehensible
All interactive elements have a visible focus

Testing

Tests for new code are written
All tests pass
axe-core dev tools show no critical or major issues

Additional Notes

Scope widened during planning: the export now mirrors the member overview (all filters, sort incl. tie-breakers, selection rule of the bulk actions) instead of keeping the export's own legacy sort/filter code byte-identical. CSV and PDF output change accordingly (see CHANGELOG); cell format, headers and formula-injection protection are pinned byte-exact by golden files.
The CSV import now validates e-mail custom fields with the same rule used when saving them: values like a@b get a clear error instead of "is invalid", and values the storage type accepts but the old import check rejected (e.g. a@-.de) are now imported.
The review loop hit its 4-iteration cap; the remaining state was accepted, and one follow-up fix (custom-field e-mail check) was added afterwards with green CI and Dialyzer.
Larger commits (more than 5 files or 150 lines): the import module extraction, the export golden tests, the payload/overview-query/PDF/CSV export switch, the legacy export pipeline removal, the interval validation move, and the test consolidation.
Follow-up: moving the overview query from MvWeb.MemberLive.Index into the Membership domain is deliberately deferred to a separate issue.

simon added 29 commits

test(import): pin every import error and warning message exactly 0120249bfd

refactor(import): extract custom field value coercion into its own module 208168493d

refactor(import): extract import error and warning texts into a formatter 6d49342ffa

refactor(import): let unexpected exceptions fail the chunk instead of a row 5d2c65cc25

Only database and Ash failures around member creation and group linking are still reported per row; anything else is a programming error that should surface instead of being hidden as a row error.

refactor(import): rescue only expected failures when reading custom fields and parsing CSV 11409e5e39

docs(import): describe the split import modules and row failure rules 22c86a81aa

fix(import): validate custom-field emails with the rule used when saving them 43fea7a546

test(export): pin the CSV export format and both controllers' error responses b46c7e083c

refactor(export): encode prepared export rows as CSV 90699f72af

refactor(export): share the export column labels between CSV and PDF e03c1145f9

refactor(web): derive overview query options from plain assigns 0cad8d0802

feat(export): carry the complete overview filter state in the export payload 38c04bc6f0

fix(export): accept text boolean filters and the legacy fee status column name eadc2af9c8

feat(export): load export members through the overview query dea53a9af3

feat(export): export PDF members exactly as the overview lists them 93ddec3b87

feat(export): export CSV members exactly as the overview lists them 32e16c3fd0