mitgliederverwaltung/docs/test-performance-optimization.md

# Test Performance Optimization

**Last Updated:** 2026-01-28  
**Status:** ✅ Active optimization program

---

## Executive Summary

This document provides a comprehensive overview of test performance optimizations, risk assessments, and future opportunities. The test suite execution time has been reduced through systematic analysis and targeted optimizations.

### Current Performance Metrics

| Metric | Value |
|--------|-------|
| **Total Execution Time** (without `:slow` tests) | ~368 seconds (~6.1 minutes) |
| **Total Tests** | 1,336 tests (+ 25 doctests) |
| **Async Execution** | 163.5 seconds |
| **Sync Execution** | 281.5 seconds |
| **Slow Tests Excluded** | 25 tests (tagged with `@tag :slow`) |
| **Top 50 Slowest Tests** | 121.9 seconds (27.4% of total time) |

### Optimization Impact Summary

| Optimization | Tests Affected | Time Saved | Status |
|--------------|----------------|------------|--------|
| Seeds tests reduction | 13 → 4 tests | ~10-16s | ✅ Completed |
| Performance tests tagging | 9 tests | ~3-4s per run | ✅ Completed |
| Critical test query filtering | 1 test | ~8-10s | ✅ Completed |
| Nightly suite tagging | 25 tests | ~77s per run | ✅ Completed |
| **Total Saved** | | **~98-107s** | |

---

## Completed Optimizations

### 1. Seeds Test Suite Optimization

**Date:** 2026-01-28  
**Status:** ✅ Completed

#### What Changed

- **Reduced test count:** From 13 tests to 4 tests (69% reduction)
- **Reduced seeds executions:** From 8-10 times to 5 times per test run
- **Execution time:** From 24-30 seconds to 13-17 seconds
- **Time saved:** ~10-16 seconds per test run (40-50% faster)

#### Removed Tests (9 tests)

Tests were removed because their functionality is covered by domain-specific test suites:

1. `"at least one member has no membership fee type assigned"` → Covered by `membership_fees/*_test.exs`
2. `"each membership fee type has at least one member"` → Covered by `membership_fees/*_test.exs`
3. `"members with fee types have cycles with various statuses"` → Covered by `cycle_generator_test.exs`
4. `"creates all 5 authorization roles with correct permission sets"` → Covered by `authorization/*_test.exs`
5. `"all roles have valid permission_set_names"` → Covered by `authorization/permission_sets_test.exs`
6. `"does not change role of users who already have a role"` → Merged into idempotency test
7. `"role creation is idempotent"` (detailed) → Merged into general idempotency test

#### Retained Tests (4 tests)

Critical deployment requirements are still covered:

1. ✅ **Smoke Test:** Seeds run successfully and create basic data
2. ✅ **Idempotency Test:** Seeds can be run multiple times without duplicating data
3. ✅ **Admin Bootstrap:** Admin user exists with Admin role (critical for initial access)
4. ✅ **System Role Bootstrap:** Mitglied system role exists (critical for user registration)

#### Risk Assessment

| Removed Test Category | Alternative Coverage | Risk Level |
|----------------------|---------------------|------------|
| Member/fee type distribution | `membership_fees/*_test.exs` | ⚠️ Low |
| Cycle status variations | `cycle_generator_test.exs` | ⚠️ Low |
| Detailed role configs | `authorization/*_test.exs` | ⚠️ Very Low |
| Permission set validation | `permission_sets_test.exs` | ⚠️ Very Low |

**Overall Risk:** ⚠️ **Low** - All removed tests have equivalent or better coverage in domain-specific test suites.

---

### 2. Nightly Suite Implementation (`@tag :slow`)

**Date:** 2026-01-28  
**Status:** ✅ Completed

#### What Changed

Tests with **low risk** and **execution time >1 second** are now tagged with `@tag :slow` and excluded from standard test runs. These tests are important but not critical for every commit and can be run in nightly CI builds.

#### Tagging Criteria

**Tagged as `@tag :slow` when:**
- ✅ Test execution time >1 second
- ✅ Low risk (not critical for catching regressions in core business logic)
- ✅ UI/Display tests (formatting, rendering)
- ✅ Workflow detail tests (not core functionality)
- ✅ Edge cases with large datasets

**NOT tagged when:**
- ❌ Core CRUD operations (Member/User Create/Update/Destroy)
- ❌ Basic Authentication/Authorization
- ❌ Critical Bootstrap (Admin user, system roles)
- ❌ Email Synchronization
- ❌ Representative tests per Permission Set + Action

#### Identified Nightly Suite Tests (25 tests)

**1. Seeds Tests (2 tests) - 18.1s**
- `"runs successfully and creates basic data"` (9.0s)
- `"is idempotent when run multiple times"` (9.1s)
- **Note:** Critical bootstrap tests remain in fast suite

**2. UserLive.ShowTest (3 tests) - 10.8s**
- `"mounts successfully with valid user ID"` (4.2s)
- `"displays linked member when present"` (2.4s)
- `"redirects to user list when viewing system actor user"` (4.2s)

**3. UserLive.IndexTest (5 tests) - 25.0s**
- `"displays users in a table"` (1.0s)
- `"initially sorts by email ascending"` (2.2s)
- `"can sort email descending by clicking sort button"` (3.4s)
- `"select all automatically checks when all individual users are selected"` (2.0s)
- `"displays linked member name in user list"` (1.9s)

**4. MemberLive.IndexCustomFieldsDisplayTest (3 tests) - 4.9s**
- `"displays custom field with show_in_overview: true"` (1.6s)
- `"formats date custom field values correctly"` (1.5s)
- `"formats email custom field values correctly"` (1.8s)

**5. MemberLive.IndexCustomFieldsEdgeCasesTest (3 tests) - 3.6s**
- `"displays custom field column even when no members have values"` (1.1s)
- `"displays very long custom field values correctly"` (1.4s)
- `"handles multiple custom fields with show_in_overview correctly"` (1.2s)

**6. RoleLive Tests (7 tests) - 7.7s**
- `role_live_test.exs`: `"mounts successfully"` (1.5s), `"deletes non-system role"` (2.1s)
- `role_live/show_test.exs`: 5 tests >1s (mount, display, navigation)

**7. MemberAvailableForLinkingTest (1 test) - 1.5s**
- `"limits results to 10 members even when more exist"` (1.5s)

**8. Performance Tests (1 test) - 3.8s**
- `"boolean filter performance with 150 members"` (3.8s)

**Total:** 25 tests, ~77 seconds saved

#### Execution Commands

**Fast Tests (Default):**
```bash
just test-fast
# or
mix test --exclude slow
```

**Slow/Nightly Tests Only:**
```bash
just test-slow
# or
mix test --only slow
```

**All Tests:**
```bash
just test
# or
mix test
```

#### CI/CD Integration

- **Standard CI:** Runs `mix test --exclude slow` for faster feedback loops (~6 minutes)
- **Nightly Builds:** Separate pipeline runs daily and executes `mix test --only slow` (~1.3 minutes)
- **Pre-Merge:** Full test suite (`mix test`) runs before merging to main
- **Manual Execution:** Can be triggered via Drone CLI or Web UI

#### Risk Assessment

**Risk Level:** ✅ **Very Low**

- All tagged tests have **low risk** - they don't catch critical regressions
- Core functionality remains tested (CRUD, Auth, Bootstrap)
- Standard test runs are faster (~6 minutes vs ~7.4 minutes)
- Nightly builds ensure comprehensive coverage
- No functionality is lost, only execution timing changed

**Critical Tests Remain in Fast Suite:**
- Core CRUD operations (Member/User Create/Update/Destroy)
- Basic Authentication/Authorization
- Critical Bootstrap (Admin user, system roles)
- Email Synchronization
- Representative Policy tests (one per Permission Set + Action)

---

### 3. Critical Test Optimization

**Date:** 2026-01-28  
**Status:** ✅ Completed

#### Problem Identified

The test `test respects show_in_overview config` was the slowest test in the suite:
- **Isolated execution:** 4.8 seconds
- **In full test run:** 14.7 seconds
- **Difference:** 9.9 seconds (test isolation issue)

#### Root Cause

The test loaded **all members** from the database, not just the 2 members from the test setup. In full test runs, many members from other tests were present in the database, significantly slowing down the query.

#### Solution Implemented

**Query Filtering:** Added search query parameter to filter to only the expected member.

**Code Change:**
```elixir
# Before:
{:ok, _view, html} = live(conn, "/members")

# After:
{:ok, _view, html} = live(conn, "/members?query=Alice")
```

#### Results

| Execution | Before | After | Improvement |
|-----------|--------|-------|-------------|
| **Isolated** | 4.8s | 1.1s | **-77%** (3.7s saved) |
| **In Module** | 4.2s | 0.4s | **-90%** (3.8s saved) |
| **Expected in Full Run** | 14.7s | ~4-6s | **-65% to -73%** (8-10s saved) |

#### Risk Assessment

**Risk Level:** ✅ **Very Low**

- Test functionality unchanged - only loads expected data
- All assertions still pass
- Test is now faster and more isolated
- No impact on test coverage

---

### 3. Nightly Suite Analysis and Categorization

**Date:** 2026-01-28  
**Status:** ✅ Completed

#### Analysis Methodology

A comprehensive analysis was performed to identify tests suitable for the nightly suite based on:
- **Execution time:** Tests taking >1 second
- **Risk assessment:** Tests that don't catch critical regressions
- **Test category:** UI/Display, workflow details, edge cases

#### Test Categorization

**🔴 CRITICAL - Must Stay in Fast Suite:**
- Core Business Logic (Member/User CRUD)
- Authentication & Authorization Basics
- Critical Bootstrap (Admin user, system roles)
- Email Synchronization
- Representative Policy Tests (one per Permission Set + Action)

**🟡 LOW RISK - Moved to Nightly Suite:**
- Seeds Tests (non-critical: smoke test, idempotency)
- LiveView Display/Formatting Tests
- UserLive.ShowTest (core functionality covered by Index/Form)
- UserLive.IndexTest UI Features (sorting, checkboxes, navigation)
- RoleLive Tests (role management, not core authorization)
- MemberLive Custom Fields Display Tests
- Edge Cases with Large Datasets

#### Risk Assessment Summary

| Category | Tests | Time Saved | Risk Level | Rationale |
|----------|-------|------------|------------|-----------|
| Seeds (non-critical) | 2 | 18.1s | ⚠️ Low | Critical bootstrap tests remain |
| UserLive.ShowTest | 3 | 10.8s | ⚠️ Low | Core CRUD covered by Index/Form |
| UserLive.IndexTest (UI) | 5 | 25.0s | ⚠️ Low | UI features, not core functionality |
| Custom Fields Display | 6 | 8.5s | ⚠️ Low | Formatting tests, visible in code review |
| RoleLive Tests | 7 | 7.7s | ⚠️ Low | Role management, not authorization |
| Edge Cases | 1 | 1.5s | ⚠️ Low | Edge case, not critical path |
| Performance Tests | 1 | 3.8s | ✅ Very Low | Explicit performance validation |
| **Total** | **25** | **~77s** | **⚠️ Low** | |

**Overall Risk:** ⚠️ **Low** - All moved tests have low risk and don't catch critical regressions. Core functionality remains fully tested.

#### Tests Excluded from Nightly Suite

The following tests were **NOT** moved to nightly suite despite being slow:

- **Policy Tests:** Medium risk - kept in fast suite (representative tests remain)
- **UserLive.FormTest:** Medium risk - core CRUD functionality
- **Tests <1s:** Don't meet execution time threshold
- **Critical Bootstrap Tests:** High risk - deployment critical

---

## Current Performance Analysis

### Top 20 Slowest Tests (without `:slow`)

After implementing the nightly suite, the remaining slowest tests are:

| Rank | Test | File | Time | Category |
|------|------|------|------|----------|
| 1 | `test Critical bootstrap invariants Mitglied system role exists` | `seeds_test.exs` | 6.7s | Critical Bootstrap |
| 2 | `test Critical bootstrap invariants Admin user has Admin role` | `seeds_test.exs` | 5.0s | Critical Bootstrap |
| 3 | `test normal_user permission set can read own user record` | `user_policies_test.exs` | 2.6s | Policy Test |
| 4 | `test normal_user permission set can create member` | `member_policies_test.exs` | 2.5s | Policy Test |
| 5-20 | Various Policy and LiveView tests | Multiple files | 1.5-2.4s each | Policy/LiveView |

**Total Top 20:** ~44 seconds (12% of total time without `:slow`)

**Note:** Many previously slow tests (UserLive.IndexTest, UserLive.ShowTest, Display/Formatting tests) are now tagged with `@tag :slow` and excluded from standard runs.

### Performance Hotspots Identified

#### 1. Seeds Tests (~16.2s for 4 tests)

**Status:** ✅ Optimized (reduced from 13 tests)  
**Remaining Optimization Potential:** 3-5 seconds

**Opportunities:**
- Settings update could potentially be moved to `setup_all` (if sandbox allows)
- Seeds execution could be further optimized (less data in test mode)
- Idempotency test could be optimized (only 1x seeds instead of 2x)

#### 2. User LiveView Tests (~35.5s for 10 tests)

**Status:** ⏳ Identified for optimization  
**Optimization Potential:** 15-20 seconds

**Files:**
- `test/mv_web/user_live/index_test.exs` (3 tests, ~10.2s)
- `test/mv_web/user_live/form_test.exs` (4 tests, ~15.0s)
- `test/mv_web/user_live/show_test.exs` (3 tests, ~10.3s)

**Patterns:**
- Many tests create user/member data
- LiveView mounts are expensive
- Form submissions with validations are slow

**Recommended Actions:**
- Move shared fixtures to `setup_all`
- Reduce test data volume (3-5 users instead of 10+)
- Optimize setup patterns for recurring patterns

#### 3. Policy Tests (~8.7s for 3 tests)

**Status:** ⏳ Identified for optimization  
**Optimization Potential:** 5-8 seconds

**Files:**
- `test/mv/membership/member_policies_test.exs` (2 tests, ~6.1s)
- `test/mv/accounts/user_policies_test.exs` (1 test, ~2.6s)

**Pattern:**
- Each test creates new roles/users/members
- Roles are identical across tests

**Recommended Actions:**
- Create roles in `setup_all` (shared across tests)
- Reuse common fixtures
- Maintain test isolation while optimizing setup

---

## Future Optimization Opportunities

### Priority 1: User LiveView Tests Optimization

**Estimated Savings:** 15-20 seconds

**Actions:**
1. Move shared fixtures to `setup_all` where possible
2. Reduce test data volume (use 3-5 users instead of 10+)
3. Analyze and optimize recurring setup patterns
4. Consider mocking expensive operations (if appropriate)

**Risk Assessment:** ⚠️ **Low**
- Requires careful testing to ensure isolation is maintained
- Should verify that shared fixtures don't cause test interdependencies

### Priority 2: Policy Tests Optimization

**Estimated Savings:** 5-8 seconds

**Actions:**
1. Create roles in `setup_all` (they're identical across tests)
2. Reuse common fixtures (users, members)
3. Maintain test isolation while optimizing setup

**Risk Assessment:** ⚠️ **Low**
- Roles are read-only in tests, safe to share
- Need to ensure user/member fixtures don't interfere with each other

### Priority 3: Seeds Tests Further Optimization

**Estimated Savings:** 3-5 seconds

**Actions:**
1. Investigate if settings update can be moved to `setup_all`
2. Introduce seeds mode for tests (less data in test mode)
3. Optimize idempotency test (only 1x seeds instead of 2x)

**Risk Assessment:** ⚠️ **Low to Medium**
- Sandbox limitations may prevent `setup_all` usage
- Seeds mode would require careful implementation
- Idempotency test optimization needs to maintain test validity

### Priority 4: Additional Test Isolation Improvements

**Estimated Savings:** Variable (depends on specific tests)

**Actions:**
1. Review tests that load all records (similar to the critical test fix)
2. Add query filters where appropriate
3. Ensure proper test isolation in async tests

**Risk Assessment:** ⚠️ **Very Low**
- Similar to the critical test optimization (proven approach)
- Improves test isolation and reliability

---

## Estimated Total Optimization Potential

| Priority | Optimization | Estimated Savings |
|----------|-------------|-------------------|
| 1 | User LiveView Tests | 15-20s |
| 2 | Policy Tests | 5-8s |
| 3 | Seeds Tests Further | 3-5s |
| 4 | Additional Isolation | Variable |
| **Total Potential** | | **23-33 seconds** |

**Projected Final Time:** From ~368 seconds (fast suite) to **~340-350 seconds** (~5.7-5.8 minutes) with remaining optimizations

---

## Risk Assessment Summary

### Overall Risk Level: ⚠️ **Low**

All optimizations maintain test coverage while improving performance:

| Optimization | Risk Level | Mitigation |
|-------------|------------|------------|
| Seeds tests reduction | ⚠️ Low | Coverage mapped to domain tests |
| Performance tests tagging | ✅ Very Low | Tests still executed, just separately |
| Critical test optimization | ✅ Very Low | Functionality unchanged, better isolation |
| Future optimizations | ⚠️ Low | Careful implementation with verification |

### Monitoring Plan

#### Success Criteria

- ✅ Seeds tests execute in <20 seconds consistently
- ✅ No increase in seeds-related deployment failures
- ✅ No regression in authorization or membership fee bugs
- ✅ Top 20 slowest tests: < 60 seconds (currently ~44s)
- ✅ Total execution time (without `:slow`): < 10 minutes (currently 6.1 min)
- ⏳ Nightly suite execution time: < 2 minutes (currently ~1.3 min)

#### What to Watch For

1. **Production Seeds Failures:**
   - Monitor deployment logs for seeds errors
   - If failures increase, consider restoring detailed tests

2. **Authorization Bugs After Seeds Changes:**
   - If role/permission bugs appear after seeds modifications
   - May indicate need for more seeds-specific role validation

3. **Test Performance Regression:**
   - Monitor test execution times in CI
   - Alert if times increase significantly

4. **Developer Feedback:**
   - If developers report missing test coverage
   - Adjust based on real-world experience

---

## Benchmarking and Analysis

### How to Benchmark Tests

**ExUnit Built-in Benchmarking:**

The test suite is configured to show the slowest tests automatically:

```elixir
# test/test_helper.exs
ExUnit.start(
  slowest: 10  # Shows 10 slowest tests at the end of test run
)
```

**Run Benchmark Analysis:**

```bash
# Show slowest tests
mix test --slowest 20

# Exclude slow tests for faster feedback
mix test --exclude slow --slowest 20

# Run only slow tests
mix test --only slow --slowest 10

# Benchmark specific test file
mix test test/mv_web/member_live/index_member_fields_display_test.exs --slowest 5
```

### Benchmarking Best Practices

1. **Run benchmarks regularly** (e.g., monthly) to catch performance regressions
2. **Compare isolated vs. full runs** to identify test isolation issues
3. **Monitor CI execution times** to track trends over time
4. **Document significant changes** in test performance

---

## Test Suite Structure

### Test Execution Modes

**Fast Tests (Default):**
- Excludes slow tests (`@tag :slow`)
- Used for standard development workflow
- Execution time: ~6 minutes
- Command: `mix test --exclude slow` or `just test-fast`

**Slow/Nightly Tests:**
- Tests tagged with `@tag :slow` (25 tests)
- Low risk, >1 second execution time
- UI/Display tests, workflow details, edge cases
- Execution time: ~1.3 minutes
- Command: `mix test --only slow` or `just test-slow`
- Run in nightly CI builds

**All Tests:**
- Includes both fast and slow tests
- Used for comprehensive validation (pre-merge)
- Execution time: ~7.4 minutes
- Command: `mix test` or `just test`

### Test Organization

Tests are organized to mirror the `lib/` directory structure:

```
test/
├── accounts/              # Accounts domain tests
├── membership/            # Membership domain tests
├── membership_fees/      # Membership fees domain tests
├── mv/                    # Core application tests
│   ├── accounts/         # User-related tests
│   ├── membership/       # Member-related tests
│   └── authorization/    # Authorization tests
├── mv_web/               # Web layer tests
│   ├── controllers/     # Controller tests
│   ├── live/             # LiveView tests
│   └── components/       # Component tests
└── support/              # Test helpers
    ├── conn_case.ex      # Controller test setup
    └── data_case.ex      # Database test setup
```

---

## Best Practices for Test Performance

### When Writing New Tests

1. **Use `async: true`** when possible (for parallel execution)
2. **Filter queries** to only load necessary data
3. **Share fixtures** in `setup_all` when appropriate
4. **Tag performance tests** with `@tag :slow` if they use large datasets
5. **Keep test data minimal** - only create what's needed for the test

### When Optimizing Existing Tests

1. **Measure first** - Use `mix test --slowest` to identify bottlenecks
2. **Compare isolated vs. full runs** - Identify test isolation issues
3. **Optimize setup** - Move shared data to `setup_all` where possible
4. **Filter queries** - Only load data needed for the test
5. **Verify coverage** - Ensure optimizations don't reduce test coverage

### Test Tagging Guidelines

#### Tag as `@tag :slow` when:

1. **Performance Tests:**
   - Explicitly testing performance characteristics
   - Using large datasets (50+ records)
   - Testing scalability or query optimization
   - Validating N+1 query prevention

2. **Low-Risk Tests (>1s):**
   - UI/Display/Formatting tests (not critical for every commit)
   - Workflow detail tests (not core functionality)
   - Edge cases with large datasets
   - Show page tests (core functionality covered by Index/Form tests)
   - Non-critical seeds tests (smoke tests, idempotency)

#### Do NOT tag as `@tag :slow` when:

- ❌ Test is slow due to inefficient setup (fix the setup instead)
- ❌ Test is slow due to bugs (fix the bug instead)
- ❌ Core CRUD operations (Member/User Create/Update/Destroy)
- ❌ Basic Authentication/Authorization
- ❌ Critical Bootstrap (Admin user, system roles)
- ❌ Email Synchronization
- ❌ Representative Policy tests (one per Permission Set + Action)
- ❌ It's an integration test (use `@tag :integration` instead)

---

## Changelog

### 2026-01-28: Initial Optimization Phase

**Completed:**
- ✅ Reduced seeds tests from 13 to 4 tests
- ✅ Tagged 9 performance tests with `@tag :slow`
- ✅ Optimized critical test with query filtering
- ✅ Created slow test suite infrastructure
- ✅ Updated CI/CD to exclude slow tests from standard runs
- ✅ Added nightly CI pipeline for slow tests

**Time Saved:** ~21-30 seconds per test run

### 2026-01-28: Nightly Suite Implementation

**Completed:**
- ✅ Analyzed all tests for nightly suite candidates
- ✅ Identified 36 tests with low risk and >1s execution time
- ✅ Tagged 25 tests with `@tag :slow` for nightly suite
- ✅ Categorized tests by risk level and execution time
- ✅ Documented tagging criteria and guidelines

**Tests Tagged:**
- 2 Seeds tests (non-critical) - 18.1s
- 3 UserLive.ShowTest tests - 10.8s
- 5 UserLive.IndexTest tests - 25.0s
- 3 MemberLive.IndexCustomFieldsDisplayTest tests - 4.9s
- 3 MemberLive.IndexCustomFieldsEdgeCasesTest tests - 3.6s
- 7 RoleLive tests - 7.7s
- 1 MemberAvailableForLinkingTest - 1.5s
- 1 Performance test (already tagged) - 3.8s

**Time Saved:** ~77 seconds per test run

**Total Optimization Impact:**
- **Before:** ~445 seconds (7.4 minutes)
- **After (fast suite):** ~368 seconds (6.1 minutes)
- **Time saved:** ~77 seconds (17% reduction)

**Next Steps:**
- ⏳ Monitor nightly suite execution in CI
- ⏳ Optimize remaining slow tests (Policy tests, etc.)
- ⏳ Further optimize Seeds tests (Priority 3)

---

## References

- **Testing Standards:** `CODE_GUIDELINES.md` - Section 4 (Testing Standards)
- **CI/CD Configuration:** `.drone.yml`
- **Test Helper:** `test/test_helper.exs`
- **Justfile Commands:** `Justfile` (test-fast, test-slow, test-all)

---

## Questions & Answers

**Q: What if seeds create wrong data and break the system?**  
A: The smoke test will fail if seeds raise errors. Domain tests ensure business logic is correct regardless of seeds content.

**Q: What if we add a new critical bootstrap requirement?**  
A: Add a new test to the "Critical bootstrap invariants" section in `test/seeds_test.exs`.

**Q: How do we know the removed tests aren't needed?**  
A: Monitor for 2-3 months. If no seeds-related bugs appear that would have been caught by removed tests, they were redundant.

**Q: Should we restore the tests for important releases?**  
A: Consider running the full test suite (including slow tests) before major releases. Daily development uses the optimized suite.

**Q: How do I add a new performance test?**  
A: Tag it with `@tag :slow` or `@moduletag :slow`. See "Performance Test Guidelines" section above.

**Q: Can I run slow tests locally?**  
A: Yes, use `just test-slow` or `mix test --only slow`. They're excluded from standard runs for faster feedback.

**Q: What is the "nightly suite"?**  
A: The "nightly suite" refers to tests tagged with `@tag :slow`. We use a single tag (`:slow`) for both performance tests and tests suitable for nightly execution. All tests tagged with `:slow` are excluded from standard runs and can be executed in nightly CI builds. 

**Q: Which tests should I tag as `:slow`?**  
A: Tag tests with `@tag :slow` if they: (1) take >1 second, (2) have low risk (not critical for catching regressions), and (3) test UI/Display/Formatting or workflow details. See "Test Tagging Guidelines" section for details.

**Q: What if a slow test fails in nightly builds?**  
A: If a test in the nightly suite fails, investigate the failure. If it indicates a critical regression, consider moving it back to the fast suite. If it's a flaky test, fix the test itself.