* make it so that the actual tests can be moved anywhere, for example in abra recipe repos
-> major refactoring with pytest test discovery magic
* create RUNNER_DICT dynamically with importlib
-> none of the tests are hardcoded, more tests can be added by placing a folder
* autoload fixtures with pytest plugins
* add URL fixture to navigate on web pages. Includes url parser based on python urllib to generate correct links
* fix nextcloud setups and tests
* add email groundwork with imbox
Reviewed-on: local-it-infrastructure/e2e_tests#7
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* add EnvManager class
* holds all functions that are env file related
* integrates runner dependency resolution
* add integration and unit tests for EnvManager
Reviewed-on: local-it-infrastructure/e2e_tests#6
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* refactoring and rework: runner now has setups / tests / cleanups as lists
* add nextcloud runner
* add email testing prototype with imap fixture
* add dependency resolution (sort env files in input so that test order is correct)
Reviewed-on: local-it-infrastructure/e2e_tests#5
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
Before, a test had only access to it's own env file / configuration (wordpress could see wordpress env file). Now, all env files are available. Wordpress test can also read authentik env file, for example to get the authentik domain.
Reviewed-on: local-it-infrastructure/e2e_tests#4
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* fix flakey tests in authentik / wordpress
* make it possible to rerun tests partially -> passed will be skipped, failed will be repeated
* improve organization of all outputs (moving, renaming, keeping multiple versions etc.)
* add html reports, replace .txt tracebacks
* combine all html reports into one
* add demo runner with comments for documentation purposes
Reviewed-on: local-it-infrastructure/e2e_tests#3
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* authentik sessions created successfully during setup without breaking tracing
* setup works on EN and DE localization by using regex patterns
* automated tracing with pytest --trace option, manual hook no longer needed
Reviewed-on: local-it-infrastructure/e2e_tests#2
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
