* add full integration test of cli / pytest_abra with all tests
* save path of runner_*.py in runner subclass to improve test discovery -> allows for same test name in two different runners
* reorganize output dir names
* use URL fixture everywhere
* rework coordinator interface
* add --session_id to cli args
* add log results table
* plenty of refactoring
* add assert messages
* add plenty of tests
* add /docs dir with plenty of documentation
* fix authentik setup
* add authentik cleanup, remove test user
* add random test user credential generation and integrate into test routine. random creds are saved to STATES
Reviewed-on: local-it-infrastructure/e2e_tests#16
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* add tests for merge_html_reports function
Reviewed-on: local-it-infrastructure/e2e_tests#15
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* general project refactoring
* various small improvements
* improve imap fixture with helper functions and typing
* add wordpress send email setup
* add wordpress receive email test
* add various documentation
Reviewed-on: local-it-infrastructure/e2e_tests#13
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
Before, recipes_dir had to be present in the importable paths of the python interpreter. This was solved by adding it to the PYTHONPATH env var. Now, abratest handles this by itself.
Reviewed-on: local-it-infrastructure/e2e_tests#11
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* fix docker
-> use "pip install -e ." in installation
-> add symlinks in docker image
-> docker / non docker execution can run same main.py + cli
* remove sh scripts
* remove requirements.txt
Reviewed-on: local-it-infrastructure/e2e_tests#10
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* turn repo into installable package (pip install -e .)
* add hatchling build packend
* call it pytest-abra
* add pytest entrypoint, so that it gets loaded automatically if installed (and pytest is run)
* make fixtures optional, so that pytest can still be used in other context
* add cli script -> you can now directly run "pytest-abra" in console
Reviewed-on: local-it-infrastructure/e2e_tests#9
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
-> all program states available
Reviewed-on: local-it-infrastructure/e2e_tests#8
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* make it so that the actual tests can be moved anywhere, for example in abra recipe repos
-> major refactoring with pytest test discovery magic
* create RUNNER_DICT dynamically with importlib
-> none of the tests are hardcoded, more tests can be added by placing a folder
* autoload fixtures with pytest plugins
* add URL fixture to navigate on web pages. Includes url parser based on python urllib to generate correct links
* fix nextcloud setups and tests
* add email groundwork with imbox
Reviewed-on: local-it-infrastructure/e2e_tests#7
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* refactoring and rework: runner now has setups / tests / cleanups as lists
* add nextcloud runner
* add email testing prototype with imap fixture
* add dependency resolution (sort env files in input so that test order is correct)
Reviewed-on: local-it-infrastructure/e2e_tests#5
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
* fix flakey tests in authentik / wordpress
* make it possible to rerun tests partially -> passed will be skipped, failed will be repeated
* improve organization of all outputs (moving, renaming, keeping multiple versions etc.)
* add html reports, replace .txt tracebacks
* combine all html reports into one
* add demo runner with comments for documentation purposes
Reviewed-on: local-it-infrastructure/e2e_tests#3
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
