local-it/e2e_tests
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

various (#16)

Browse source 
* add full integration test of cli / pytest_abra with all tests

* save path of runner_*.py in runner subclass to improve test discovery -> allows for same test name in two different runners

* reorganize output dir names

* use URL fixture everywhere

* rework coordinator interface

* add --session_id to cli args

* add log results table

* plenty of refactoring

* add assert messages

* add plenty of tests

* add /docs dir with plenty of documentation

* fix authentik setup

* add authentik cleanup, remove test user

* add random test user credential generation and integrate into test routine. random creds are saved to STATES

Reviewed-on: local-it-infrastructure/e2e_tests#16
Co-authored-by: Daniel <d.brummerloh@gmail.com>
Co-committed-by: Daniel <d.brummerloh@gmail.com>
This commit is contained in:
Daniel 2023-12-14 14:03:58 +01:00 • committed by dan
parent 016b88a68d
commit 2dd765a974
36 changed files with 1145 additions and 432 deletions
Download patch file Download diff file Expand all files Collapse all files

154
README.md
View file

@ -1,90 +1,18 @@
# pytest-abra
Pytest-Abra is an installable python package design to test instances created with [abra](https://docs.coopcloud.tech/abra/). After installation, you will have two things:
Pytest-Abra is an installable python package baed on pytest, designed to test instances created with [abra](https://docs.coopcloud.tech/abra/). After installation, you will have two things:
- `abratest` CLI command. *Used to initialize the testing.*
- `pytest-abra` Pytest plugin. *Automatically loads custom fixtures in any pytest (see `pytest_abra/custom_fixtures.py`)*
## CLI (`abratest`)
`abratest` can be called via terminal:
```bash
abratest [arguments]
```
To run successfully, very specific arguments are required. The easiest way to use abratest is with the helper script in `main.py`. Of yourse you can implement a similar helper script in the language of your liking. The cli command `abratest` has 3 **required arguments**:
- `--env_paths`: list of the .env files used in the test
- `--recipes_dir`: directory of all available abra recipes
- `--output_dir`: target directory for all test results
### env_paths [string]
The variable env_paths consists of one or more paths pointing at .env files. The paths are separated with ";". These .env files are actually configuration files for `abra` recipes, but `pytest-abra` uses the same files for test configuration.
To run `abratest` with these `.env` configuration files
```
/path/to/config_1.env
/path/to/config_2.env
/path/to/config_3.env
```
we simply call
```
abratest --env_paths /path/to/config_1.env;/path/to/config_2.env;/path/to/config_3.env
```
Under the hood, each `.env` file in `--env_paths` will create one instance of a `Runner` subclass. Let's say we have `wordpress_configuration.env` containing `TYPE=wordpress`. This will create an instance of `RunnerWordpress`. This class has to be imported from `recipes_dir`.
### recipes_dir [string]
The required argument `--recipes_dir` has to point to the directory, where all the abra recipes are stored. We can call `abratest` with
```
abratest --recipes_dir /path/to/abra/recipes
```
The expected dir structure inside of `recipes_dir` is as follows:
```
DIR recipes_dir [contains abra recipes]
│
├── DIR authentik [authentik recipe]
│ ├── [files from authentik recipe]
│ └── DIR tests_authentik [pytest tests for authentik]
│ ├── FILE runner_authentik.py # containing RunnerAuthentik class
│ └── [pytest_files]
│
└── DIR wordpress [wordpress recipe]
├── [files from wordpress recipe]
└── DIR tests_wordpress [pytest tests for wordpress]
├── FILE runner_wordpress.py # containing RunnerWordpress class
└── [pytest_files]
```
The class `RunnerWordpress` will be automatically imported using `importlib` library, which is equivalent to the code below. Note that `recipes_dir` will be added to sys.path automatically for the import to work and that every `Runner` class matching `recipes_dir.rglob("*/runner*.py")` will be imported.
```python
from wordpress.tests_wordpress.runner_wordpress import RunnerWordpress
```
### output_dir [string]
Path to the directory where all test outputs are stored (test report, tracebacks, playwright traces etc.)
```
abratest --output_dir /path/to/output
```
- `pytest-abra` Pytest plugin. *Automatically loads custom fixtures in any pytest run (see `pytest_abra/custom_fixtures.py`)*
# Usage
To use pytest-abra, follow these steps:
Pytest-abra can easily be installed on any system but also offers a Docker image. To use pytest-abra, follow these steps:
## 1. GIT clone [with & without Docker]
## Usage [without Docker]
### Installation [without Docker]
To clone with submodules, use these git commands:
@ -95,14 +23,6 @@ git submodule update --init // add submodule after normal cloning
git submodule update --remote // update submodules
```
## Run
You can run pytest-abra with and without Docker. Choose now and follow the steps accordingly:
## 2.1 Run without Docker
### Installation
Create a python environment and install all dependencies via
```bash
@ -110,46 +30,40 @@ pip install -e .
playwright install
```
Run the script with
### Run [without Docker]
Run the helper script or directly use the cli command (see docs)
```bash
python main.py
python main.py # run pytest-abra
abratest [options]
```
## 2.2 Run with Docker
## Usage [with docker]
### Installation [with docker]
To clone with submodules, use these git commands:
```bash
git clone --recurse-submodules <repository>
// optional:
git submodule update --init // add submodule after normal cloning
git submodule update --remote // update submodules
```
Build the image
```bash
docker compose build # build the image
docker compose build --no-cache # Force rebuild without cache
```
### Run [with docker]
Run the script
```bash
docker compose run --rm app python main.py # run pytest-abra
docker compose run --rm -it app /bin/bash # debug the container
```
Force rebuild with cache
```bash
docker-compose up --build
```
Force rebuild without cache
```bash
docker-compose build --no-cache
```
## Playwright Debug & Codegen
Use playwright debug mode or codegen to create testing code easily by recording browser actions https://playwright.dev/python/docs/codegen
```bash
abratest --debug # launch your tests in debug mode
playwright codegen demo.playwright.dev/todomvc # visit given url in codegen mode
```
## Development
```bash
pytest # test pytest-abra
pytest -m "not slow" # test pytest-abra without slow tests
pytest --collect-only # debug test pytest-abra
docker compose run --rm app pytest # run pytest-abra
docker compose run --rm -it app /bin/bash # use the container interactively
```