improve docs
This commit is contained in:
parent
5d6696cf19
commit
622b2a1f8e
2 changed files with 201 additions and 129 deletions
160
README.md
160
README.md
|
|
@ -1,96 +1,18 @@
|
|||
# pytest-abra
|
||||
|
||||
Pytest-Abra is an installable python package design to test instances created with [abra](https://docs.coopcloud.tech/abra/). After installation, you will have two things:
|
||||
Pytest-Abra is an installable python package baed on pytest, designed to test instances created with [abra](https://docs.coopcloud.tech/abra/). After installation, you will have two things:
|
||||
|
||||
- `abratest` CLI command. *Used to initialize the testing.*
|
||||
|
||||
- `pytest-abra` Pytest plugin. *Automatically loads custom fixtures in any pytest (see `pytest_abra/custom_fixtures.py`)*
|
||||
|
||||
## CLI (`abratest`)
|
||||
|
||||
`abratest` can be called via terminal:
|
||||
|
||||
```bash
|
||||
abratest [arguments]
|
||||
```
|
||||
|
||||
To run successfully, very specific arguments are required. The easiest way to use abratest is with the helper script in `main.py`. Of yourse you can implement a similar helper script in the language of your liking. The cli command `abratest` has 3 **required arguments**:
|
||||
|
||||
- `--env_paths`: list of the .env files used in the test
|
||||
- `--recipes_dir`: directory of all available abra recipes
|
||||
- `--output_dir`: target directory for all test results
|
||||
|
||||
Furtheremore, there are these optional arguments:
|
||||
|
||||
- `--resume`: `abratest` will take the directory in `output_dir` with the most recent creation date and resume the tests there.
|
||||
- `--debug`: enables playwright debug mode, see docs [here](https://playwright.dev/python/docs/running-tests#debugging-tests)
|
||||
- `--timeout`: will overwrite the default playwright timeouts in [ms], see docs [here](https://playwright.dev/python/docs/api/class-browsercontext#browser-context-set-default-timeout) and [here](https://playwright.dev/python/docs/test-assertions#global-timeout). In our current setup, some tests can fail at 10s but will pass with 20s.
|
||||
|
||||
### env_paths [string]
|
||||
|
||||
The variable env_paths consists of one or more paths pointing at .env files. The paths are separated with ";". These .env files are actually configuration files for `abra` recipes, but `pytest-abra` uses the same files for test configuration.
|
||||
|
||||
To run `abratest` with these `.env` configuration files
|
||||
|
||||
```
|
||||
/path/to/config_1.env
|
||||
/path/to/config_2.env
|
||||
/path/to/config_3.env
|
||||
```
|
||||
|
||||
we simply call
|
||||
|
||||
```
|
||||
abratest --env_paths /path/to/config_1.env;/path/to/config_2.env;/path/to/config_3.env
|
||||
```
|
||||
|
||||
Under the hood, each `.env` file in `--env_paths` will create one instance of a `Runner` subclass. Let's say we have `wordpress_configuration.env` containing `TYPE=wordpress`. This will create an instance of `RunnerWordpress`. This class has to be imported from `recipes_dir`.
|
||||
|
||||
### recipes_dir [string]
|
||||
|
||||
The required argument `--recipes_dir` has to point to the directory, where all the abra recipes are stored. We can call `abratest` with
|
||||
|
||||
```
|
||||
abratest --recipes_dir /path/to/abra/recipes
|
||||
```
|
||||
|
||||
The expected dir structure inside of `recipes_dir` is as follows:
|
||||
|
||||
```
|
||||
DIR recipes_dir [contains abra recipes]
|
||||
│
|
||||
├── DIR authentik [authentik recipe]
|
||||
│ ├── [files from authentik recipe]
|
||||
│ └── DIR tests_authentik [pytest tests for authentik]
|
||||
│ ├── FILE runner_authentik.py # containing RunnerAuthentik class
|
||||
│ └── [pytest_files]
|
||||
│
|
||||
└── DIR wordpress [wordpress recipe]
|
||||
├── [files from wordpress recipe]
|
||||
└── DIR tests_wordpress [pytest tests for wordpress]
|
||||
├── FILE runner_wordpress.py # containing RunnerWordpress class
|
||||
└── [pytest_files]
|
||||
```
|
||||
|
||||
The class `RunnerWordpress` will be automatically imported using `importlib` library, which is equivalent to the code below. Note that `recipes_dir` will be added to sys.path automatically for the import to work and that every `Runner` class matching `recipes_dir.rglob("*/runner*.py")` will be imported.
|
||||
|
||||
```python
|
||||
from wordpress.tests_wordpress.runner_wordpress import RunnerWordpress
|
||||
```
|
||||
|
||||
### output_dir [string]
|
||||
|
||||
Path to the directory where all test outputs are stored (test report, tracebacks, playwright traces etc.)
|
||||
|
||||
```
|
||||
abratest --output_dir /path/to/output
|
||||
```
|
||||
- `pytest-abra` Pytest plugin. *Automatically loads custom fixtures in any pytest run (see `pytest_abra/custom_fixtures.py`)*
|
||||
|
||||
# Usage
|
||||
|
||||
To use pytest-abra, follow these steps:
|
||||
Pytest-abra can easily be installed on any system but also offers a Docker image. To use pytest-abra, follow these steps:
|
||||
|
||||
## 1. GIT clone [with & without Docker]
|
||||
## Usage [without Docker]
|
||||
|
||||
### Installation [without Docker]
|
||||
|
||||
To clone with submodules, use these git commands:
|
||||
|
||||
|
|
@ -101,14 +23,6 @@ git submodule update --init // add submodule after normal cloning
|
|||
git submodule update --remote // update submodules
|
||||
```
|
||||
|
||||
## Run
|
||||
|
||||
You can run pytest-abra with and without Docker. Choose now and follow the steps accordingly:
|
||||
|
||||
## 2.1 Run without Docker
|
||||
|
||||
### Installation
|
||||
|
||||
Create a python environment and install all dependencies via
|
||||
|
||||
```bash
|
||||
|
|
@ -116,46 +30,40 @@ pip install -e .
|
|||
playwright install
|
||||
```
|
||||
|
||||
Run the script with
|
||||
### Run [without Docker]
|
||||
|
||||
Run the helper script or directly use the cli command (see docs)
|
||||
|
||||
```bash
|
||||
python main.py
|
||||
python main.py # run pytest-abra
|
||||
abratest [options]
|
||||
```
|
||||
|
||||
## 2.2 Run with Docker
|
||||
## Usage [with docker]
|
||||
|
||||
### Installation [with docker]
|
||||
|
||||
To clone with submodules, use these git commands:
|
||||
|
||||
```bash
|
||||
git clone --recurse-submodules <repository>
|
||||
// optional:
|
||||
git submodule update --init // add submodule after normal cloning
|
||||
git submodule update --remote // update submodules
|
||||
```
|
||||
|
||||
Build the image
|
||||
|
||||
```bash
|
||||
docker compose build # build the image
|
||||
docker compose build --no-cache # Force rebuild without cache
|
||||
```
|
||||
|
||||
### Run [with docker]
|
||||
|
||||
Run the script
|
||||
|
||||
```bash
|
||||
docker compose run --rm app python main.py # run pytest-abra
|
||||
docker compose run --rm -it app /bin/bash # debug the container
|
||||
```
|
||||
|
||||
Force rebuild with cache
|
||||
|
||||
```bash
|
||||
docker-compose up --build
|
||||
```
|
||||
|
||||
Force rebuild without cache
|
||||
|
||||
```bash
|
||||
docker-compose build --no-cache
|
||||
```
|
||||
|
||||
## Playwright Debug & Codegen
|
||||
|
||||
Use playwright debug mode or codegen to create testing code easily by recording browser actions https://playwright.dev/python/docs/codegen
|
||||
|
||||
```bash
|
||||
abratest --debug # launch your tests in debug mode
|
||||
playwright codegen demo.playwright.dev/todomvc # visit given url in codegen mode
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
pytest # test pytest-abra
|
||||
pytest -m "not slow" # test pytest-abra without slow tests
|
||||
pytest --collect-only # debug test pytest-abra
|
||||
docker compose run --rm app pytest # run pytest-abra
|
||||
docker compose run --rm -it app /bin/bash # use the container interactively
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,15 +1,161 @@
|
|||
# RUN
|
||||
# pytest-abra
|
||||
|
||||
Pytest-Abra is an installable python package baed on pytest, designed to test instances created with [abra](https://docs.coopcloud.tech/abra/). After installation, you will have two things:
|
||||
|
||||
- `abratest` CLI command. *Used to initialize the testing.*
|
||||
|
||||
- `pytest-abra` Pytest plugin. *Automatically loads custom fixtures in any pytest run (see `pytest_abra/custom_fixtures.py`)*
|
||||
|
||||
# Getting Started
|
||||
|
||||
Pytest-abra can easily be installed on any system but also offers a Docker image. To use pytest-abra, follow these steps:
|
||||
|
||||
## Usage [without Docker]
|
||||
|
||||
### Installation [without Docker]
|
||||
|
||||
To clone with submodules, use these git commands:
|
||||
|
||||
```bash
|
||||
git clone --recurse-submodules <repository>
|
||||
// optional:
|
||||
git submodule update --init // add submodule after normal cloning
|
||||
git submodule update --remote // update submodules
|
||||
```
|
||||
|
||||
Create a python environment and install all dependencies via
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
playwright install
|
||||
```
|
||||
|
||||
### Run [without Docker]
|
||||
|
||||
Run the helper script or directly use the cli command (see docs)
|
||||
|
||||
```bash
|
||||
python main.py # run pytest-abra
|
||||
abratest [options]
|
||||
```
|
||||
|
||||
## Usage [with docker]
|
||||
|
||||
### Installation [with docker]
|
||||
|
||||
To clone with submodules, use these git commands:
|
||||
|
||||
```bash
|
||||
git clone --recurse-submodules <repository>
|
||||
// optional:
|
||||
git submodule update --init // add submodule after normal cloning
|
||||
git submodule update --remote // update submodules
|
||||
```
|
||||
|
||||
Build the image
|
||||
|
||||
```bash
|
||||
docker compose build # build the image
|
||||
docker compose build --no-cache # Force rebuild without cache
|
||||
```
|
||||
|
||||
### Run [with docker]
|
||||
|
||||
Run the script
|
||||
|
||||
```bash
|
||||
docker compose run --rm app python main.py # run pytest-abra
|
||||
docker compose run --rm -it app /bin/bash # use the container interactively
|
||||
```
|
||||
|
||||
# Documentation
|
||||
|
||||
After Installation, `abratest` can be called via terminal:
|
||||
|
||||
```bash
|
||||
abratest [arguments]
|
||||
```
|
||||
|
||||
To run successfully, very specific arguments are required. The easiest way to use `abratest` is with the helper script `main.py`. Of yourse you can implement a similar helper script in the language of your liking.
|
||||
|
||||
## CLI Interface
|
||||
|
||||
The cli command `abratest` has 3 **required arguments**:
|
||||
|
||||
- `--env_paths ENV_PATHS`: list of the .env files used in the test
|
||||
- `--recipes_dir RECIPES_DIR`: directory of all available abra recipes
|
||||
- `--output_dir OUTPUT_DIR`: target directory for all test results
|
||||
|
||||
Furtheremore, there are these optional arguments:
|
||||
|
||||
- `--resume`: `abratest` will take the directory in `output_dir` with the most recent creation date and resume the tests there.
|
||||
- `--session_id SESSION_ID`: Instead of generating a new session_id, the given session_id is used to run or resume the test. Overwrites --resume to False.
|
||||
- `--debug`: enables playwright debug mode, see docs [here](https://playwright.dev/python/docs/running-tests#debugging-tests)
|
||||
- `--timeout`: will overwrite the default playwright timeouts in [ms], see docs [here](https://playwright.dev/python/docs/api/class-browsercontext#browser-context-set-default-timeout) and [here](https://playwright.dev/python/docs/test-assertions#global-timeout). In our current setup, some tests can fail at 10s but will pass with 20s.
|
||||
|
||||
### env_paths [required | string]
|
||||
|
||||
The .env files provied through the `--env_paths` argument are the most important input to abratest, as they serve as configuration for the tests. One or more paths pointing at .env files can be provided, multiple paths are separated with ";". These .env files are actually the same files that are used to configure the `abra` recipes for instance creation.
|
||||
|
||||
To run `abratest` with these `.env` configuration files
|
||||
|
||||
- `/path/config_1.env` [of TYPE authentik]
|
||||
- `/path/config_2.env` [of TYPE wordpress]
|
||||
- `/path/config_3.env` [of TYPE wordpress]
|
||||
|
||||
we simply call
|
||||
|
||||
```
|
||||
abratest --env_paths /path/config_1.env;/path/config_2.env;/path/config_3.env [...other args]
|
||||
```
|
||||
|
||||
Under the hood, each `.env` file in `--env_paths` will create one instance of a `Runner` subclass. Let's say we have `config_2.env` containing `TYPE=wordpress`. This will create an instance of `RunnerWordpress`. This class has to be imported from `recipes_dir`.
|
||||
|
||||
### recipes_dir [required | string]
|
||||
|
||||
The required argument `--recipes_dir` has to point to the directory, where all the abra recipes are stored. We can call `abratest` with
|
||||
|
||||
```
|
||||
abratest --recipes_dir /path/to/abra/recipes
|
||||
```
|
||||
|
||||
The expected dir structure inside of `recipes_dir` is as follows:
|
||||
|
||||
```
|
||||
DIR recipes_dir [contains abra recipes]
|
||||
│
|
||||
├── DIR authentik [authentik recipe]
|
||||
│ ├── [files from authentik recipe]
|
||||
│ └── DIR tests_authentik [pytest tests for authentik]
|
||||
│ ├── FILE runner_authentik.py # containing RunnerAuthentik class
|
||||
│ └── [pytest_files]
|
||||
│
|
||||
└── DIR wordpress [wordpress recipe]
|
||||
├── [files from wordpress recipe]
|
||||
└── DIR tests_wordpress [pytest tests for wordpress]
|
||||
├── FILE runner_wordpress.py # containing RunnerWordpress class
|
||||
└── [pytest_files]
|
||||
```
|
||||
|
||||
The class `RunnerWordpress` will be automatically imported using `importlib` library, which is equivalent to the code below. Note that `recipes_dir` will be added to sys.path automatically for the import to work and that every `Runner` class matching `recipes_dir.rglob("*/runner*.py")` will be imported.
|
||||
|
||||
```python
|
||||
from wordpress.tests_wordpress.runner_wordpress import RunnerWordpress
|
||||
```
|
||||
|
||||
### output_dir [required | string]
|
||||
|
||||
Path to the directory where all test outputs are stored (test report, tracebacks, playwright traces etc.)
|
||||
|
||||
```
|
||||
abratest --output_dir /path/to/output
|
||||
```
|
||||
|
||||
# Functionality
|
||||
|
||||
Abratest has 3 required inputs, but most importantly the test configuration is done through the .env files given with the --env_paths argument. So let's say we want to run abratest with these 3 .env files:
|
||||
|
||||
- config1.env [of TYPE authentik]
|
||||
- `config1.env` [of TYPE authentik]
|
||||
|
||||
- config2.env [of TYPE wordpress]
|
||||
|
||||
|
|
@ -117,3 +263,21 @@ DIR parent_dir
|
|||
The test files are written in the same way as any other pytest test file. The only difference is that pytest-abra provides custom fixtures that make it easy to get the configuration by the provided .env files and to deal with URLS etc.
|
||||
|
||||
# todo: add example
|
||||
|
||||
# Playwright Debug & Codegen
|
||||
|
||||
Use playwright debug mode or codegen to create testing code easily by recording browser actions https://playwright.dev/python/docs/codegen
|
||||
|
||||
```bash
|
||||
abratest --debug # launch your tests in debug mode
|
||||
playwright codegen demo.playwright.dev/todomvc # visit given url in codegen mode
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
pytest # test pytest-abra
|
||||
pytest -m "not slow" # test pytest-abra without slow tests
|
||||
pytest --collect-only # debug test pytest-abra
|
||||
docker compose run --rm app pytest # run pytest-abra
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue