e2e_tests/README.md

pytest-abra

Pytest-Abra is an installable python package design to test instances created with abra. After installation, you will have two things:

• abratest CLI command

• pytest-abra Pytest plugin

CLI (abratest)

The easiest way to call abratest is via the helper script in main.py. You can also directly call abratest via terminal, but you will have to make sure that the requirements below are met. To do that, you can call abratest with:

abratest [arguments]

The cli command abratest has 3 required arguments:

• --env_paths: list of the .env files used in the test
• --recipes_dir: directory of all available abra recipes
• --output_dir: target directory for all test results

It is also absolutely necessary, that python can import from recipes_dir. This can be achieved by adding the path to the PYTHONPATH environment variable before the python interpreter is started, i.e. before abratest is called.

env_paths [string]

The variable env_paths consists of one or more paths pointing at .env files. The paths are separated with ";". These .env files are actually configuration files for abra recipes, but pytest-abra uses the same files for test configuration.

To run abratest with these .env configuration files

/path/to/config_1.env
/path/to/config_2.env
/path/to/config_3.env

we simply call

abratest --env_paths /path/to/config_1.env;/path/to/config_2.env;/path/to/config_3.env

Under the hood, each .env file in --env_paths will create one instance of a Runner subclass. Let's say we have wordpress_configuration.env containing TYPE=wordpress. This will create an instance of RunnerWordpress. This class has to be imported from recipes_dir.

recipes_dir [string]

The required argument --recipes_dir has to point to the directory, where all the abra recipes are stored. We cann call abratest with

abratest --recipes_dir /path/to/abra/recipes

DIR recipes_dir [contains abra recipes]
│
├── DIR authentik [authentik recipe]
│   ├── [files from authentik recipe]
│   └── DIR tests_authentik [pytest tests for authentik]
│       ├── FILE runner_authentik.py  # containing RunnerAuthentik class
│       └── [pytest_files]
│
└── DIR wordpress [wordpress recipe]
    ├── [files from wordpress recipe]
    └── DIR tests_wordpress [pytest tests for wordpress]
        ├── FILE runner_wordpress.py  # containing RunnerWordpress class
        └── [pytest_files]

The class RunnerWordpress will be imported via

from wordpress.tests_wordpress.runner_wordpress import RunnerWordpress

which requires that python can find and import from the module wordpress. One way to achive this is by adding recipes_dir to the environment variable PYTHONPATH before i.e. abratest cannot do this itself -> workaround? todo

Usage

To use pytest-abra, follow these steps:

1. GIT clone [with & without Docker]

To clone with submodules, use these git commands:

git clone --recurse-submodules <repository>
// optional:
git submodule update --init    // add submodule after normal cloning
git submodule update --remote  // update submodules

Run

You can run pytest-abra with and without Docker. Choose now and follow the steps accordingly:

2.1 Run without Docker

Installation

Create a python environment and install all dependencies via

pip install -e .
playwright install

Run the script with

python main.py

2.2 Run with Docker

docker compose build  # build the image
docker compose run --rm app python main.py  # run pytest-abra
docker compose run --rm -it app /bin/bash  # debug the container

Force rebuild with cache

docker-compose up --build

Force rebuild without cache

docker-compose build --no-cache

Codegen

Use playwright codegen to create code for new testes easily https://playwright.dev/python/docs/codegen

playwright codegen demo.playwright.dev/todomvc

Development

pytest  # test pytest-abra
pytest  --collect-only  # debug test pytest-abra
docker compose run --rm app pytest  # run pytest-abra