119 lines
No EOL
4.3 KiB
Markdown
119 lines
No EOL
4.3 KiB
Markdown
# RUN
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Abratest has 3 required inputs, but most importantly the test configuration is done through the .env files given with the --env_paths argument. So let's say we want to run abratest with these 3 .env files:
|
|
|
|
- config1.env [of TYPE authentik]
|
|
|
|
- config2.env [of TYPE wordpress]
|
|
|
|
- config3.env [of TYPE wordpress]
|
|
|
|
Now we run
|
|
|
|
```bash
|
|
abratest --env_paths path/config1.env;path/config2.env;path/config3.env [...other args]
|
|
```
|
|
|
|
|
|
```
|
|
abratest -> create Coordinator() instance
|
|
└── Coordinator() -> create Runner() subclass instances
|
|
├── RunnerAuthentik() [based on config1.env, loaded
|
|
│ │ from abra/recipes/authentik]
|
|
│ │ # RunnerAuthentik with 3 test files:
|
|
│ ├── RUN pytest path/setup_authentik.py
|
|
│ ├── RUN pytest path/test_authentik_1.py
|
|
│ └── RUN pytest path/test_authentik_2.py
|
|
├── RunnerWordpress() [based on config2.env, loaded
|
|
│ │ from abra/recipes/wordpress]
|
|
│ │ # RunnerWordpress with 1 test file
|
|
│ ├── RUN pytest path/setup_authentik.py
|
|
│ ├── RUN pytest path/test_authentik_1.py
|
|
│ └── RUN pytest path/test_authentik_2.py
|
|
└── RunnerWordpress() [based on config3.env, loaded
|
|
│ from abra/recipes/wordpress]
|
|
│ # RunnerWordpress with 1 test file
|
|
├── RUN pytest path/setup_authentik.py
|
|
├── RUN pytest path/test_authentik_1.py
|
|
└── RUN pytest path/test_authentik_2.py
|
|
|
|
|
|
```
|
|
|
|
Coordinator will take care of the correct order of the tests. In general, tests are placed in one of 3 categories: `setups`, `tests` and `cleanups`. To associate a test with one of these categories, place the Test in the corresponding list of the Runner class, i.e. Runner.setups = [test] or Runner.tests = [test]. The execution order will be.
|
|
|
|
> [setups] ➔ [tests] ➔ [cleanups]
|
|
|
|
|
|
Furthermore, some `Runner` classes can depend on others. For example, `RunnerWordpress` depends on `RunnerAuthentik`. Therefore, `Coordinator` will make sure that `RunnerAuthentik` runs before `RunnerWordpress`. We will end up with with this order:
|
|
|
|
| # | Runner | Type |
|
|
| --- | -------------- | -------- |
|
|
| 1. | Authentik | setups |
|
|
| 2. | Wordpress-1 | setups |
|
|
| 3. | Wordpress-2 | setups |
|
|
| 4. | Authentik | tests |
|
|
| 5. | Wordpress-1 | tests |
|
|
| 6. | Wordpress-2 | tests |
|
|
| 7. | Authentik | cleanups |
|
|
| 8. | Wordpress-1 | cleanups |
|
|
| 9. | Wordpress-2 | cleanups |
|
|
|
|
|
|
# Create a custom Runner
|
|
|
|
To comprehend the process of creating a new subclass of `Runner`, let's examine a simplified rendition of the `RunnerWordpress` class. Within it, there exist two setup scripts and two test scripts, one of which operates conditionally.
|
|
|
|
|
|
```python
|
|
from pytest_abra import Runner, Test
|
|
|
|
class RunnerWordpress(Runner):
|
|
env_type = "wordpress"
|
|
dependencies = ["authentik"]
|
|
setups = [
|
|
Test(test_file="setup_wordpress_1.py"),
|
|
Test(test_file="setup_wordpress_2.py"),
|
|
]
|
|
tests = [
|
|
Test(test_file="test_wordpress.py"),
|
|
Test(condition=condition_function, test_file="test_wordpress_conditional.py"),
|
|
]
|
|
cleanups = []
|
|
```
|
|
|
|
The signature of condition functions can be seen below. The function takes one `NamedTuple` and returns of type `bool`. You can learn about the contents of the input by looking up the class `ConditionArgs`. Generally speaking, it provides access to all of the .env files, especially the one related to the current Runner.
|
|
|
|
```python
|
|
def condition_function(args: ConditionArgs) -> bool:
|
|
...
|
|
```
|
|
|
|
## Discovery of `Runners` and `Tests`
|
|
|
|
- Runners will be discovered, if they are defined in a moduled of name `runner_*.py` including a class of name `Runner*`.
|
|
|
|
- Tests will be discovered by filename as long as they are placed in the parent dir of `runner_*.py` or in any subdirectory.
|
|
|
|
```
|
|
DIR parent_dir
|
|
├── FILE runner_*.py
|
|
├── FILE test1.py
|
|
└── DIR subdir
|
|
├── DIR subsubdir
|
|
│ └── test2.py
|
|
└── test3.py
|
|
```
|
|
|
|
# Create custom Tests
|
|
|
|
The test files are written in the same way as any other pytest test file. The only difference is that pytest-abra provides custom fixtures that make it easy to get the configuration by the provided .env files and to deal with URLS etc.
|
|
|
|
# todo: add example |