# SPL2 Testing Framework
## Overview
The **SPL2 Testing Framework** enables running SPL2 tests both locally ( or on any Splunk instance with SPL2
orchestrator), remotely (using external cloud environments) or using cli.
- **For Cloud**: It uses
the Search Service API.
- **For Splunk**: It uses the Splunk Search API (with SPL2 support)
- **For cli** - it uses the internal spl2-processor-cli library. This option is not available for public usage, as
spl2-processor-cli is a Splunk internal tool.
## Prerequisites
### 1. Install Python and Poetry
1. Ensure `python3.x` is available
2. Install testing framework:
- Install `poetry` and execute the command `poetry install` to create a virtual environment and install required
dependencies.
- OR
- Install the library using pip:
`pip install spl2-testing-framework`
### 2. Set configuration. It may be done in spl2_test_config.json file or environment variables
Note: setting configuration is necessary for running tests using splunk or cloud environment.
For running tests using cli no configuration is required.
The only requirement is to have this library installed, as described below
#### spl2_test_config.json - local file present in the current working directory (the directory from where tests are executed)
##### Configuration for running tests using cloud search client (Ingest processor)
* `cloud_instance` - address of Cloud host where the tests can be executed
- e.g.: `staging.scs.splunk.com`
* `tenant` - tenant to use for testing
- e.g.: `spl2-content`
* `bearer_token` - token used for authentication. To obtain the token, go
to: `https://console.[cloud_instance]/[tenant]/settings`
##### Configuration for running tests using splunk search client (Splunk instance)
* `host` - address of Splunk host where the tests can be executed
- e.g.: `localhost` or `https://10.202.35.219`
* `port` - port of Splunk host where the tests can be executed
- usually `8089`, but can be different
* `user` - user to authenticate
* `password` - password to authenticate
#### The same configuration can be done using environment variables(however, spl2_test_config.json has higher priority):
##### cloud search client
* `SPL2_TF_CLOUD_INSTANCE` => `cloud_instance`
* `SPL2_TF_TENANT` => `tenant`
* `SPL2_TF_BEARER_TOKEN` => `bearer_token`
##### splunk search client
* `SPL2_TF_HOST` => `host`
* `SPL2_TF_PORT` => `port`
* `SPL2_TF_USER` => `user`
* `SPL2_TF_PASSWORD` => `password`
### 3. Installing spl2-processor-cli (Splunk internal tool)
`spl2-processor-cli` can be installed using brew:
`brew install spl2-processor-cli`
Before installation, it may be necessary to authenticate to artifactory by running:
`okta-artifactory-login -t generic`
## Running tests
To run tests, execute the command:
`spl2_tests_run [cli|splunk|cloud]`
In the directory where the tests are located.
Test discovery is recursive, so it's possible to run tests even from the root directory of the project.
It is possible to pass more options to the command, which works also with pytest, e.g.:
* `-k "filter"` - to run only tests which name contains "filter"
* `-v[vv]` - to see more verbose output
* `-n [auto|<number>]` - to run tests in parallel
- `auto` - to use all available cores,
- `<number>` - to use specific number of cores
- however, it's recommended to run tests in parallel on cli mostly, as running on splunk or cloud doesn't give
significant performance improvement
* `-x` - to stop on first failure
* `-pdb` - to enter debugger on failure
* ... and much more, whatever is supported by pytest
Note: The `pytest.ini.sample` file allows you to define command parameters. Just update the configurations, rename the
file by removing the `.sample` extension, and execute the command.
### Run tests in IDE [PyCharm]
It's also possible to run tests in PyCharm. To do this, it's necessary to set `Run Configurations`
Sample configuration which may be used:
* Run configuration
- Type: `Python test`
- Module: `spl2_testing_framework.test_runner`
- Parameters:
`--type [cli | splunk | cloud] --test_dir /tests/resources -o log_cli=true --log-cli-level=INFO --verbose`
- If test dir is not specified, current working directory will be used
- If necessary another pytest options can be added
Note: It's necessary to set "pytest" as default test runner in PyCharm settings
## Executing a spl2 file
This framework also supports executing a single spl2 file and prints the results in command line as well as a log file.
This will help developers to get the results of the spl2 pipeline as and when they are developing the pipeline.
It requires 3 additional parameters:
* --template_file
* --sample_file
* --sample_delimiter
It will execute the template_file provided in the `--template_file` parameter. It will read samples if `--sample_file`
parameter is provided and will separate the samples by using `--sample_delimiter`. If `--sample_file` is not provided,
then it will look for the samples in the respective `module.json` file corresponding to the template_file.
To run a single spl2 file, execute the command:
`single_spl2_file_run [cli|splunk|cloud]`
It is possible to pass more options to the command, which works also with pytest, e.g.:
* `--test_dir` - Path in which the template file and module.json are available. If not provided, it will look for the
current directory for the template file and module.json file
* `--template_file` - The spl2 template file to execute
* `--sample_file` - A file containing all the samples required for the template file. If not provided, it will look for
the samples in module.json file of the corresponding template file
* `--sample_delimiter` - Separator for separating the samples provided in the sample file. If not provided, it will use
newline as a default separator.
* ... and much more, whatever is supported by pytest
Note: The `pytest.ini.sample` file allows you to define command parameters. Just update the configurations, rename the
file by removing the `.sample` extension, and execute the command.
### Performance check
It is possible to measure execution time of spl2 pipeline, or even do more advanced time checks using flag:
* `--performance_check=time` - to run basic time measurements - time of execution of spl2 pipeline will be printed to
stdout
* `--performance_check=detailed_time` - to do more advanced time checks which injects more timestamps into spl2
pipeline.
Running `detailed_time` check creates text file with spl2 pipeline code with injected timestamps after every
command ("|")
Content of this file will also be printed to stdout.
This checks can be applied only to box tests, as assertions which are used in unit tests may impact spl2 pipeline
performance.
Raw data
{
"_id": null,
"home_page": "https://cd.splunkdev.com/data-availability/spl2-content-tools",
"name": "spl2-testing-framework",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.7",
"maintainer_email": null,
"keywords": "SPL2 testing framework",
"author": "mgazda-splunk",
"author_email": "mgazda@splunk.com",
"download_url": "https://files.pythonhosted.org/packages/06/07/3595bd796ba3e7691ce5e7a2da81f0007a2d8180c212edf997711610620f/spl2_testing_framework-1.0.1.tar.gz",
"platform": null,
"description": "# SPL2 Testing Framework\n\n## Overview\n\nThe **SPL2 Testing Framework** enables running SPL2 tests both locally ( or on any Splunk instance with SPL2\norchestrator), remotely (using external cloud environments) or using cli.\n\n- **For Cloud**: It uses\n the Search Service API.\n- **For Splunk**: It uses the Splunk Search API (with SPL2 support)\n- **For cli** - it uses the internal spl2-processor-cli library. This option is not available for public usage, as\n spl2-processor-cli is a Splunk internal tool.\n\n## Prerequisites\n\n### 1. Install Python and Poetry\n\n1. Ensure `python3.x` is available\n2. Install testing framework:\n - Install `poetry` and execute the command `poetry install` to create a virtual environment and install required\n dependencies.\n - OR\n - Install the library using pip:\n `pip install spl2-testing-framework`\n\n### 2. Set configuration. It may be done in spl2_test_config.json file or environment variables\n\nNote: setting configuration is necessary for running tests using splunk or cloud environment.\nFor running tests using cli no configuration is required.\nThe only requirement is to have this library installed, as described below\n\n#### spl2_test_config.json - local file present in the current working directory (the directory from where tests are executed)\n\n##### Configuration for running tests using cloud search client (Ingest processor)\n\n* `cloud_instance` - address of Cloud host where the tests can be executed\n - e.g.: `staging.scs.splunk.com`\n* `tenant` - tenant to use for testing\n - e.g.: `spl2-content`\n* `bearer_token` - token used for authentication. To obtain the token, go\n to: `https://console.[cloud_instance]/[tenant]/settings`\n\n##### Configuration for running tests using splunk search client (Splunk instance)\n\n* `host` - address of Splunk host where the tests can be executed\n - e.g.: `localhost` or `https://10.202.35.219`\n* `port` - port of Splunk host where the tests can be executed\n - usually `8089`, but can be different\n* `user` - user to authenticate\n* `password` - password to authenticate\n\n#### The same configuration can be done using environment variables(however, spl2_test_config.json has higher priority):\n\n##### cloud search client\n\n* `SPL2_TF_CLOUD_INSTANCE` => `cloud_instance`\n* `SPL2_TF_TENANT` => `tenant`\n* `SPL2_TF_BEARER_TOKEN` => `bearer_token`\n\n##### splunk search client\n\n* `SPL2_TF_HOST` => `host`\n* `SPL2_TF_PORT` => `port`\n* `SPL2_TF_USER` => `user`\n* `SPL2_TF_PASSWORD` => `password`\n\n### 3. Installing spl2-processor-cli (Splunk internal tool)\n\n`spl2-processor-cli` can be installed using brew:\n\n`brew install spl2-processor-cli`\n\nBefore installation, it may be necessary to authenticate to artifactory by running:\n\n`okta-artifactory-login -t generic`\n\n## Running tests\n\nTo run tests, execute the command:\n\n`spl2_tests_run [cli|splunk|cloud]`\n\nIn the directory where the tests are located.\nTest discovery is recursive, so it's possible to run tests even from the root directory of the project.\n\nIt is possible to pass more options to the command, which works also with pytest, e.g.:\n\n* `-k \"filter\"` - to run only tests which name contains \"filter\"\n* `-v[vv]` - to see more verbose output\n* `-n [auto|<number>]` - to run tests in parallel\n - `auto` - to use all available cores,\n - `<number>` - to use specific number of cores\n - however, it's recommended to run tests in parallel on cli mostly, as running on splunk or cloud doesn't give\n significant performance improvement\n* `-x` - to stop on first failure\n* `-pdb` - to enter debugger on failure\n* ... and much more, whatever is supported by pytest\n\nNote: The `pytest.ini.sample` file allows you to define command parameters. Just update the configurations, rename the\nfile by removing the `.sample` extension, and execute the command.\n\n### Run tests in IDE [PyCharm]\n\nIt's also possible to run tests in PyCharm. To do this, it's necessary to set `Run Configurations`\n\nSample configuration which may be used:\n\n* Run configuration\n - Type: `Python test`\n - Module: `spl2_testing_framework.test_runner`\n - Parameters:\n `--type [cli | splunk | cloud] --test_dir /tests/resources -o log_cli=true --log-cli-level=INFO --verbose`\n - If test dir is not specified, current working directory will be used\n - If necessary another pytest options can be added\n\nNote: It's necessary to set \"pytest\" as default test runner in PyCharm settings\n\n## Executing a spl2 file\n\nThis framework also supports executing a single spl2 file and prints the results in command line as well as a log file.\nThis will help developers to get the results of the spl2 pipeline as and when they are developing the pipeline.\n\nIt requires 3 additional parameters:\n\n* --template_file\n* --sample_file\n* --sample_delimiter\n\nIt will execute the template_file provided in the `--template_file` parameter. It will read samples if `--sample_file`\nparameter is provided and will separate the samples by using `--sample_delimiter`. If `--sample_file` is not provided,\nthen it will look for the samples in the respective `module.json` file corresponding to the template_file.\n\nTo run a single spl2 file, execute the command:\n\n`single_spl2_file_run [cli|splunk|cloud]`\n\nIt is possible to pass more options to the command, which works also with pytest, e.g.:\n\n* `--test_dir` - Path in which the template file and module.json are available. If not provided, it will look for the\n current directory for the template file and module.json file\n* `--template_file` - The spl2 template file to execute\n* `--sample_file` - A file containing all the samples required for the template file. If not provided, it will look for\n the samples in module.json file of the corresponding template file\n* `--sample_delimiter` - Separator for separating the samples provided in the sample file. If not provided, it will use\n newline as a default separator.\n\n* ... and much more, whatever is supported by pytest\n\nNote: The `pytest.ini.sample` file allows you to define command parameters. Just update the configurations, rename the\nfile by removing the `.sample` extension, and execute the command.\n\n### Performance check\n\nIt is possible to measure execution time of spl2 pipeline, or even do more advanced time checks using flag:\n\n* `--performance_check=time` - to run basic time measurements - time of execution of spl2 pipeline will be printed to\n stdout\n* `--performance_check=detailed_time` - to do more advanced time checks which injects more timestamps into spl2\n pipeline.\n\nRunning `detailed_time` check creates text file with spl2 pipeline code with injected timestamps after every\ncommand (\"|\")\nContent of this file will also be printed to stdout.\n\nThis checks can be applied only to box tests, as assertions which are used in unit tests may impact spl2 pipeline\nperformance. ",
"bugtrack_url": null,
"license": null,
"summary": "SPL2 Testing Framework",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://cd.splunkdev.com/data-availability/spl2-content-tools",
"Repository": "https://cd.splunkdev.com/data-availability/spl2-content-tools"
},
"split_keywords": [
"spl2",
"testing",
"framework"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bcd86641b4d50641215e425dfd90fe588fd59b94c79317fc8523d46d70ab69b3",
"md5": "9e04b5ecd06ca8c8ffdb6da177c2b78d",
"sha256": "3af8b2510c9fa25b8cb916900b4da00630fbde24a241ea7aabc7e6bf20cb2daf"
},
"downloads": -1,
"filename": "spl2_testing_framework-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9e04b5ecd06ca8c8ffdb6da177c2b78d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.7",
"size": 43377,
"upload_time": "2025-07-11T12:46:11",
"upload_time_iso_8601": "2025-07-11T12:46:11.879681Z",
"url": "https://files.pythonhosted.org/packages/bc/d8/6641b4d50641215e425dfd90fe588fd59b94c79317fc8523d46d70ab69b3/spl2_testing_framework-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "06073595bd796ba3e7691ce5e7a2da81f0007a2d8180c212edf997711610620f",
"md5": "254b78a195653ed7869119e8d8a2596a",
"sha256": "50a4fd549a8b993a0692f513d7631a188facc938cfcc33ff6f0d165448977a6d"
},
"downloads": -1,
"filename": "spl2_testing_framework-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "254b78a195653ed7869119e8d8a2596a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.7",
"size": 26954,
"upload_time": "2025-07-11T12:46:13",
"upload_time_iso_8601": "2025-07-11T12:46:13.139040Z",
"url": "https://files.pythonhosted.org/packages/06/07/3595bd796ba3e7691ce5e7a2da81f0007a2d8180c212edf997711610620f/spl2_testing_framework-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-11 12:46:13",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "spl2-testing-framework"
}
JSON