| Name | scriptman JSON |
| Version |
2.9.556
JSON |
| download |
| home_page | None |
| Summary | A powerful Python package for automation, script management, and workflow orchestration. |
| upload_time | 2025-07-09 12:21:21 |
| maintainer | None |
| docs_url | None |
| author | Nelson Ombuya |
| requires_python | <4.0,>=3.12 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Scriptman
A powerful Python automation framework for web scraping, data processing, and task orchestration.
## Features
- 🚀 **Selenium Automation**: Cross-platform browser automation with intelligent fallback mechanisms
- 📊 **ETL Operations**: Extract, Transform, Load data processing with pandas integration
- 🗄️ **Database Support**: Multi-database support with SQLAlchemy and pyodbc
- ⏰ **Task Scheduling**: Advanced task scheduling and resource management
- 🔄 **Retry Mechanisms**: Robust retry logic with exponential backoff
- 🧹 **Cleanup Utilities**: Automatic cleanup of temporary files and caches
- ⚙️ **Configuration Management**: Flexible configuration with TOML support
## Quick Start
### Installation
```bash
pip install scriptman
```
### Basic Usage
```python
from scriptman.powers.selenium import SeleniumInstance
# Initialize Selenium with automatic downloads directory
selenium = SeleniumInstance()
# Navigate to a website
selenium.driver.get("https://example.com")
# Download a file (will be saved to your system's Downloads folder)
# ... download logic here ...
# Wait for download to complete
downloaded_file = selenium.wait_for_downloads_to_finish()
print(f"File downloaded to: {downloaded_file}")
```
## Downloads Directory
Scriptman automatically uses your system's default Downloads directory:
- **Windows**: `C:\Users\<username>\Downloads` (with OneDrive fallback)
- **macOS**: `/Users/<username>/Downloads`
- **Linux**: `/home/<username>/Downloads`
### Smart Download Handling
Scriptman uses an intelligent download mechanism that:
1. **Lets Chrome use its default download directory** (usually your Downloads folder)
2. **Monitors for completed downloads** in Chrome's default location
3. **Automatically moves files** to your configured downloads directory
4. **Handles filename conflicts** by adding counters to duplicate names
This approach prevents download issues that can occur when forcing Chrome to use a specific download directory.
### Configuration
You can customize the downloads directory in your configuration:
```toml
# scriptman.toml
[scriptman]
downloads_dir = "/path/to/custom/downloads"
```
### File Organization
- **Selenium Downloads**: Files downloaded through Selenium are saved to the configured directory
- **Chrome/Selenium Files**: Browser executables are stored in `.selenium/chrome/` subdirectory
- **Temporary Files**: Fallback to system temp directory if Downloads is not writable
## Selenium Features
### Cross-Platform Support
Scriptman's Selenium implementation includes:
- **Automatic Browser Management**: Downloads and manages Chrome/ChromeDriver
- **Fallback Mechanisms**: Graceful handling of permission issues
- **Headless Mode**: Optimized for server environments
- **Download Monitoring**: Automatic detection and relocation of completed downloads
### Example: Web Scraping
```python
from scriptman.powers.selenium import SeleniumInstance
selenium = SeleniumInstance()
# Navigate and interact
selenium.driver.get("https://example.com")
selenium.interact_with_element("//button[@id='download']", mode="click")
# Wait for download and get file path
file_path = selenium.wait_for_downloads_to_finish("report.pdf")
print(f"Downloaded: {file_path}")
```
### Download Process Flow
1. **Chrome downloads** to its default directory (usually Downloads)
2. **Scriptman monitors** the default directory for new files
3. **File detection** occurs when download completes
4. **Automatic move** to configured directory
5. **Filename conflict resolution** if needed
6. **Return final path** in configured directory
## ETL Operations
```python
from scriptman.powers.etl import ETL
# Load data from various sources
data = ETL.from_csv("data.csv")
data = ETL.from_json("data.json")
data = ETL.from_db(database_handler, "SELECT * FROM table")
# Transform data
transformed = data.filter(lambda x: x['status'] == 'active')
transformed = transformed.to_snake_case()
# Save results
transformed.to_csv("output.csv")
transformed.to_db(database_handler, "output_table")
```
## Configuration
Scriptman uses TOML configuration files:
```toml
# scriptman.toml
[scriptman]
# Downloads directory (defaults to system Downloads folder)
downloads_dir = "~/Downloads"
# Selenium settings
selenium_optimizations = true
selenium_headless = true
selenium_local_mode = true
# Logging
log_level = "INFO"
# Task settings
concurrent = true
retries = 3
task_timeout = 30
```
## Advanced Features
### Task Scheduling
```python
from scriptman.powers.scheduler import TaskScheduler
scheduler = TaskScheduler()
# Schedule a daily task
scheduler.add_daily_task(
"daily_report",
task_function,
hour=9,
minute=0
)
# Schedule a periodic task
scheduler.add_periodic_task(
"data_sync",
sync_function,
interval_minutes=30
)
```
### Database Operations
```python
from scriptman.powers.database import DatabaseHandler
# Connect to database
db = DatabaseHandler(
connection_string="sqlite:///data.db"
)
# Execute queries
results = db.execute_read_query("SELECT * FROM users")
db.execute_write_query("INSERT INTO logs VALUES (?)", ["log_entry"])
```
### Cleanup Utilities
```python
from scriptman.powers.cleanup import CleanUp
cleaner = CleanUp()
# Clean up various resources
cleaner.cleanup() # General cleanup
cleaner.selenium_cleanup() # Selenium downloads
cleaner.diskcache_cleanup() # Cache files
```
## Development
### Installation for Development
```bash
git clone <repository>
cd scriptman
pip install -e ".[dev]"
```
### Running Tests
```bash
pytest
```
## License
[Add your license information here]
## Contributing
[Add contribution guidelines here]
Raw data
{
"_id": null,
"home_page": null,
"name": "scriptman",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": null,
"author": "Nelson Ombuya",
"author_email": "developer@incognitouser.anonaddy.me",
"download_url": "https://files.pythonhosted.org/packages/fa/1c/270075d8d24026bb34fe1e4d6dfd0820a932d25aea8196e8c2d046745939/scriptman-2.9.556.tar.gz",
"platform": null,
"description": "# Scriptman\n\nA powerful Python automation framework for web scraping, data processing, and task orchestration.\n\n## Features\n\n- \ud83d\ude80 **Selenium Automation**: Cross-platform browser automation with intelligent fallback mechanisms\n- \ud83d\udcca **ETL Operations**: Extract, Transform, Load data processing with pandas integration\n- \ud83d\uddc4\ufe0f **Database Support**: Multi-database support with SQLAlchemy and pyodbc\n- \u23f0 **Task Scheduling**: Advanced task scheduling and resource management\n- \ud83d\udd04 **Retry Mechanisms**: Robust retry logic with exponential backoff\n- \ud83e\uddf9 **Cleanup Utilities**: Automatic cleanup of temporary files and caches\n- \u2699\ufe0f **Configuration Management**: Flexible configuration with TOML support\n\n## Quick Start\n\n### Installation\n\n```bash\npip install scriptman\n```\n\n### Basic Usage\n\n```python\nfrom scriptman.powers.selenium import SeleniumInstance\n\n# Initialize Selenium with automatic downloads directory\nselenium = SeleniumInstance()\n\n# Navigate to a website\nselenium.driver.get(\"https://example.com\")\n\n# Download a file (will be saved to your system's Downloads folder)\n# ... download logic here ...\n\n# Wait for download to complete\ndownloaded_file = selenium.wait_for_downloads_to_finish()\nprint(f\"File downloaded to: {downloaded_file}\")\n```\n\n## Downloads Directory\n\nScriptman automatically uses your system's default Downloads directory:\n\n- **Windows**: `C:\\Users\\<username>\\Downloads` (with OneDrive fallback)\n- **macOS**: `/Users/<username>/Downloads`\n- **Linux**: `/home/<username>/Downloads`\n\n### Smart Download Handling\n\nScriptman uses an intelligent download mechanism that:\n\n1. **Lets Chrome use its default download directory** (usually your Downloads folder)\n2. **Monitors for completed downloads** in Chrome's default location\n3. **Automatically moves files** to your configured downloads directory\n4. **Handles filename conflicts** by adding counters to duplicate names\n\nThis approach prevents download issues that can occur when forcing Chrome to use a specific download directory.\n\n### Configuration\n\nYou can customize the downloads directory in your configuration:\n\n```toml\n# scriptman.toml\n[scriptman]\ndownloads_dir = \"/path/to/custom/downloads\"\n```\n\n### File Organization\n\n- **Selenium Downloads**: Files downloaded through Selenium are saved to the configured directory\n- **Chrome/Selenium Files**: Browser executables are stored in `.selenium/chrome/` subdirectory\n- **Temporary Files**: Fallback to system temp directory if Downloads is not writable\n\n## Selenium Features\n\n### Cross-Platform Support\n\nScriptman's Selenium implementation includes:\n\n- **Automatic Browser Management**: Downloads and manages Chrome/ChromeDriver\n- **Fallback Mechanisms**: Graceful handling of permission issues\n- **Headless Mode**: Optimized for server environments\n- **Download Monitoring**: Automatic detection and relocation of completed downloads\n\n### Example: Web Scraping\n\n```python\nfrom scriptman.powers.selenium import SeleniumInstance\n\nselenium = SeleniumInstance()\n\n# Navigate and interact\nselenium.driver.get(\"https://example.com\")\nselenium.interact_with_element(\"//button[@id='download']\", mode=\"click\")\n\n# Wait for download and get file path\nfile_path = selenium.wait_for_downloads_to_finish(\"report.pdf\")\nprint(f\"Downloaded: {file_path}\")\n```\n\n### Download Process Flow\n\n1. **Chrome downloads** to its default directory (usually Downloads)\n2. **Scriptman monitors** the default directory for new files\n3. **File detection** occurs when download completes\n4. **Automatic move** to configured directory\n5. **Filename conflict resolution** if needed\n6. **Return final path** in configured directory\n\n## ETL Operations\n\n```python\nfrom scriptman.powers.etl import ETL\n\n# Load data from various sources\ndata = ETL.from_csv(\"data.csv\")\ndata = ETL.from_json(\"data.json\")\ndata = ETL.from_db(database_handler, \"SELECT * FROM table\")\n\n# Transform data\ntransformed = data.filter(lambda x: x['status'] == 'active')\ntransformed = transformed.to_snake_case()\n\n# Save results\ntransformed.to_csv(\"output.csv\")\ntransformed.to_db(database_handler, \"output_table\")\n```\n\n## Configuration\n\nScriptman uses TOML configuration files:\n\n```toml\n# scriptman.toml\n[scriptman]\n# Downloads directory (defaults to system Downloads folder)\ndownloads_dir = \"~/Downloads\"\n\n# Selenium settings\nselenium_optimizations = true\nselenium_headless = true\nselenium_local_mode = true\n\n# Logging\nlog_level = \"INFO\"\n\n# Task settings\nconcurrent = true\nretries = 3\ntask_timeout = 30\n```\n\n## Advanced Features\n\n### Task Scheduling\n\n```python\nfrom scriptman.powers.scheduler import TaskScheduler\n\nscheduler = TaskScheduler()\n\n# Schedule a daily task\nscheduler.add_daily_task(\n \"daily_report\",\n task_function,\n hour=9,\n minute=0\n)\n\n# Schedule a periodic task\nscheduler.add_periodic_task(\n \"data_sync\",\n sync_function,\n interval_minutes=30\n)\n```\n\n### Database Operations\n\n```python\nfrom scriptman.powers.database import DatabaseHandler\n\n# Connect to database\ndb = DatabaseHandler(\n connection_string=\"sqlite:///data.db\"\n)\n\n# Execute queries\nresults = db.execute_read_query(\"SELECT * FROM users\")\ndb.execute_write_query(\"INSERT INTO logs VALUES (?)\", [\"log_entry\"])\n```\n\n### Cleanup Utilities\n\n```python\nfrom scriptman.powers.cleanup import CleanUp\n\ncleaner = CleanUp()\n\n# Clean up various resources\ncleaner.cleanup() # General cleanup\ncleaner.selenium_cleanup() # Selenium downloads\ncleaner.diskcache_cleanup() # Cache files\n```\n\n## Development\n\n### Installation for Development\n\n```bash\ngit clone <repository>\ncd scriptman\npip install -e \".[dev]\"\n```\n\n### Running Tests\n\n```bash\npytest\n```\n\n## License\n\n[Add your license information here]\n\n## Contributing\n\n[Add contribution guidelines here]\n",
"bugtrack_url": null,
"license": null,
"summary": "A powerful Python package for automation, script management, and workflow orchestration.",
"version": "2.9.556",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2b1ca04073abde1cf74cde94d51b454d0f03f6c6ffe1010f388c1767afb2b195",
"md5": "603e33452ab2ea55abb05d8a807c6177",
"sha256": "3fca3ee06ae64108a165aa39f8672caab29ac3414da60dc3d390330170453eff"
},
"downloads": -1,
"filename": "scriptman-2.9.556-py3-none-any.whl",
"has_sig": false,
"md5_digest": "603e33452ab2ea55abb05d8a807c6177",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 139177,
"upload_time": "2025-07-09T12:21:19",
"upload_time_iso_8601": "2025-07-09T12:21:19.375743Z",
"url": "https://files.pythonhosted.org/packages/2b/1c/a04073abde1cf74cde94d51b454d0f03f6c6ffe1010f388c1767afb2b195/scriptman-2.9.556-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fa1c270075d8d24026bb34fe1e4d6dfd0820a932d25aea8196e8c2d046745939",
"md5": "5979b332e4a65dd7cd62015beef18842",
"sha256": "48d4cf0d0e732e44d8a7a11c5f92c6cd68e65e6a666bc883c3f0d77d0cac78ba"
},
"downloads": -1,
"filename": "scriptman-2.9.556.tar.gz",
"has_sig": false,
"md5_digest": "5979b332e4a65dd7cd62015beef18842",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 106552,
"upload_time": "2025-07-09T12:21:21",
"upload_time_iso_8601": "2025-07-09T12:21:21.004734Z",
"url": "https://files.pythonhosted.org/packages/fa/1c/270075d8d24026bb34fe1e4d6dfd0820a932d25aea8196e8c2d046745939/scriptman-2.9.556.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-09 12:21:21",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "scriptman"
}
