awsquery

Name	awsquery JSON
Version	0.5.0 JSON
	download
home_page	None
Summary	Advanced CLI tool for querying AWS APIs with flexible filtering and automatic parameter resolution
upload_time	2025-10-06 10:49:50
maintainer	None
docs_url	None
author	None
requires_python	>=3.8
license	MIT
keywords	aws cli query cloud devops boto3
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # awsquery - AWS API Query Tool

AWS CLI tool to run any awscli read command and filter the resulting Values and JSON for custom table output.

The following command will find all ec2 instances where `prod`and `web` are found somewhere in the Value
of any key of the whole response. For all matching resources it will extract all fields that match
any of the second filters, so anything that matches `Tags.Name`, `State`, `InstanceId` or `vpcid`.
```
awsquery ec2 describe-instances prod web -- Tags.Name State InstanceId vpcid
```

This creates endless flexibility to shape your aws cli calls, filter any output and present exactly
the data you need during review, debugging or development.

## Features

- **Smart Multi-Level Calls**: Automatically resolves missing parameters by inferring and calling list operations
- **Flexible Filtering**: Multi-level filtering with `--` separators for resource, value, and column filters
- **Partial Matching**: All filters use case-insensitive partial matching (both value and column filters)
- **Keys Discovery**: Show all available fields from any API response with `-k`/`--keys` (fixed to show keys from successful responses)
- **Debug Mode**: Comprehensive debug output with `-d`/`--debug` featuring structured output with timestamps and DebugContext tracking
- **Security Validation**: Enforces ReadOnly AWS operations with comprehensive validation
- **Smart Auto-completion**: Enhanced tab completion with split matching and prefix priority for AWS services and actions
- **Smart Parameter Extraction**: Handles both specific fields and standard AWS field patterns (Name, Id, Arn)
- **Intelligent Response Processing**: Clean extraction of list data, ignoring metadata
- **Tabular Output**: Customizable column display with automatic filtering
- **Pagination Support**: Handles large AWS responses automatically
- **Region/Profile Support**: AWS CLI-compatible `--region` and `--profile` arguments for session management
- **Tag Transformation**: Automatic conversion of AWS Tags list to key-value pairs for better readability
- **Default Column Filters**: Configuration-based default columns for common AWS queries
- **Parameter Passing**: Direct parameter passing to AWS APIs with `-p`/`--parameter` for advanced use cases
- **Hint-Based Resolution**: Function selection hints with `-i`/`--input` for multi-step calls, including cross-service support and field extraction targeting

## Installation

### Via pip (Recommended)

```bash
pip install awsquery
```

### Development Installation

```bash
git clone https://github.com/yourusername/awsquery.git
cd awsquery
pip install -e ".[dev]"
```

### Enable Shell Autocomplete

awsquery supports tab completion for AWS services and actions through argcomplete.

#### Setup

##### Bash
```bash
# Add to ~/.bashrc or ~/.bash_profile
eval "$(register-python-argcomplete awsquery)"
```

##### Zsh
```bash
# Add to ~/.zshrc
autoload -U bashcompinit && bashcompinit
eval "$(register-python-argcomplete awsquery)"
```

##### Fish
```bash
# Add to ~/.config/fish/config.fish
register-python-argcomplete --shell fish awsquery | source
```

After adding the appropriate line to your shell configuration, restart your shell or source the file:
```bash
source ~/.bashrc  # or ~/.zshrc, etc.
```

Now you can use enhanced tab completion with smart matching:
```bash
awsquery <TAB>              # Shows available services
awsquery ec2 <TAB>          # Shows available ec2 actions
awsquery s3 list-<TAB>      # Shows s3 list actions
awsquery ec2 desc-inst<TAB> # Smart completion: "desc-inst" matches "describe-instances"
awsquery cloudformation des-sta<TAB> # Matches "describe-stacks"
```

The autocomplete system now features:
- **Split matching**: "desc-inst" matches "describe-instances"
- **Prefix priority**: Exact prefix matches are prioritized over substring matches
- **Security filtering**: Only shows ReadOnly operations

## Usage

### Basic Query
```bash
# Query EC2 instances with filtering
awsquery ec2 describe-instances

# Filter by values (partial match, case-insensitive on ANY field)
awsquery ec2 describe-instances prod web

# Specify output columns (partial match, case-insensitive)
awsquery ec2 describe-instances prod -- Name State InstanceId

# List S3 buckets containing "backup" (matches if "backup" appears anywhere in any field)
awsquery s3 list-buckets backup

# JSON output format
awsquery -j s3 list-buckets
```

### Keys Discovery
```bash
# Show available fields/keys from an API response
awsquery -k ec2 describe-instances
awsquery --keys s3 list-buckets
```

### Discovery and Debug
```bash
# List available services
awsquery

# Debug mode for troubleshooting with enhanced DebugContext output
# Shows structured debug information with timestamps and execution flow
awsquery -d ec2 describe-instances
```

## Command Structure

```
awsquery [-j|--json] [-k|--keys] [-d|--debug] [-p PARAM] [-i HINT] [--region REGION] [--profile PROFILE] SERVICE ACTION [VALUE_FILTERS...] [-- TABLE_OUTPUT_FILTERS...]
```

- **SERVICE**: AWS service name (ec2, s3, iam, etc.)
- **ACTION**: Service action (describe-instances, list-buckets, etc.)
- **VALUE_FILTERS**: Space-separated filters - ALL must match, using case-insensitive partial matching on any field
- **TABLE_OUTPUT_FILTERS**: Column selection - partial, case-insensitive matching on column names
- **-j, --json**: Output results in JSON format instead of table
- **-k, --keys**: Show all available keys for the command
- **-d, --debug**: Enable debug output for troubleshooting
- **-p, --parameter PARAM**: Pass parameters directly to AWS API (key=value format, propagates to list operations)
- **-i, --input HINT**: Multi-step control with cross-service support and function/field/limit hints (e.g., "ec2:desc-inst:instanceid", "desc-clus", ":arn", "::5")
- **--region REGION**: AWS region to use for requests (e.g., us-west-2)
- **--profile PROFILE**: AWS profile to use from ~/.aws/credentials

## Security

- **ReadOnly Enforcement**: Only AWS ReadOnly operations are permitted
- **Input Sanitization**: Prevents injection attacks through parameter validation
- **Operation Validation**: All actions validated before execution
- **Wildcard Matching**: Supports AWS IAM wildcard patterns (e.g., `ec2:Describe*`)
- **Session Isolation**: Each profile/region maintains separate boto3 sessions
- **Security Testing**: Comprehensive test suite validates security constraints

## Examples

```bash
# Find instances with "prod" AND "web" in any field (partial match)
# e.g., matches "production-web-server", "web.prod.example.com"
awsquery ec2 describe-instances prod web

# Show only columns matching Name, State and InstanceId (partial match)
# e.g., "Name" matches "InstanceName", "Tags.Name", "SecurityGroupName"
awsquery ec2 describe-instances prod web -- Name State InstanceId

# Find S3 buckets with "backup" anywhere in the data
# e.g., matches "my-backup-bucket", "bucket-backup-2024", "backups"
awsquery s3 list-buckets backup

# Multi-level CloudFormation query with parameter resolution
# Filters: "prod" in stack data, columns containing "Created" or "StackName"
awsquery cloudformation describe-stack-events prod -- Created StackName

# Targeted field extraction with hint for multi-step calls
awsquery elbv2 describe-tags -i desc-clus:clusterarn prod

# Discover all available keys
awsquery -k ec2 describe-instances

# JSON output with filtering (partial column name matching)
awsquery -j ec2 describe-instances prod -- InstanceId State.Name

# Debug mode with enhanced output showing parameter resolution and API calls
awsquery -d cloudformation describe-stack-resources workers -- EKS

# Use specific AWS region
awsquery --region eu-west-1 ec2 describe-instances

# Use specific AWS profile
awsquery --profile production s3 list-buckets

# Combine region and profile
awsquery --region us-east-2 --profile dev ec2 describe-vpcs

# View transformed tags (partial match on column names)
# Shows any column containing "Tags.Name" or "Tags.Environment"
awsquery ec2 describe-instances -- Tags.Name Tags.Environment

# CloudTrail LookupEvents with complex parameter structures
# Filter events by event name with automatic type conversion
awsquery cloudtrail lookup-events -p LookupAttributes=AttributeKey=EventName,AttributeValue=ConsoleLogin

# Multiple CloudTrail attributes using semicolon separation
awsquery cloudtrail lookup-events -p LookupAttributes=AttributeKey=EventName,AttributeValue=AssumeRole;AttributeKey=Username,AttributeValue=admin

# CloudTrail with time range and resource type filtering
awsquery cloudtrail lookup-events -p StartTime=2024-01-01 -p EndTime=2024-01-31 -p LookupAttributes=AttributeKey=ResourceType,AttributeValue=AWS::S3::Bucket

### Advanced Parameter Passing

# Pass specific parameters to AWS API calls
awsquery ec2 describe-instances -p MaxResults=10
awsquery ec2 describe-instances -p InstanceIds=i-123,i-456 -p MaxResults=5

# Complex parameter structures for SSM with automatic type conversion
awsquery ssm describe-parameters -p ParameterFilters=Key=Name,Option=Contains,Values=Ubuntu,2024

# Multiple complex structures using semicolon separation
awsquery ssm describe-parameters -p ParameterFilters=Key=Name,Option=Contains,Values=Ubuntu;Key=Type,Option=Equal,Values=String

# Complex structures with nested arrays and type conversion
awsquery ec2 describe-instances -p Filters=Name=instance-state-name,Values=running,stopped;Name=tag:Environment,Values=prod,staging

### Hint-Based Multi-Step Resolution

# Use hints to guide automatic parameter resolution
# When describe-tags needs resource ARNs, hint at using describe-clusters
awsquery elbv2 describe-tags -i desc-clus prod

# CloudFormation stack resources with hint for stack selection
awsquery cloudformation describe-stack-resources -i desc-stacks production

# ECS service details with task definition hint
awsquery ecs describe-services -i desc-task web-service

# Field-specific extraction with function:field format
awsquery elbv2 describe-tags -i desc-clus:clusterarn prod  # Extract ClusterArn specifically
awsquery eks describe-fargate-profile -i desc-clus:rolearn  # Extract RoleArn instead of default

# Limit results from multi-step calls (default is 10)
awsquery ssm get-parameters -i ::5  # Limit to 5 parameters
awsquery ec2 describe-network-interfaces -i ::20  # Limit to 20 interfaces
awsquery ec2 describe-network-interfaces -i ::0  # Unlimited (remove default limit)

# Field override without function hint (uses inferred function)
awsquery ecs describe-tasks -i :clusterarn  # Extract ClusterArn field
awsquery rds describe-db-clusters -i :endpoint  # Extract Endpoint field

# Combined: field and limit
awsquery elbv2 describe-tags -i :arn:3  # Extract ARN field, limit to 3 resources

# Full syntax: function, field, and limit
awsquery elbv2 describe-tags -i desc-clus:clusterarn:5  # Use desc-clus, extract ClusterArn, limit to 5

### Cross-Service Parameter Resolution

# Use service prefixes to resolve parameters from different AWS services
# Resolve EC2 instance IDs to use with SSM patch state queries
awsquery ssm describe-instance-patch-states -i ec2:describe-instances:instanceid prod

# Service-only hint: Let EC2 service auto-infer the operation
awsquery ssm describe-instance-patch-states -i ec2 prod

# Use S3 bucket names for CloudTrail event lookup
awsquery cloudtrail lookup-events -i s3:list-buckets:name backup

# Get IAM user names for access key queries
awsquery iam list-access-keys -i iam:list-users:username:5

# Cross-service with EKS clusters for ECS task definitions
awsquery ecs describe-task-definition -i eks:describe-clusters:name prod-cluster

# Combine cross-service with field extraction and limits
awsquery autoscaling describe-auto-scaling-instances -i ec2:desc-inst:instanceid:10 prod

# Service-only with field hint (operation auto-inferred)
awsquery ssm describe-instance-patch-states -i ec2::instanceid prod
```

## Configuration

### AWS Credentials

Ensure AWS credentials are configured via:
- `~/.aws/credentials` (supports multiple profiles)
- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
- IAM roles (if running on EC2/ECS/Lambda)
- AWS SSO profiles

### Default Filters Configuration

The tool uses `default_filters.yaml` to define default columns for common queries. This comprehensive configuration file (3000+ lines)
provides extensive pre-configured filters for dozens of AWS services, loaded from the package directory to provide you with standard
columns if you don't add any.

Example configuration:
```yaml
ec2:
  describe_instances:
    columns:
      - InstanceId
      - Tags.Name
      - State.Name
      - InstanceType
      - PrivateIpAddress

s3:
  list_buckets:
    columns:
      - Name
      - CreationDate
```

## Development

### Running Tests

```bash
# Install development dependencies
make install-dev

# Run all tests
make test

# Run tests with coverage
make coverage

# Run specific test categories
make test-unit
make test-integration
make test-critical  # Run all tests (comprehensive test suite)

# Code formatting and linting
make format
make format-check  # Check without modifying
make lint
make type-check    # Run mypy type checking
make security-check  # Run security analysis

# Mutation testing
make mutmut        # Run mutation tests
make mutmut-results # Show mutation test results
make mutmut-html   # Generate HTML mutation test report

# Pre-commit hooks
make pre-commit  # Run all pre-commit hooks

# Continuous Integration
make ci  # Run comprehensive CI pipeline (tests, linting, type-check, security)
```


### Docker Usage

```bash
# Build development container
make docker-build

# Open interactive shell
make shell

# Run tests in Docker
make test-in-docker

# Clean Docker artifacts
make docker-clean
```

### Development Workflow

```bash
# Watch tests (auto-run on file changes)
make watch-tests

# Build distribution packages
make build

# Clean build artifacts
make clean

# Publish to Test PyPI
make publish-test

# Publish to PyPI (production)
make publish

# Version management
make version  # Show current version
make release  # Create a new release with version bump
```

## Advanced Usage

### Filter Matching Behavior

All filters in awsquery use **case-insensitive matching** with optional anchoring:

#### Filter Operators
- `^` at the start: matches values that START WITH the pattern (prefix match)
  - Both `^` (U+005E) and `ˆ` (U+02C6) are supported for keyboard compatibility
- `$` at the end: matches values that END WITH the pattern (suffix match)
- `^...$`: matches values that EXACTLY equal the pattern (exact match)
- No operators: matches values that CONTAIN the pattern (partial match)

#### Value Filters (before `--`)
- Match against ANY field in the response data
- ALL specified filters must match (AND logic)
- Case-insensitive matching with optional anchoring

```bash
# "prod" matches: "production", "prod-server", "my-prod-app" (contains)
awsquery ec2 describe-instances prod

# "^prod" matches: "production", "prod-server" (starts with)
awsquery ec2 describe-instances ^prod

# "prod$" matches: "my-prod", "dev-prod" (ends with)
awsquery ec2 describe-instances prod$

# "^prod$" matches: only exactly "prod" (exact match)
awsquery ec2 describe-instances ^prod$

# Both filters must match (AND logic)
awsquery ec2 describe-instances ^prod web$
```

#### Column Filters (after `--`)
- Match against column/field names in the output
- Case-insensitive matching with optional anchoring
- Multiple columns can be specified

```bash
# "Instance" matches: "InstanceId", "InstanceType", "InstanceName" (contains)
awsquery ec2 describe-instances -- Instance

# "^Instance" matches: "InstanceId", "InstanceType" (starts with)
awsquery ec2 describe-instances -- ^Instance

# "Name$" matches: "InstanceName", "GroupName", "Tags.Name" (ends with)
awsquery ec2 describe-instances -- Name$

# "^State.Name$" matches: only exactly "State.Name" (exact match)
awsquery ec2 describe-instances -- ^State.Name$

# Multiple patterns
awsquery ec2 describe-instances -- ^Instance Name$ State
```

### Multi-Level API Calls

The tool automatically handles parameter resolution for nested API calls:

```bash
# Automatically fetches stack list, then events for matching stacks
awsquery cloudformation describe-stack-events production

# Three-level call: list stacks → get resources → filter results
awsquery cloudformation describe-stack-resources prod -- Lambda
```

### Parameter Passing (`-p`/`--parameter`)

Pass parameters directly to AWS API calls using the `-p` flag. This is useful for fine-tuning API behavior:

```bash
# Limit results for large responses
awsquery ec2 describe-instances -p MaxResults=20

# Filter specific resources by ID
awsquery ec2 describe-instances -p InstanceIds=i-1234567890abcdef0,i-0987654321fedcba0

# Multiple parameters (can use multiple -p flags)
awsquery elbv2 describe-load-balancers -p PageSize=10 -p Names=my-alb

# Complex parameter structures (for APIs like SSM)
awsquery ssm describe-parameters -p ParameterFilters=Key=Name,Option=Contains,Values=Ubuntu
```

**Parameter Format:**
- Simple: `Key=Value`
- Lists: `Key=Value1,Value2,Value3`
- Complex structures: Use semicolons to separate multiple objects (e.g., `Key1=Val1,Val2;Key2=Val3`)
- Type conversion: Numbers and booleans are automatically converted
- Nested structures: Comma-separated values within objects, semicolon-separated objects
- CloudTrail example: `LookupAttributes=AttributeKey=EventName,AttributeValue=Login;AttributeKey=Username,AttributeValue=admin`

### Hint-Based Resolution (`-i`/`--input`)

Guide multi-step parameter resolution with function/field/limit hints:

```bash
# Basic function hint - guides which list operation to use
awsquery cloudformation describe-stack-resources -i list-sta production

# Field extraction hint - specify exact field to extract
awsquery elbv2 describe-tags -i desc-clus:clusterarn prod

# Result limiting - control how many resources are processed (default: 10)
awsquery ssm get-parameters -i ::5  # Limit to 5 parameters
awsquery ec2 describe-instances -i ::20  # Limit to 20 instances
awsquery s3api list-objects -i ::0  # Unlimited (remove default limit)

# Field override without function (uses inferred function)
awsquery ecs describe-tasks -i :clusterarn  # Extract ClusterArn field

# Combined hints - function:field:limit
awsquery elbv2 describe-tags -i desc-clus:clusterarn:5  # Function + field + limit
awsquery rds describe-db-snapshots -i :arn:10  # Field + limit (inferred function)
```

**Hint Syntax:** `[function]:[field]:[limit]`
- **function**: Operation to use (e.g., "desc-clus", smart matching enabled)
- **field**: Specific field to extract (e.g., "arn", "clusterarn")
- **limit**: Max resources to process (e.g., "5", "10", "0" for unlimited)
- Any component can be omitted (e.g., "::5" for limit only, ":arn" for field only)

**Key Features:**
- **Smart matching**: "desc-inst" matches "describe-instances"
- **Field targeting**: Override default field extraction
- **Result limiting**: Default 10 items, configurable with ::N syntax
- **Flexible syntax**: Mix and match function/field/limit as needed

### Filtering Strategies

```bash
# All filters must match (AND logic), each using partial matching
# Finds instances where ALL three terms appear somewhere in the data
awsquery ec2 describe-instances production web database

# Column filters use partial, case-insensitive matching
# Shows columns containing "Instance", "State", or "Private" in their names
awsquery ec2 describe-instances -- Instance State Private

# Combine value and column filters
# Finds buckets with "backup" in any field, shows Name and Creation columns
awsquery s3 list-buckets backup -- Name Creation

# Partial matching examples:
# "prod" matches "production", "prod-server", "myproduct"
# "PROD" matches "production" (case-insensitive)
# "i-123" matches "i-1234567890abcdef0"
```

### Performance Tips

- Use specific filters to reduce API calls
- Column filters (`--`) don't affect API calls, only display
- Use `--region` to avoid cross-region latency
- Keys mode (`-k`) adds overhead; use only for discovery

## Troubleshooting

### Common Issues

**"No matching resources found"**
- Check your filters are not too restrictive
- Use debug mode (`-d`) to see actual API responses
- Verify you have permissions in the target region/account

**"Access Denied" errors**
- Ensure your AWS credentials have ReadOnly permissions
- Check if using the correct profile with `--profile`
- Verify the operation is permitted

**"Parameter validation failed"**
- Some APIs require specific parameters
- The tool attempts automatic resolution but may need manual input
- Use debug mode to see the parameter resolution process

**Performance issues**
- Large result sets may take time to process
- Use more specific filters to reduce data volume
- Consider using `--region` to target specific regions

## Requirements

The package dependencies are:

- boto3>=1.35.0
- botocore>=1.35.0
- tabulate>=0.9.0
- argcomplete>=3.0.0
- PyYAML>=6.0.0

## License

MIT License - see LICENSE file for details.

## Contributing

Contributions are welcome! Please ensure tests pass and follow the existing code style.

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "awsquery",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "aws, cli, query, cloud, devops, boto3",
    "author": null,
    "author_email": "Florian Motlik <flo@flomotlik.me>",
    "download_url": "https://files.pythonhosted.org/packages/3c/f6/e5fae8f1f855be6bac934b94a1a68dbeb0bd7aa926606166268e2210ec5e/awsquery-0.5.0.tar.gz",
    "platform": null,
    "description": "# awsquery - AWS API Query Tool\n\nAWS CLI tool to run any awscli read command and filter the resulting Values and JSON for custom table output.\n\nThe following command will find all ec2 instances where `prod`and `web` are found somewhere in the Value\nof any key of the whole response. For all matching resources it will extract all fields that match\nany of the second filters, so anything that matches `Tags.Name`, `State`, `InstanceId` or `vpcid`.\n```\nawsquery ec2 describe-instances prod web -- Tags.Name State InstanceId vpcid\n```\n\nThis creates endless flexibility to shape your aws cli calls, filter any output and present exactly\nthe data you need during review, debugging or development.\n\n## Features\n\n- **Smart Multi-Level Calls**: Automatically resolves missing parameters by inferring and calling list operations\n- **Flexible Filtering**: Multi-level filtering with `--` separators for resource, value, and column filters\n- **Partial Matching**: All filters use case-insensitive partial matching (both value and column filters)\n- **Keys Discovery**: Show all available fields from any API response with `-k`/`--keys` (fixed to show keys from successful responses)\n- **Debug Mode**: Comprehensive debug output with `-d`/`--debug` featuring structured output with timestamps and DebugContext tracking\n- **Security Validation**: Enforces ReadOnly AWS operations with comprehensive validation\n- **Smart Auto-completion**: Enhanced tab completion with split matching and prefix priority for AWS services and actions\n- **Smart Parameter Extraction**: Handles both specific fields and standard AWS field patterns (Name, Id, Arn)\n- **Intelligent Response Processing**: Clean extraction of list data, ignoring metadata\n- **Tabular Output**: Customizable column display with automatic filtering\n- **Pagination Support**: Handles large AWS responses automatically\n- **Region/Profile Support**: AWS CLI-compatible `--region` and `--profile` arguments for session management\n- **Tag Transformation**: Automatic conversion of AWS Tags list to key-value pairs for better readability\n- **Default Column Filters**: Configuration-based default columns for common AWS queries\n- **Parameter Passing**: Direct parameter passing to AWS APIs with `-p`/`--parameter` for advanced use cases\n- **Hint-Based Resolution**: Function selection hints with `-i`/`--input` for multi-step calls, including cross-service support and field extraction targeting\n\n## Installation\n\n### Via pip (Recommended)\n\n```bash\npip install awsquery\n```\n\n### Development Installation\n\n```bash\ngit clone https://github.com/yourusername/awsquery.git\ncd awsquery\npip install -e \".[dev]\"\n```\n\n### Enable Shell Autocomplete\n\nawsquery supports tab completion for AWS services and actions through argcomplete.\n\n#### Setup\n\n##### Bash\n```bash\n# Add to ~/.bashrc or ~/.bash_profile\neval \"$(register-python-argcomplete awsquery)\"\n```\n\n##### Zsh\n```bash\n# Add to ~/.zshrc\nautoload -U bashcompinit && bashcompinit\neval \"$(register-python-argcomplete awsquery)\"\n```\n\n##### Fish\n```bash\n# Add to ~/.config/fish/config.fish\nregister-python-argcomplete --shell fish awsquery | source\n```\n\nAfter adding the appropriate line to your shell configuration, restart your shell or source the file:\n```bash\nsource ~/.bashrc  # or ~/.zshrc, etc.\n```\n\nNow you can use enhanced tab completion with smart matching:\n```bash\nawsquery <TAB>              # Shows available services\nawsquery ec2 <TAB>          # Shows available ec2 actions\nawsquery s3 list-<TAB>      # Shows s3 list actions\nawsquery ec2 desc-inst<TAB> # Smart completion: \"desc-inst\" matches \"describe-instances\"\nawsquery cloudformation des-sta<TAB> # Matches \"describe-stacks\"\n```\n\nThe autocomplete system now features:\n- **Split matching**: \"desc-inst\" matches \"describe-instances\"\n- **Prefix priority**: Exact prefix matches are prioritized over substring matches\n- **Security filtering**: Only shows ReadOnly operations\n\n## Usage\n\n### Basic Query\n```bash\n# Query EC2 instances with filtering\nawsquery ec2 describe-instances\n\n# Filter by values (partial match, case-insensitive on ANY field)\nawsquery ec2 describe-instances prod web\n\n# Specify output columns (partial match, case-insensitive)\nawsquery ec2 describe-instances prod -- Name State InstanceId\n\n# List S3 buckets containing \"backup\" (matches if \"backup\" appears anywhere in any field)\nawsquery s3 list-buckets backup\n\n# JSON output format\nawsquery -j s3 list-buckets\n```\n\n### Keys Discovery\n```bash\n# Show available fields/keys from an API response\nawsquery -k ec2 describe-instances\nawsquery --keys s3 list-buckets\n```\n\n### Discovery and Debug\n```bash\n# List available services\nawsquery\n\n# Debug mode for troubleshooting with enhanced DebugContext output\n# Shows structured debug information with timestamps and execution flow\nawsquery -d ec2 describe-instances\n```\n\n## Command Structure\n\n```\nawsquery [-j|--json] [-k|--keys] [-d|--debug] [-p PARAM] [-i HINT] [--region REGION] [--profile PROFILE] SERVICE ACTION [VALUE_FILTERS...] [-- TABLE_OUTPUT_FILTERS...]\n```\n\n- **SERVICE**: AWS service name (ec2, s3, iam, etc.)\n- **ACTION**: Service action (describe-instances, list-buckets, etc.)\n- **VALUE_FILTERS**: Space-separated filters - ALL must match, using case-insensitive partial matching on any field\n- **TABLE_OUTPUT_FILTERS**: Column selection - partial, case-insensitive matching on column names\n- **-j, --json**: Output results in JSON format instead of table\n- **-k, --keys**: Show all available keys for the command\n- **-d, --debug**: Enable debug output for troubleshooting\n- **-p, --parameter PARAM**: Pass parameters directly to AWS API (key=value format, propagates to list operations)\n- **-i, --input HINT**: Multi-step control with cross-service support and function/field/limit hints (e.g., \"ec2:desc-inst:instanceid\", \"desc-clus\", \":arn\", \"::5\")\n- **--region REGION**: AWS region to use for requests (e.g., us-west-2)\n- **--profile PROFILE**: AWS profile to use from ~/.aws/credentials\n\n## Security\n\n- **ReadOnly Enforcement**: Only AWS ReadOnly operations are permitted\n- **Input Sanitization**: Prevents injection attacks through parameter validation\n- **Operation Validation**: All actions validated before execution\n- **Wildcard Matching**: Supports AWS IAM wildcard patterns (e.g., `ec2:Describe*`)\n- **Session Isolation**: Each profile/region maintains separate boto3 sessions\n- **Security Testing**: Comprehensive test suite validates security constraints\n\n## Examples\n\n```bash\n# Find instances with \"prod\" AND \"web\" in any field (partial match)\n# e.g., matches \"production-web-server\", \"web.prod.example.com\"\nawsquery ec2 describe-instances prod web\n\n# Show only columns matching Name, State and InstanceId (partial match)\n# e.g., \"Name\" matches \"InstanceName\", \"Tags.Name\", \"SecurityGroupName\"\nawsquery ec2 describe-instances prod web -- Name State InstanceId\n\n# Find S3 buckets with \"backup\" anywhere in the data\n# e.g., matches \"my-backup-bucket\", \"bucket-backup-2024\", \"backups\"\nawsquery s3 list-buckets backup\n\n# Multi-level CloudFormation query with parameter resolution\n# Filters: \"prod\" in stack data, columns containing \"Created\" or \"StackName\"\nawsquery cloudformation describe-stack-events prod -- Created StackName\n\n# Targeted field extraction with hint for multi-step calls\nawsquery elbv2 describe-tags -i desc-clus:clusterarn prod\n\n# Discover all available keys\nawsquery -k ec2 describe-instances\n\n# JSON output with filtering (partial column name matching)\nawsquery -j ec2 describe-instances prod -- InstanceId State.Name\n\n# Debug mode with enhanced output showing parameter resolution and API calls\nawsquery -d cloudformation describe-stack-resources workers -- EKS\n\n# Use specific AWS region\nawsquery --region eu-west-1 ec2 describe-instances\n\n# Use specific AWS profile\nawsquery --profile production s3 list-buckets\n\n# Combine region and profile\nawsquery --region us-east-2 --profile dev ec2 describe-vpcs\n\n# View transformed tags (partial match on column names)\n# Shows any column containing \"Tags.Name\" or \"Tags.Environment\"\nawsquery ec2 describe-instances -- Tags.Name Tags.Environment\n\n# CloudTrail LookupEvents with complex parameter structures\n# Filter events by event name with automatic type conversion\nawsquery cloudtrail lookup-events -p LookupAttributes=AttributeKey=EventName,AttributeValue=ConsoleLogin\n\n# Multiple CloudTrail attributes using semicolon separation\nawsquery cloudtrail lookup-events -p LookupAttributes=AttributeKey=EventName,AttributeValue=AssumeRole;AttributeKey=Username,AttributeValue=admin\n\n# CloudTrail with time range and resource type filtering\nawsquery cloudtrail lookup-events -p StartTime=2024-01-01 -p EndTime=2024-01-31 -p LookupAttributes=AttributeKey=ResourceType,AttributeValue=AWS::S3::Bucket\n\n### Advanced Parameter Passing\n\n# Pass specific parameters to AWS API calls\nawsquery ec2 describe-instances -p MaxResults=10\nawsquery ec2 describe-instances -p InstanceIds=i-123,i-456 -p MaxResults=5\n\n# Complex parameter structures for SSM with automatic type conversion\nawsquery ssm describe-parameters -p ParameterFilters=Key=Name,Option=Contains,Values=Ubuntu,2024\n\n# Multiple complex structures using semicolon separation\nawsquery ssm describe-parameters -p ParameterFilters=Key=Name,Option=Contains,Values=Ubuntu;Key=Type,Option=Equal,Values=String\n\n# Complex structures with nested arrays and type conversion\nawsquery ec2 describe-instances -p Filters=Name=instance-state-name,Values=running,stopped;Name=tag:Environment,Values=prod,staging\n\n### Hint-Based Multi-Step Resolution\n\n# Use hints to guide automatic parameter resolution\n# When describe-tags needs resource ARNs, hint at using describe-clusters\nawsquery elbv2 describe-tags -i desc-clus prod\n\n# CloudFormation stack resources with hint for stack selection\nawsquery cloudformation describe-stack-resources -i desc-stacks production\n\n# ECS service details with task definition hint\nawsquery ecs describe-services -i desc-task web-service\n\n# Field-specific extraction with function:field format\nawsquery elbv2 describe-tags -i desc-clus:clusterarn prod  # Extract ClusterArn specifically\nawsquery eks describe-fargate-profile -i desc-clus:rolearn  # Extract RoleArn instead of default\n\n# Limit results from multi-step calls (default is 10)\nawsquery ssm get-parameters -i ::5  # Limit to 5 parameters\nawsquery ec2 describe-network-interfaces -i ::20  # Limit to 20 interfaces\nawsquery ec2 describe-network-interfaces -i ::0  # Unlimited (remove default limit)\n\n# Field override without function hint (uses inferred function)\nawsquery ecs describe-tasks -i :clusterarn  # Extract ClusterArn field\nawsquery rds describe-db-clusters -i :endpoint  # Extract Endpoint field\n\n# Combined: field and limit\nawsquery elbv2 describe-tags -i :arn:3  # Extract ARN field, limit to 3 resources\n\n# Full syntax: function, field, and limit\nawsquery elbv2 describe-tags -i desc-clus:clusterarn:5  # Use desc-clus, extract ClusterArn, limit to 5\n\n### Cross-Service Parameter Resolution\n\n# Use service prefixes to resolve parameters from different AWS services\n# Resolve EC2 instance IDs to use with SSM patch state queries\nawsquery ssm describe-instance-patch-states -i ec2:describe-instances:instanceid prod\n\n# Service-only hint: Let EC2 service auto-infer the operation\nawsquery ssm describe-instance-patch-states -i ec2 prod\n\n# Use S3 bucket names for CloudTrail event lookup\nawsquery cloudtrail lookup-events -i s3:list-buckets:name backup\n\n# Get IAM user names for access key queries\nawsquery iam list-access-keys -i iam:list-users:username:5\n\n# Cross-service with EKS clusters for ECS task definitions\nawsquery ecs describe-task-definition -i eks:describe-clusters:name prod-cluster\n\n# Combine cross-service with field extraction and limits\nawsquery autoscaling describe-auto-scaling-instances -i ec2:desc-inst:instanceid:10 prod\n\n# Service-only with field hint (operation auto-inferred)\nawsquery ssm describe-instance-patch-states -i ec2::instanceid prod\n```\n\n## Configuration\n\n### AWS Credentials\n\nEnsure AWS credentials are configured via:\n- `~/.aws/credentials` (supports multiple profiles)\n- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)\n- IAM roles (if running on EC2/ECS/Lambda)\n- AWS SSO profiles\n\n### Default Filters Configuration\n\nThe tool uses `default_filters.yaml` to define default columns for common queries. This comprehensive configuration file (3000+ lines)\nprovides extensive pre-configured filters for dozens of AWS services, loaded from the package directory to provide you with standard\ncolumns if you don't add any.\n\nExample configuration:\n```yaml\nec2:\n  describe_instances:\n    columns:\n      - InstanceId\n      - Tags.Name\n      - State.Name\n      - InstanceType\n      - PrivateIpAddress\n\ns3:\n  list_buckets:\n    columns:\n      - Name\n      - CreationDate\n```\n\n## Development\n\n### Running Tests\n\n```bash\n# Install development dependencies\nmake install-dev\n\n# Run all tests\nmake test\n\n# Run tests with coverage\nmake coverage\n\n# Run specific test categories\nmake test-unit\nmake test-integration\nmake test-critical  # Run all tests (comprehensive test suite)\n\n# Code formatting and linting\nmake format\nmake format-check  # Check without modifying\nmake lint\nmake type-check    # Run mypy type checking\nmake security-check  # Run security analysis\n\n# Mutation testing\nmake mutmut        # Run mutation tests\nmake mutmut-results # Show mutation test results\nmake mutmut-html   # Generate HTML mutation test report\n\n# Pre-commit hooks\nmake pre-commit  # Run all pre-commit hooks\n\n# Continuous Integration\nmake ci  # Run comprehensive CI pipeline (tests, linting, type-check, security)\n```\n\n\n### Docker Usage\n\n```bash\n# Build development container\nmake docker-build\n\n# Open interactive shell\nmake shell\n\n# Run tests in Docker\nmake test-in-docker\n\n# Clean Docker artifacts\nmake docker-clean\n```\n\n### Development Workflow\n\n```bash\n# Watch tests (auto-run on file changes)\nmake watch-tests\n\n# Build distribution packages\nmake build\n\n# Clean build artifacts\nmake clean\n\n# Publish to Test PyPI\nmake publish-test\n\n# Publish to PyPI (production)\nmake publish\n\n# Version management\nmake version  # Show current version\nmake release  # Create a new release with version bump\n```\n\n## Advanced Usage\n\n### Filter Matching Behavior\n\nAll filters in awsquery use **case-insensitive matching** with optional anchoring:\n\n#### Filter Operators\n- `^` at the start: matches values that START WITH the pattern (prefix match)\n  - Both `^` (U+005E) and `\u02c6` (U+02C6) are supported for keyboard compatibility\n- `$` at the end: matches values that END WITH the pattern (suffix match)\n- `^...$`: matches values that EXACTLY equal the pattern (exact match)\n- No operators: matches values that CONTAIN the pattern (partial match)\n\n#### Value Filters (before `--`)\n- Match against ANY field in the response data\n- ALL specified filters must match (AND logic)\n- Case-insensitive matching with optional anchoring\n\n```bash\n# \"prod\" matches: \"production\", \"prod-server\", \"my-prod-app\" (contains)\nawsquery ec2 describe-instances prod\n\n# \"^prod\" matches: \"production\", \"prod-server\" (starts with)\nawsquery ec2 describe-instances ^prod\n\n# \"prod$\" matches: \"my-prod\", \"dev-prod\" (ends with)\nawsquery ec2 describe-instances prod$\n\n# \"^prod$\" matches: only exactly \"prod\" (exact match)\nawsquery ec2 describe-instances ^prod$\n\n# Both filters must match (AND logic)\nawsquery ec2 describe-instances ^prod web$\n```\n\n#### Column Filters (after `--`)\n- Match against column/field names in the output\n- Case-insensitive matching with optional anchoring\n- Multiple columns can be specified\n\n```bash\n# \"Instance\" matches: \"InstanceId\", \"InstanceType\", \"InstanceName\" (contains)\nawsquery ec2 describe-instances -- Instance\n\n# \"^Instance\" matches: \"InstanceId\", \"InstanceType\" (starts with)\nawsquery ec2 describe-instances -- ^Instance\n\n# \"Name$\" matches: \"InstanceName\", \"GroupName\", \"Tags.Name\" (ends with)\nawsquery ec2 describe-instances -- Name$\n\n# \"^State.Name$\" matches: only exactly \"State.Name\" (exact match)\nawsquery ec2 describe-instances -- ^State.Name$\n\n# Multiple patterns\nawsquery ec2 describe-instances -- ^Instance Name$ State\n```\n\n### Multi-Level API Calls\n\nThe tool automatically handles parameter resolution for nested API calls:\n\n```bash\n# Automatically fetches stack list, then events for matching stacks\nawsquery cloudformation describe-stack-events production\n\n# Three-level call: list stacks \u2192 get resources \u2192 filter results\nawsquery cloudformation describe-stack-resources prod -- Lambda\n```\n\n### Parameter Passing (`-p`/`--parameter`)\n\nPass parameters directly to AWS API calls using the `-p` flag. This is useful for fine-tuning API behavior:\n\n```bash\n# Limit results for large responses\nawsquery ec2 describe-instances -p MaxResults=20\n\n# Filter specific resources by ID\nawsquery ec2 describe-instances -p InstanceIds=i-1234567890abcdef0,i-0987654321fedcba0\n\n# Multiple parameters (can use multiple -p flags)\nawsquery elbv2 describe-load-balancers -p PageSize=10 -p Names=my-alb\n\n# Complex parameter structures (for APIs like SSM)\nawsquery ssm describe-parameters -p ParameterFilters=Key=Name,Option=Contains,Values=Ubuntu\n```\n\n**Parameter Format:**\n- Simple: `Key=Value`\n- Lists: `Key=Value1,Value2,Value3`\n- Complex structures: Use semicolons to separate multiple objects (e.g., `Key1=Val1,Val2;Key2=Val3`)\n- Type conversion: Numbers and booleans are automatically converted\n- Nested structures: Comma-separated values within objects, semicolon-separated objects\n- CloudTrail example: `LookupAttributes=AttributeKey=EventName,AttributeValue=Login;AttributeKey=Username,AttributeValue=admin`\n\n### Hint-Based Resolution (`-i`/`--input`)\n\nGuide multi-step parameter resolution with function/field/limit hints:\n\n```bash\n# Basic function hint - guides which list operation to use\nawsquery cloudformation describe-stack-resources -i list-sta production\n\n# Field extraction hint - specify exact field to extract\nawsquery elbv2 describe-tags -i desc-clus:clusterarn prod\n\n# Result limiting - control how many resources are processed (default: 10)\nawsquery ssm get-parameters -i ::5  # Limit to 5 parameters\nawsquery ec2 describe-instances -i ::20  # Limit to 20 instances\nawsquery s3api list-objects -i ::0  # Unlimited (remove default limit)\n\n# Field override without function (uses inferred function)\nawsquery ecs describe-tasks -i :clusterarn  # Extract ClusterArn field\n\n# Combined hints - function:field:limit\nawsquery elbv2 describe-tags -i desc-clus:clusterarn:5  # Function + field + limit\nawsquery rds describe-db-snapshots -i :arn:10  # Field + limit (inferred function)\n```\n\n**Hint Syntax:** `[function]:[field]:[limit]`\n- **function**: Operation to use (e.g., \"desc-clus\", smart matching enabled)\n- **field**: Specific field to extract (e.g., \"arn\", \"clusterarn\")\n- **limit**: Max resources to process (e.g., \"5\", \"10\", \"0\" for unlimited)\n- Any component can be omitted (e.g., \"::5\" for limit only, \":arn\" for field only)\n\n**Key Features:**\n- **Smart matching**: \"desc-inst\" matches \"describe-instances\"\n- **Field targeting**: Override default field extraction\n- **Result limiting**: Default 10 items, configurable with ::N syntax\n- **Flexible syntax**: Mix and match function/field/limit as needed\n\n### Filtering Strategies\n\n```bash\n# All filters must match (AND logic), each using partial matching\n# Finds instances where ALL three terms appear somewhere in the data\nawsquery ec2 describe-instances production web database\n\n# Column filters use partial, case-insensitive matching\n# Shows columns containing \"Instance\", \"State\", or \"Private\" in their names\nawsquery ec2 describe-instances -- Instance State Private\n\n# Combine value and column filters\n# Finds buckets with \"backup\" in any field, shows Name and Creation columns\nawsquery s3 list-buckets backup -- Name Creation\n\n# Partial matching examples:\n# \"prod\" matches \"production\", \"prod-server\", \"myproduct\"\n# \"PROD\" matches \"production\" (case-insensitive)\n# \"i-123\" matches \"i-1234567890abcdef0\"\n```\n\n### Performance Tips\n\n- Use specific filters to reduce API calls\n- Column filters (`--`) don't affect API calls, only display\n- Use `--region` to avoid cross-region latency\n- Keys mode (`-k`) adds overhead; use only for discovery\n\n## Troubleshooting\n\n### Common Issues\n\n**\"No matching resources found\"**\n- Check your filters are not too restrictive\n- Use debug mode (`-d`) to see actual API responses\n- Verify you have permissions in the target region/account\n\n**\"Access Denied\" errors**\n- Ensure your AWS credentials have ReadOnly permissions\n- Check if using the correct profile with `--profile`\n- Verify the operation is permitted\n\n**\"Parameter validation failed\"**\n- Some APIs require specific parameters\n- The tool attempts automatic resolution but may need manual input\n- Use debug mode to see the parameter resolution process\n\n**Performance issues**\n- Large result sets may take time to process\n- Use more specific filters to reduce data volume\n- Consider using `--region` to target specific regions\n\n## Requirements\n\nThe package dependencies are:\n\n- boto3>=1.35.0\n- botocore>=1.35.0\n- tabulate>=0.9.0\n- argcomplete>=3.0.0\n- PyYAML>=6.0.0\n\n## License\n\nMIT License - see LICENSE file for details.\n\n## Contributing\n\nContributions are welcome! Please ensure tests pass and follow the existing code style.\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Advanced CLI tool for querying AWS APIs with flexible filtering and automatic parameter resolution",
    "version": "0.5.0",
    "project_urls": {
        "Changelog": "https://github.com/yourusername/awsquery/blob/main/CHANGELOG.md",
        "Documentation": "https://awsquery.readthedocs.io",
        "Homepage": "https://github.com/yourusername/awsquery",
        "Issues": "https://github.com/yourusername/awsquery/issues",
        "Repository": "https://github.com/yourusername/awsquery.git"
    },
    "split_keywords": [
        "aws",
        " cli",
        " query",
        " cloud",
        " devops",
        " boto3"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "4ee0b1b94aa76cf76f67ab4725693241ce63e4bf0a87d87ca3931e5d4dc98847",
                "md5": "fa24734f309bd6a1ec6e344b9aafe4a5",
                "sha256": "175e665b158b742a25e9c12519ff825735102e8592c5cc990f4c859f9631f5b8"
            },
            "downloads": -1,
            "filename": "awsquery-0.5.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "fa24734f309bd6a1ec6e344b9aafe4a5",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 74103,
            "upload_time": "2025-10-06T10:49:48",
            "upload_time_iso_8601": "2025-10-06T10:49:48.466720Z",
            "url": "https://files.pythonhosted.org/packages/4e/e0/b1b94aa76cf76f67ab4725693241ce63e4bf0a87d87ca3931e5d4dc98847/awsquery-0.5.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "3cf6e5fae8f1f855be6bac934b94a1a68dbeb0bd7aa926606166268e2210ec5e",
                "md5": "4fde82e9578a3163918ab41226d53ea5",
                "sha256": "5b49bb91ccf8ebf9cceb781a283be2e05c47ad7213f1e9a8bc1e363f4e422341"
            },
            "downloads": -1,
            "filename": "awsquery-0.5.0.tar.gz",
            "has_sig": false,
            "md5_digest": "4fde82e9578a3163918ab41226d53ea5",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 262916,
            "upload_time": "2025-10-06T10:49:50",
            "upload_time_iso_8601": "2025-10-06T10:49:50.057566Z",
            "url": "https://files.pythonhosted.org/packages/3c/f6/e5fae8f1f855be6bac934b94a1a68dbeb0bd7aa926606166268e2210ec5e/awsquery-0.5.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-10-06 10:49:50",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "yourusername",
    "github_project": "awsquery",
    "github_not_found": true,
    "lcname": "awsquery"
}