BDNS Fetch
===========
[](https://badge.fury.io/py/bdns-fetch)
[](https://www.gnu.org/licenses/gpl-3.0)
[](https://www.python.org/downloads/)
A comprehensive command-line tool for accessing and processing data from the Base de Datos Nacional de Subvenciones (BDNS) API.
## ✨ Features
- **29+ Data Extraction Commands**: Covers all key data extraction endpoints from the BDNS API
- **JSONL Output Format**: Clean JSON Lines format for easy data processing
- **Flexible Configuration**: Customizable parameters for each command
- **Clean Error Handling**: User-friendly error messages for API issues
- **Verbose Logging**: Detailed HTTP request/response logging for debugging
- **Concurrent Processing**: Built-in pagination and concurrent request handling
## 📋 Available Commands
This tool provides access to **29+ BDNS API data extraction endpoints**. Each command fetches specific data from the Base de Datos Nacional de Subvenciones (BDNS).
For a complete list of all commands and their parameters, use:
```bash
bdns-fetch --help
```
For help on a specific command:
```bash
bdns-fetch [command-name] --help
# Example: bdns-fetch organos --help
```
**📖 API Documentation**: Complete endpoint documentation is available at [BDNS API Swagger](https://www.infosubvenciones.es/bdnstrans/estaticos/doc/snpsap-api.json)
## 🚀 Quick Start
### Installation
**From PyPI (recommended):**
```bash
pip install bdns-fetch
```
**From source:**
```bash
git clone https://github.com/cruzlorite/bdns-fetch.git
cd bdns-fetch
poetry install
```
### CLI Usage
**Getting Help:**
```bash
# List all available commands
bdns-fetch --help
# Get help for a specific command
bdns-fetch organos --help
bdns-fetch ayudasestado-busqueda --help
```
**Basic Examples:**
```bash
# Fetch government organs data to file
bdns-fetch organos --output-file government_organs.jsonl
# Get economic activities (to stdout by default)
bdns-fetch actividades
# Search state aids with filters and verbose logging
bdns-fetch --verbose ayudasestado-busqueda \
--descripcion "innovation" \
--num-pages 3 \
--pageSize 1000 \
--output-file innovation_aids.jsonl
# Get specific strategic plan by ID with debugging
bdns-fetch --verbose planesestrategicos \
--idPES 459 \
--output-file plan_459.jsonl
```
**Common Parameters:**
- `--output-file FILE`: Save output to file (defaults to stdout)
- `--verbose, -v`: Enable detailed HTTP request/response logging
- `--vpd CODE`: Territory code (GE=Spain, specific regions available)
- `--num-pages N`: Number of pages to fetch (for paginated commands)
- `--pageSize N`: Records per page (default: 10000, max: 10000)
- `--max-concurrent-requests N`: Maximum concurrent API requests (default: 5)
**Advanced Search Example:**
```bash
# Search concessions with multiple filters and verbose logging
bdns-fetch --verbose concesiones-busqueda \
--descripcion "research" \
--fechaDesde "2023-01-01" \
--fechaHasta "2024-12-31" \
--tipoAdministracion "C" \
--num-pages 10 \
--max-concurrent-requests 8 \
--output-file research_concessions.jsonl
```
## 📖 More Examples
```bash
# Download all government organs
bdns-fetch organos --output-file government_structure.jsonl
# Search for innovation-related subsidies with verbose logging
bdns-fetch --verbose ayudasestado-busqueda \
--descripcion "innovation" \
--output-file innovation_aids.jsonl
# Get latest calls for proposals
bdns-fetch convocatorias-ultimas --output-file latest_calls.jsonl
# Search sanctions data with detailed HTTP logging
bdns-fetch --verbose sanciones-busqueda --output-file sanctions.jsonl
```
Output format (JSON Lines):
```json
{"id": 1, "descripcion": "MINISTERIO DE AGRICULTURA, PESCA Y ALIMENTACIÓN", "codigo": "E04"}
{"id": 2, "descripcion": "MINISTERIO DE ASUNTOS EXTERIORES, UNIÓN EUROPEA Y COOPERACIÓN", "codigo": "E05"}
```
## 🔧 Error Handling & Debugging
### Clean Error Messages
The tool provides user-friendly error messages for common API issues:
```bash
# Invalid parameter example
$ bdns-fetch ayudasestado-busqueda --vpd INVALID_PORTAL
Error (ERR_VALIDACION): El parámetro 'vpd' indica un portal no válido.
```
### Verbose Logging
Use the `--verbose` flag to see detailed HTTP request and response information:
```bash
# Enable verbose logging for debugging
$ bdns-fetch --verbose ayudasestado-busqueda --pageSize 1
DEBUG:bdns.fetch.fetch_write:HTTP REQUEST: GET https://www.infosubvenciones.es/bdnstrans/api/ayudasestado/busqueda?vpd=GE&pageSize=1&page=0
DEBUG:bdns.fetch.fetch_write:HTTP RESPONSE: 200 - 163.6ms
DEBUG:bdns.fetch.fetch_write:Response Headers: {'Date': 'Wed, 20 Aug 2025 21:27:27 GMT', 'Content-Type': 'application/json', ...}
DEBUG:bdns.fetch.fetch_write:Response Content-Length: 1814 bytes
DEBUG:bdns.fetch.fetch_write:Response contains 1 items
DEBUG:bdns.fetch.fetch_write:Total pages available: 5783747
DEBUG:bdns.fetch.fetch_write:Current page: 0
```
**Verbose logging includes:**
- Timestamps for all operations
- Complete HTTP request URLs
- Response status codes and timing (in milliseconds)
- Full HTTP response headers
- Response content size and structure analysis
- Error details for troubleshooting
**When to use verbose mode:**
- Debugging API connectivity issues
- Analyzing response times and performance
- Understanding API rate limiting behavior
- Troubleshooting parameter validation errors
- Monitoring large data extraction jobs
## ⚠️ Current Limitations
### Missing Commands
**This tool implements 30 out of 46 total API endpoints**. The following 16 commands are **intentionally not included**:
#### Export/Download Endpoints (9 missing)
These endpoints generate PDF, CSV, or Excel files instead of JSON data:
- `convocatorias/exportar` - Export search results to files
- `convocatorias/ultimas/exportar` - Export latest calls to files
- `concesiones/exportar` - Export concessions search to files
- `ayudasestado/exportar` - Export state aids search to files
- `minimis/exportar` - Export minimis search to files
- `grandesbeneficiarios/exportar` - Export large beneficiaries to files
- `partidospoliticos/exportar` - Export political parties search to files
- `planesestrategicos/exportar` - Export strategic plans to files
- `sanciones/exportar` - Export sanctions search to files
**Why excluded**: These endpoints return binary file data (PDF/Excel/CSV) instead of structured JSON data, making them unsuitable for a CLI tool focused on data extraction and processing.
#### Portal Configuration Endpoints (2 missing)
- `vpd/{vpd}/configuracion` - Get portal navigation configuration
- `enlaces` - Get portal links and micro-windows
**Why excluded**: These endpoints return web portal configuration data (navigation menus, links) that are not relevant for data extraction purposes.
#### Subscription/Alert System (5 missing)
- `suscripciones/alta` - Create new alert subscription
- `suscripciones/altaidentificado` - Create subscription with token
- `suscripciones/activar` - Activate subscription
- `suscripciones/login` - Login to subscription service
- `suscripciones/cerrar` - Close session
- `suscripciones/detalle` - Get subscription details
- `suscripciones/modificar` - Modify subscription
- `suscripciones/anular` - Cancel subscription
- `suscripciones/reactivar` - Reactivate subscription
- `suscripciones/recuperarcontrasena` - Recover password
- `suscripciones/restablecercontrasena` - Reset password
**Why excluded**: The subscription system requires user authentication, password management, and email verification - functionality better suited for the official web portal rather than a CLI data extraction tool.
### Recommended Usage
- **Test First**: Always test commands with small datasets before large-scale usage
- **Use Verbose Mode**: Enable `--verbose` for debugging API issues or monitoring large extractions
- **Check API Status**: Verify that specific endpoints are working before relying on them for production use
- **Monitor for Updates**: The Spanish government may update the API without notice
## 🛠️ Development
### Prerequisites
- Python 3.11+
- Poetry for dependency management
### Development Setup
```bash
# Clone and setup
git clone https://github.com/cruzlorite/bdns-fetch.git
cd bdns-fetch
poetry install --with dev
# Available Make targets
make help # Show all available targets
make install # Install project dependencies
make dev-install # Install with development dependencies
make lint # Run code linting with ruff
make format # Format code with ruff formatter
make test-integration # Run integration tests
make clean # Remove build artifacts
make all # Install, lint, format, and test
```
## 🙏 Acknowledgments
This project is inspired by previous work from [Jaime Ortega Obregón](https://github.com/JaimeObregon/subvenciones/tree/main).
## 📜 License
This project is licensed under the GNU General Public License v3.0 (GPL-3.0). See the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "bdns-fetch",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "bdns, subsidies, spanish-government, api, data, cli",
"author": "Jos\u00e9 Mar\u00eda Cruz Lorite",
"author_email": "josemariacruzlorite@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/fa/bc/6545557829f35a1ecd7fe8423cc2a80f51b72ff246d52c3164c63e343ce2/bdns_fetch-2.1.0.tar.gz",
"platform": null,
"description": "BDNS Fetch\n===========\n[](https://badge.fury.io/py/bdns-fetch)\n[](https://www.gnu.org/licenses/gpl-3.0)\n[](https://www.python.org/downloads/)\n\nA comprehensive command-line tool for accessing and processing data from the Base de Datos Nacional de Subvenciones (BDNS) API.\n\n## \u2728 Features\n\n- **29+ Data Extraction Commands**: Covers all key data extraction endpoints from the BDNS API\n- **JSONL Output Format**: Clean JSON Lines format for easy data processing\n- **Flexible Configuration**: Customizable parameters for each command\n- **Clean Error Handling**: User-friendly error messages for API issues\n- **Verbose Logging**: Detailed HTTP request/response logging for debugging\n- **Concurrent Processing**: Built-in pagination and concurrent request handling\n\n## \ud83d\udccb Available Commands\n\nThis tool provides access to **29+ BDNS API data extraction endpoints**. Each command fetches specific data from the Base de Datos Nacional de Subvenciones (BDNS).\n\nFor a complete list of all commands and their parameters, use:\n```bash\nbdns-fetch --help\n```\n\nFor help on a specific command:\n```bash\nbdns-fetch [command-name] --help\n# Example: bdns-fetch organos --help\n```\n\n**\ud83d\udcd6 API Documentation**: Complete endpoint documentation is available at [BDNS API Swagger](https://www.infosubvenciones.es/bdnstrans/estaticos/doc/snpsap-api.json)\n\n## \ud83d\ude80 Quick Start\n\n### Installation\n\n**From PyPI (recommended):**\n```bash\npip install bdns-fetch\n```\n\n**From source:**\n```bash\ngit clone https://github.com/cruzlorite/bdns-fetch.git\ncd bdns-fetch\npoetry install\n```\n\n### CLI Usage\n\n**Getting Help:**\n```bash\n# List all available commands\nbdns-fetch --help\n\n# Get help for a specific command \nbdns-fetch organos --help\nbdns-fetch ayudasestado-busqueda --help\n```\n\n**Basic Examples:**\n```bash\n# Fetch government organs data to file\nbdns-fetch organos --output-file government_organs.jsonl\n\n# Get economic activities (to stdout by default)\nbdns-fetch actividades\n\n# Search state aids with filters and verbose logging\nbdns-fetch --verbose ayudasestado-busqueda \\\n --descripcion \"innovation\" \\\n --num-pages 3 \\\n --pageSize 1000 \\\n --output-file innovation_aids.jsonl\n\n# Get specific strategic plan by ID with debugging\nbdns-fetch --verbose planesestrategicos \\\n --idPES 459 \\\n --output-file plan_459.jsonl\n```\n\n**Common Parameters:**\n- `--output-file FILE`: Save output to file (defaults to stdout) \n- `--verbose, -v`: Enable detailed HTTP request/response logging\n- `--vpd CODE`: Territory code (GE=Spain, specific regions available)\n- `--num-pages N`: Number of pages to fetch (for paginated commands)\n- `--pageSize N`: Records per page (default: 10000, max: 10000)\n- `--max-concurrent-requests N`: Maximum concurrent API requests (default: 5)\n\n**Advanced Search Example:**\n```bash\n# Search concessions with multiple filters and verbose logging\nbdns-fetch --verbose concesiones-busqueda \\\n --descripcion \"research\" \\\n --fechaDesde \"2023-01-01\" \\\n --fechaHasta \"2024-12-31\" \\\n --tipoAdministracion \"C\" \\\n --num-pages 10 \\\n --max-concurrent-requests 8 \\\n --output-file research_concessions.jsonl\n```\n\n## \ud83d\udcd6 More Examples\n\n```bash\n# Download all government organs\nbdns-fetch organos --output-file government_structure.jsonl\n\n# Search for innovation-related subsidies with verbose logging\nbdns-fetch --verbose ayudasestado-busqueda \\\n --descripcion \"innovation\" \\\n --output-file innovation_aids.jsonl\n\n# Get latest calls for proposals\nbdns-fetch convocatorias-ultimas --output-file latest_calls.jsonl\n\n# Search sanctions data with detailed HTTP logging\nbdns-fetch --verbose sanciones-busqueda --output-file sanctions.jsonl\n```\n\nOutput format (JSON Lines):\n```json\n{\"id\": 1, \"descripcion\": \"MINISTERIO DE AGRICULTURA, PESCA Y ALIMENTACI\u00d3N\", \"codigo\": \"E04\"}\n{\"id\": 2, \"descripcion\": \"MINISTERIO DE ASUNTOS EXTERIORES, UNI\u00d3N EUROPEA Y COOPERACI\u00d3N\", \"codigo\": \"E05\"}\n```\n\n## \ud83d\udd27 Error Handling & Debugging\n\n### Clean Error Messages\nThe tool provides user-friendly error messages for common API issues:\n\n```bash\n# Invalid parameter example\n$ bdns-fetch ayudasestado-busqueda --vpd INVALID_PORTAL\nError (ERR_VALIDACION): El par\u00e1metro 'vpd' indica un portal no v\u00e1lido.\n```\n\n### Verbose Logging\nUse the `--verbose` flag to see detailed HTTP request and response information:\n\n```bash\n# Enable verbose logging for debugging\n$ bdns-fetch --verbose ayudasestado-busqueda --pageSize 1\n\nDEBUG:bdns.fetch.fetch_write:HTTP REQUEST: GET https://www.infosubvenciones.es/bdnstrans/api/ayudasestado/busqueda?vpd=GE&pageSize=1&page=0\nDEBUG:bdns.fetch.fetch_write:HTTP RESPONSE: 200 - 163.6ms\nDEBUG:bdns.fetch.fetch_write:Response Headers: {'Date': 'Wed, 20 Aug 2025 21:27:27 GMT', 'Content-Type': 'application/json', ...}\nDEBUG:bdns.fetch.fetch_write:Response Content-Length: 1814 bytes\nDEBUG:bdns.fetch.fetch_write:Response contains 1 items\nDEBUG:bdns.fetch.fetch_write:Total pages available: 5783747\nDEBUG:bdns.fetch.fetch_write:Current page: 0\n```\n\n**Verbose logging includes:**\n- Timestamps for all operations\n- Complete HTTP request URLs\n- Response status codes and timing (in milliseconds) \n- Full HTTP response headers\n- Response content size and structure analysis\n- Error details for troubleshooting\n\n**When to use verbose mode:**\n- Debugging API connectivity issues\n- Analyzing response times and performance\n- Understanding API rate limiting behavior\n- Troubleshooting parameter validation errors\n- Monitoring large data extraction jobs\n\n## \u26a0\ufe0f Current Limitations\n\n### Missing Commands\n**This tool implements 30 out of 46 total API endpoints**. The following 16 commands are **intentionally not included**:\n\n#### Export/Download Endpoints (9 missing)\nThese endpoints generate PDF, CSV, or Excel files instead of JSON data:\n- `convocatorias/exportar` - Export search results to files\n- `convocatorias/ultimas/exportar` - Export latest calls to files\n- `concesiones/exportar` - Export concessions search to files\n- `ayudasestado/exportar` - Export state aids search to files\n- `minimis/exportar` - Export minimis search to files\n- `grandesbeneficiarios/exportar` - Export large beneficiaries to files\n- `partidospoliticos/exportar` - Export political parties search to files\n- `planesestrategicos/exportar` - Export strategic plans to files\n- `sanciones/exportar` - Export sanctions search to files\n\n**Why excluded**: These endpoints return binary file data (PDF/Excel/CSV) instead of structured JSON data, making them unsuitable for a CLI tool focused on data extraction and processing.\n\n#### Portal Configuration Endpoints (2 missing)\n- `vpd/{vpd}/configuracion` - Get portal navigation configuration\n- `enlaces` - Get portal links and micro-windows\n\n**Why excluded**: These endpoints return web portal configuration data (navigation menus, links) that are not relevant for data extraction purposes.\n\n#### Subscription/Alert System (5 missing)\n- `suscripciones/alta` - Create new alert subscription\n- `suscripciones/altaidentificado` - Create subscription with token\n- `suscripciones/activar` - Activate subscription\n- `suscripciones/login` - Login to subscription service\n- `suscripciones/cerrar` - Close session\n- `suscripciones/detalle` - Get subscription details\n- `suscripciones/modificar` - Modify subscription\n- `suscripciones/anular` - Cancel subscription\n- `suscripciones/reactivar` - Reactivate subscription\n- `suscripciones/recuperarcontrasena` - Recover password\n- `suscripciones/restablecercontrasena` - Reset password\n\n**Why excluded**: The subscription system requires user authentication, password management, and email verification - functionality better suited for the official web portal rather than a CLI data extraction tool.\n\n### Recommended Usage\n- **Test First**: Always test commands with small datasets before large-scale usage\n- **Use Verbose Mode**: Enable `--verbose` for debugging API issues or monitoring large extractions\n- **Check API Status**: Verify that specific endpoints are working before relying on them for production use\n- **Monitor for Updates**: The Spanish government may update the API without notice\n\n## \ud83d\udee0\ufe0f Development\n\n### Prerequisites\n- Python 3.11+\n- Poetry for dependency management\n\n### Development Setup\n```bash\n# Clone and setup\ngit clone https://github.com/cruzlorite/bdns-fetch.git\ncd bdns-fetch\npoetry install --with dev\n\n# Available Make targets\nmake help # Show all available targets\nmake install # Install project dependencies \nmake dev-install # Install with development dependencies\nmake lint # Run code linting with ruff\nmake format # Format code with ruff formatter\nmake test-integration # Run integration tests\nmake clean # Remove build artifacts\nmake all # Install, lint, format, and test\n```\n\n## \ud83d\ude4f Acknowledgments\n\nThis project is inspired by previous work from [Jaime Ortega Obreg\u00f3n](https://github.com/JaimeObregon/subvenciones/tree/main).\n\n## \ud83d\udcdc License\n\nThis project is licensed under the GNU General Public License v3.0 (GPL-3.0). See the [LICENSE](LICENSE) file for details.\n\n",
"bugtrack_url": null,
"license": "GPL-3.0",
"summary": "A comprehensive command-line tool for accessing Spanish government BDNS subsidies data",
"version": "2.1.0",
"project_urls": {
"Documentation": "https://github.com/cruzlorite/bdns-fetch#readme",
"Homepage": "https://github.com/cruzlorite/bdns-fetch",
"Repository": "https://github.com/cruzlorite/bdns-fetch"
},
"split_keywords": [
"bdns",
" subsidies",
" spanish-government",
" api",
" data",
" cli"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "ef75eecc472034cf51a9a5d48c294a93987b486f6df081fea2c4670b11813fd8",
"md5": "a6dcef5372cf90d35fc52a5a39da5264",
"sha256": "b8b3357a4362b2d3fa86baec49990e9aad341887b9db62e2017851fc847fb3bf"
},
"downloads": -1,
"filename": "bdns_fetch-2.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a6dcef5372cf90d35fc52a5a39da5264",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 62391,
"upload_time": "2025-08-20T21:52:11",
"upload_time_iso_8601": "2025-08-20T21:52:11.064272Z",
"url": "https://files.pythonhosted.org/packages/ef/75/eecc472034cf51a9a5d48c294a93987b486f6df081fea2c4670b11813fd8/bdns_fetch-2.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "fabc6545557829f35a1ecd7fe8423cc2a80f51b72ff246d52c3164c63e343ce2",
"md5": "8e70c6bf6c872ae6b17e3454152943e7",
"sha256": "b2145672682bb306f77f62f1cb73fa95192959b19ac352db3e0d28722886335b"
},
"downloads": -1,
"filename": "bdns_fetch-2.1.0.tar.gz",
"has_sig": false,
"md5_digest": "8e70c6bf6c872ae6b17e3454152943e7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 34138,
"upload_time": "2025-08-20T21:52:12",
"upload_time_iso_8601": "2025-08-20T21:52:12.275247Z",
"url": "https://files.pythonhosted.org/packages/fa/bc/6545557829f35a1ecd7fe8423cc2a80f51b72ff246d52c3164c63e343ce2/bdns_fetch-2.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-20 21:52:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cruzlorite",
"github_project": "bdns-fetch#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "bdns-fetch"
}
JSON
