# TradingView Scraper
[](https://www.python.org/downloads/release/python-380/)
[](https://github.com/psf/black)
[](https://opensource.org/licenses/MIT)
This is a Python library for scraping ideas and indicators from [TradingView.com](https://www.tradingview.com). The goal is to develop this package to scrape anything on [TradingView.com](https://www.tradingview.com) with real-time responses.
**Thanks to contributors!**
## To-Do List
- Export
- [x] Export as a `CSV` file
- [x] Export as `JSON`
- Scrape Symbol Subpages:
- [x] [Ideas](https://www.tradingview.com/symbols/BTCUSD/ideas/)
- [x] [Indicators](https://www.tradingview.com/symbols/BTCUSD/technicals/)
- [ ] [Overview](https://www.tradingview.com/symbols/BTCUSD/)
- [x] [News](https://www.tradingview.com/symbols/BTCUSD/news/)
- [ ] [Minds](https://www.tradingview.com/symbols/BTCUSD/minds/)
- [x] [Technical](https://www.tradingview.com/symbols/BTCUSD/technicals/)
- [ ] [Market](https://www.tradingview.com/symbols/BTCUSD/markets/)
- [ ] [Screener](https://www.tradingview.com/screener/)
- [x] Get data using TradingView WebSocket
- [ ] Additional suggestions welcome!
### To be aware of the latest changes, go to the [end of this page](https://github.com/mnwato/tradingview-scraper#changes).
## Features
- **Idea page Scraping**
- Title
- Paragraph
- Preview Image
- Author
- Comments Count
- Boosts Count
- Publication Datetime
- Is Updated
- Idea Strategy
- **News page Scraping**
- Breadcrumbs
- Title
- Published datetime
- Related symbols
- Body
- Tags
- **Webpage Scraping Options**
- Scrape All Pages
- Scrape a Specific Range of Pages
- **Indicator Extraction**
- Extract values for indicators like `RSI`, `Stoch.K`, etc.
- [Full list of indicators](https://github.com/mnwato/tradingview-scraper/blob/dev/tradingview_scraper/indicators.txt)
- **Real-Time data Extraction
- OHLCV
- Watchlist
- **Export Formats**
- CSV
- JSON
- **Output Format**
- Returns data in JSON format
Here’s a revised version of the Installation section that enhances clarity and encourages user engagement:
## Installation
To get started with the TradingView Scraper library, follow these simple steps:
1. **Open your terminal**: Launch your preferred command line interface.
2. **Install the package**: Run the following command to install the TradingView Scraper:
```sh
pip install tradingview-scraper
```
3. **Upgrade if necessary**: If you already have the library installed and want to upgrade to the latest version, use:
```sh
pip install --upgrade --no-cache tradingview-scraper
```
Here’s a revised version of the Examples section, focusing on clarity, ease of understanding, and providing essential information about default values:
## Examples
### 1. Fast Running (Default Parameters)
To quickly scrape ideas using default settings, use the following code:
```python
from symbols.ideas import Ideas
# Initialize the Ideas scraper with default parameters
ideas_scraper = Ideas() # Default: export_result=False, export_type='json'
ideas = ideas_scraper.scrape() # Default symbol: 'BTCUSD'
print("Ideas:", ideas)
```
**Default Parameters:**
- `export_result`: `False` (no file will be saved)
- `export_type`: `'json'` (output format)
### 2. Getting Ideas for a Specific Symbol, Export Type, and Pages
To scrape ideas for a specific symbol and export them as a CSV file, you can specify the parameters:
```python
from symbols.ideas import Ideas
# Initialize the Ideas scraper with custom parameters
ideas_scraper = Ideas(
export_result=True, # Set to True to save the results
export_type='csv' # Specify the export type (json or csv)
)
# Scrape ideas for the ETHUSD symbol, from page 1 to page 2
ideas = ideas_scraper.scrape(
symbol="ETHUSD",
startPage=1,
endPage=2,
sort="popular" # Could be 'popupar' or 'recent'
)
print("Ideas:", ideas)
```
**Output Format:**
The output will always be a list of Python dictionaries, structured as follows:
```json
[
{
"title": "Bitcoin -65% crash, ETH -83%, DOGE -89%, SHIBA -90%",
"paragraph": "Name your altcoin in the comment section, and I will do a technical analysis for you!\n\nThe crypto market looks ...",
"preview_image": "https://s3.tradingview.com/6/6VQphWH6_mid.png",
"author": "Xanrox",
"comments_count": "295",
"boosts_count": "678",
"publication_datetime": "2024-08-18T05:55:19.000Z",
"is_updated": "True",
"idea_strategy": "Short"
}
]
```
- When `export_result=True`, the default `export_type` is `'json'`, and a JSON file will be saved in the `/export` directory. To save as CSV, set `export_type='csv'`.
### 3. Getting Indicators Status
To scrape the status of specific indicators, use the following code:
```python
from tradingview_scraper.symbols.technicals import Indicators
# Initialize the Indicators scraper with export options
indicators_scraper = Indicators(export_result=True, export_type='json')
# Scrape indicators for the BTCUSD symbol from the BINANCE exchange
indicators = indicators_scraper.scrape(
exchange="BINANCE",
symbol="BTCUSD",
timeframe="1d",
indicators=["RSI", "Stoch.K"]
)
print("Indicators:", indicators)
```
**Output:**
```json
{"RSI": "46.34926112", "Stoch.K": "40.40173723"}
```
### 4. Getting All Indicators
If you want to retrieve all available indicators for a symbol, set `allIndicators=True`:
```python
from tradingview_scraper.symbols.technicals import Indicators
# Scrape all indicators for the BTCUSD symbol
indicators_scraper = Indicators(export_result=True, export_type='json')
indicators = indicators_scraper.scrape(
symbol="BTCUSD",
timeframe="4h",
allIndicators=True
)
print("All Indicators:", indicators)
```
### 5. Getting News Headlines/Content
```python
# Create an instance of the NewsScraper with export options
news_scraper = NewsScraper(export_result=True, export_type='json')
# Retrieve news headlines from a specific provider
news_headlines = news_scraper.scrape_headlines(
symbol='BTCUSD', # Uncomment and specify if needed
exchange='BINANCE', # Uncomment and specify if needed
# provider='newsbtc', # Specify the news provider
# area='world', # Specify the geographical area
# section='all', # Specify the section of news
sort='latest'
)
# Retrieve detailed news content for a specific story
news_content = news_scraper.scrape_news_content(
story_path=news_headlines[0]['storyPath'] # Specify the story path from scraped headlines
)
```
- Retrieve news by symbol:
- Both `symbol` and `exchange` are required parameters
- Filter result by:
- `area`, `provider` and `section` can be specified to refine the news results.
#### Output (news headline):
```json
[
{
"breadcrumbs": "News > U.Today > Bitcoin ETFs Record Enormous Outflows",
"title": "Bitcoin ETFs Record Enormous Outflows",
"published_datetime": "Wed, 04 Sep 2024 07:55:38 GMT",
"related_symbols": [
{
"name": "BTCUSDT",
"logo": "https://s3-symbol-logo.tradingview.com/crypto/XTVCUSDT.svg"
}
],
"body": ["""<List of text page content>"""],
"tags": ["Crypto", "U.Today"]}
]
```
#### Output (news content):
```json
[
{
"id": "tag:reuters.com,2024:newsml_L1N3KM09S:0",
"title": "Goldman Sachs sees biggest boost to US economy from Harris win",
"provider": "reuters",
"sourceLogoId": "reuters",
"published": 1725443676,
"source": "Reuters",
"urgency": 2,
"permission": "preview",
"relatedSymbols": [
{
"symbol": "BITMEX:XBTETH.P",
"currency-logoid": "country/US",
"base-currency-logoid": "crypto/XTVCBTC"
},
{
"symbol": "ICEUS:DXY",
"logoid": "indices/u-s-dollar-index"
}
],
"storyPath": "/news/reuters.com,2024:newsml_L1N3KM09S:0-goldman-sachs-sees-biggest-boost-to-us-economy-from-harris-win/"
}
]
```
### 6. Fetching Real-Time Trading Data
- The RealTimeData class provides functionality to fetch real-time trading data from various exchanges. Below are usage examples demonstrating how to retrieve the latest trade information and OHLCV (Open, High, Low, Close, Volume) data.
#### Retrieve OHLCV Data:
- Open
- High
- Low
- Close
- Volume
##### Example:
```python
# Create an instance of the RealTimeData class
real_time_data = RealTimeData()
# Retrieve OHLCV data for a specific symbol
data_generator = real_time_data.get_ohlcv(exchange_symbol="BINANCE:BTCUSDT")
```
#### Retrieve Watchlist Market Info
- You can send a list of exchange:symbol to get real-time market information, including:
- volume
- `lp_time` (Last Price Time)
- `lp` (Last Price)
- `ch` (Change in Price)
- `chp` (Change in Percent)
##### Example:
```python
# Create an instance of the RealTimeData class
real_time_data = RealTimeData()
# Define the exchange symbols for which to fetch data
exchange_symbol = ["BINANCE:BTCUSDT", "BINANCE:ETHUSDT", "FXOPEN:XAUUSD"]
# Retrieve the latest trade information for a specific symbol
data_generator = real_time_data.get_latest_trade_info(exchange_symbol=exchange_symbol)
```
#### Printing Results
- To display the real-time data packets, iterate over the generator as follows:
```python
for packet in data_generator:
print('-' * 50)
print(packet)
```
#### Output Examples
##### Output (OHLCV):
```text
{'m': 'du', 'p': ['cs_qgmbtglzdudl', {'sds_1': {'s': [{'i': 9, 'v': [1734082440.0, 100010.0, 100010.01, 100006.27, 100006.27, 1.3242]}], 'ns': {'d': '', 'indexes': 'nochange'}, 't': 's1', 'lbs': {'bar_close_time': 1734082500}}}]}
```
##### Output (Watchlist market info):
```text
{'m': 'qsd', 'p': ['qs_folpuhzgowtu', {'n': 'BINANCE:BTCUSDT', 's': 'ok', 'v': {'volume': 6817.46425, 'lp_time': 1734082521, 'lp': 99957.9, 'chp': -0.05, 'ch': -46.39}}]}
```
## Changes:
- Release `0.4.0`:
Update exchange list
Add real-time price streaming
- Release `0.3.2`:
Support timeframe to get Indicators
- Release `0.3.0`:
Add news scraper
- Release `0.2.9`:
Refactor for new TradingView structure
- Release `0.1.0`:
The name of `ClassA` changed to `Ideas`
## License:
```
[MIT]
```
Raw data
{
"_id": null,
"home_page": "https://github.com/mnwato/tradingview-scraper",
"name": "tradingview-scraper",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "tradingview, scraper, python, crawler, financial",
"author": "Mostafa Najmi",
"author_email": "m.n.irib@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/d2/2c/acfd1f8eddef644b9bf38a4e79e1c16c8aa3c21f4415c3cc8d8ae0b479a5/tradingview-scraper-0.4.0.tar.gz",
"platform": null,
"description": "# TradingView Scraper\n[](https://www.python.org/downloads/release/python-380/)\n[](https://github.com/psf/black)\n[](https://opensource.org/licenses/MIT)\n\n\nThis is a Python library for scraping ideas and indicators from [TradingView.com](https://www.tradingview.com). The goal is to develop this package to scrape anything on [TradingView.com](https://www.tradingview.com) with real-time responses. \n**Thanks to contributors!**\n\n\n## To-Do List\n- Export\n - [x] Export as a `CSV` file\n - [x] Export as `JSON`\n- Scrape Symbol Subpages:\n - [x] [Ideas](https://www.tradingview.com/symbols/BTCUSD/ideas/)\n - [x] [Indicators](https://www.tradingview.com/symbols/BTCUSD/technicals/)\n - [ ] [Overview](https://www.tradingview.com/symbols/BTCUSD/)\n - [x] [News](https://www.tradingview.com/symbols/BTCUSD/news/)\n - [ ] [Minds](https://www.tradingview.com/symbols/BTCUSD/minds/)\n - [x] [Technical](https://www.tradingview.com/symbols/BTCUSD/technicals/)\n - [ ] [Market](https://www.tradingview.com/symbols/BTCUSD/markets/)\n - [ ] [Screener](https://www.tradingview.com/screener/)\n - [x] Get data using TradingView WebSocket\n - [ ] Additional suggestions welcome!\n\n### To be aware of the latest changes, go to the [end of this page](https://github.com/mnwato/tradingview-scraper#changes).\n\n## Features\n\n- **Idea page Scraping**\n - Title\n - Paragraph\n - Preview Image\n - Author\n - Comments Count\n - Boosts Count\n - Publication Datetime\n - Is Updated\n - Idea Strategy\n\n- **News page Scraping**\n - Breadcrumbs\n - Title\n - Published datetime\n - Related symbols\n - Body\n - Tags\n\n- **Webpage Scraping Options**\n - Scrape All Pages\n - Scrape a Specific Range of Pages\n\n- **Indicator Extraction**\n - Extract values for indicators like `RSI`, `Stoch.K`, etc. \n - [Full list of indicators](https://github.com/mnwato/tradingview-scraper/blob/dev/tradingview_scraper/indicators.txt)\n\n- **Real-Time data Extraction\n - OHLCV\n - Watchlist\n\n- **Export Formats**\n - CSV\n - JSON\n\n- **Output Format**\n - Returns data in JSON format\n\nHere\u2019s a revised version of the Installation section that enhances clarity and encourages user engagement:\n\n\n## Installation\n\nTo get started with the TradingView Scraper library, follow these simple steps:\n\n1. **Open your terminal**: Launch your preferred command line interface.\n\n2. **Install the package**: Run the following command to install the TradingView Scraper:\n ```sh\n pip install tradingview-scraper\n ```\n\n3. **Upgrade if necessary**: If you already have the library installed and want to upgrade to the latest version, use:\n ```sh\n pip install --upgrade --no-cache tradingview-scraper\n ```\n\nHere\u2019s a revised version of the Examples section, focusing on clarity, ease of understanding, and providing essential information about default values:\n\n\n## Examples\n\n### 1. Fast Running (Default Parameters)\nTo quickly scrape ideas using default settings, use the following code:\n```python\nfrom symbols.ideas import Ideas\n\n# Initialize the Ideas scraper with default parameters\nideas_scraper = Ideas() # Default: export_result=False, export_type='json'\nideas = ideas_scraper.scrape() # Default symbol: 'BTCUSD'\nprint(\"Ideas:\", ideas)\n```\n**Default Parameters:**\n- `export_result`: `False` (no file will be saved)\n- `export_type`: `'json'` (output format)\n\n### 2. Getting Ideas for a Specific Symbol, Export Type, and Pages\nTo scrape ideas for a specific symbol and export them as a CSV file, you can specify the parameters:\n```python\nfrom symbols.ideas import Ideas\n\n# Initialize the Ideas scraper with custom parameters\nideas_scraper = Ideas(\n export_result=True, # Set to True to save the results\n export_type='csv' # Specify the export type (json or csv)\n)\n\n# Scrape ideas for the ETHUSD symbol, from page 1 to page 2\nideas = ideas_scraper.scrape(\n symbol=\"ETHUSD\",\n startPage=1,\n endPage=2,\n sort=\"popular\" # Could be 'popupar' or 'recent'\n)\nprint(\"Ideas:\", ideas)\n```\n\n**Output Format:**\nThe output will always be a list of Python dictionaries, structured as follows:\n```json\n[\n {\n \"title\": \"Bitcoin -65% crash, ETH -83%, DOGE -89%, SHIBA -90%\",\n \"paragraph\": \"Name your altcoin in the comment section, and I will do a technical analysis for you!\\n\\nThe crypto market looks ...\",\n \"preview_image\": \"https://s3.tradingview.com/6/6VQphWH6_mid.png\",\n \"author\": \"Xanrox\",\n \"comments_count\": \"295\",\n \"boosts_count\": \"678\",\n \"publication_datetime\": \"2024-08-18T05:55:19.000Z\",\n \"is_updated\": \"True\",\n \"idea_strategy\": \"Short\"\n }\n]\n```\n- When `export_result=True`, the default `export_type` is `'json'`, and a JSON file will be saved in the `/export` directory. To save as CSV, set `export_type='csv'`.\n\n### 3. Getting Indicators Status\nTo scrape the status of specific indicators, use the following code:\n```python\nfrom tradingview_scraper.symbols.technicals import Indicators\n\n# Initialize the Indicators scraper with export options\nindicators_scraper = Indicators(export_result=True, export_type='json')\n\n# Scrape indicators for the BTCUSD symbol from the BINANCE exchange\nindicators = indicators_scraper.scrape(\n exchange=\"BINANCE\",\n symbol=\"BTCUSD\",\n timeframe=\"1d\",\n indicators=[\"RSI\", \"Stoch.K\"]\n)\nprint(\"Indicators:\", indicators)\n```\n\n**Output:**\n```json\n{\"RSI\": \"46.34926112\", \"Stoch.K\": \"40.40173723\"}\n```\n\n### 4. Getting All Indicators\nIf you want to retrieve all available indicators for a symbol, set `allIndicators=True`:\n```python\nfrom tradingview_scraper.symbols.technicals import Indicators\n\n# Scrape all indicators for the BTCUSD symbol\nindicators_scraper = Indicators(export_result=True, export_type='json')\nindicators = indicators_scraper.scrape(\n symbol=\"BTCUSD\",\n timeframe=\"4h\",\n allIndicators=True\n)\nprint(\"All Indicators:\", indicators)\n```\n\n### 5. Getting News Headlines/Content\n```python\n# Create an instance of the NewsScraper with export options\nnews_scraper = NewsScraper(export_result=True, export_type='json')\n\n# Retrieve news headlines from a specific provider\nnews_headlines = news_scraper.scrape_headlines(\n symbol='BTCUSD', # Uncomment and specify if needed\n exchange='BINANCE', # Uncomment and specify if needed\n # provider='newsbtc', # Specify the news provider\n # area='world', # Specify the geographical area\n # section='all', # Specify the section of news\n sort='latest'\n)\n\n# Retrieve detailed news content for a specific story\nnews_content = news_scraper.scrape_news_content(\n story_path=news_headlines[0]['storyPath'] # Specify the story path from scraped headlines\n)\n```\n- Retrieve news by symbol:\n - Both `symbol` and `exchange` are required parameters\n- Filter result by:\n - `area`, `provider` and `section` can be specified to refine the news results.\n\n#### Output (news headline):\n```json\n[\n {\n \"breadcrumbs\": \"News > U.Today > Bitcoin ETFs Record Enormous Outflows\",\n \"title\": \"Bitcoin ETFs Record Enormous Outflows\",\n \"published_datetime\": \"Wed, 04 Sep 2024 07:55:38 GMT\",\n \"related_symbols\": [\n {\n \"name\": \"BTCUSDT\",\n \"logo\": \"https://s3-symbol-logo.tradingview.com/crypto/XTVCUSDT.svg\"\n }\n ],\n \"body\": [\"\"\"<List of text page content>\"\"\"],\n \"tags\": [\"Crypto\", \"U.Today\"]}\n]\n```\n#### Output (news content):\n```json\n[\n {\n \"id\": \"tag:reuters.com,2024:newsml_L1N3KM09S:0\",\n \"title\": \"Goldman Sachs sees biggest boost to US economy from Harris win\",\n \"provider\": \"reuters\",\n \"sourceLogoId\": \"reuters\",\n \"published\": 1725443676,\n \"source\": \"Reuters\",\n \"urgency\": 2,\n \"permission\": \"preview\",\n \"relatedSymbols\": [\n {\n \"symbol\": \"BITMEX:XBTETH.P\",\n \"currency-logoid\": \"country/US\",\n \"base-currency-logoid\": \"crypto/XTVCBTC\"\n },\n {\n \"symbol\": \"ICEUS:DXY\",\n \"logoid\": \"indices/u-s-dollar-index\"\n }\n ],\n \"storyPath\": \"/news/reuters.com,2024:newsml_L1N3KM09S:0-goldman-sachs-sees-biggest-boost-to-us-economy-from-harris-win/\"\n }\n]\n```\n\n### 6. Fetching Real-Time Trading Data\n- The RealTimeData class provides functionality to fetch real-time trading data from various exchanges. Below are usage examples demonstrating how to retrieve the latest trade information and OHLCV (Open, High, Low, Close, Volume) data.\n\n#### Retrieve OHLCV Data:\n - Open\n - High\n - Low\n - Close\n - Volume\n##### Example:\n```python\n# Create an instance of the RealTimeData class\nreal_time_data = RealTimeData()\n\n# Retrieve OHLCV data for a specific symbol\ndata_generator = real_time_data.get_ohlcv(exchange_symbol=\"BINANCE:BTCUSDT\")\n```\n\n#### Retrieve Watchlist Market Info\n - You can send a list of exchange:symbol to get real-time market information, including:\n - volume\n - `lp_time` (Last Price Time)\n - `lp` (Last Price)\n - `ch` (Change in Price)\n - `chp` (Change in Percent)\n##### Example:\n```python\n# Create an instance of the RealTimeData class\nreal_time_data = RealTimeData()\n\n# Define the exchange symbols for which to fetch data\nexchange_symbol = [\"BINANCE:BTCUSDT\", \"BINANCE:ETHUSDT\", \"FXOPEN:XAUUSD\"]\n\n# Retrieve the latest trade information for a specific symbol\ndata_generator = real_time_data.get_latest_trade_info(exchange_symbol=exchange_symbol)\n```\n\n#### Printing Results\n- To display the real-time data packets, iterate over the generator as follows:\n```python\nfor packet in data_generator:\n print('-' * 50)\n print(packet)\n```\n#### Output Examples\n##### Output (OHLCV):\n```text\n{'m': 'du', 'p': ['cs_qgmbtglzdudl', {'sds_1': {'s': [{'i': 9, 'v': [1734082440.0, 100010.0, 100010.01, 100006.27, 100006.27, 1.3242]}], 'ns': {'d': '', 'indexes': 'nochange'}, 't': 's1', 'lbs': {'bar_close_time': 1734082500}}}]}\n```\n##### Output (Watchlist market info):\n```text\n{'m': 'qsd', 'p': ['qs_folpuhzgowtu', {'n': 'BINANCE:BTCUSDT', 's': 'ok', 'v': {'volume': 6817.46425, 'lp_time': 1734082521, 'lp': 99957.9, 'chp': -0.05, 'ch': -46.39}}]}\n```\n\n\n\n## Changes:\n- Release `0.4.0`: \n Update exchange list\n Add real-time price streaming\n- Release `0.3.2`: \n Support timeframe to get Indicators\n- Release `0.3.0`: \n Add news scraper\n- Release `0.2.9`: \n Refactor for new TradingView structure\n- Release `0.1.0`: \n The name of `ClassA` changed to `Ideas`\n\n## License:\n```\n[MIT]\n```",
"bugtrack_url": null,
"license": "MIT",
"summary": "Tradingview scraper tool",
"version": "0.4.0",
"project_urls": {
"Download": "https://github.com/mnwato/tradingview-scraper/archive/refs/tags/0.4.0.zip",
"Homepage": "https://github.com/mnwato/tradingview-scraper"
},
"split_keywords": [
"tradingview",
" scraper",
" python",
" crawler",
" financial"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d22cacfd1f8eddef644b9bf38a4e79e1c16c8aa3c21f4415c3cc8d8ae0b479a5",
"md5": "ca097f031d0f915fc3c8fb0ecb9e4358",
"sha256": "8f6cb6dd1e4628ec3b677461a4f638c9b09344d7e8f316d7b48ab6461b1cfe8c"
},
"downloads": -1,
"filename": "tradingview-scraper-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "ca097f031d0f915fc3c8fb0ecb9e4358",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 26083,
"upload_time": "2025-01-14T13:03:41",
"upload_time_iso_8601": "2025-01-14T13:03:41.064867Z",
"url": "https://files.pythonhosted.org/packages/d2/2c/acfd1f8eddef644b9bf38a4e79e1c16c8aa3c21f4415c3cc8d8ae0b479a5/tradingview-scraper-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-14 13:03:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mnwato",
"github_project": "tradingview-scraper",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "tradingview-scraper"
}
JSON
