# apple-news-to-sqlite
Export Apple News Saved Stories to SQLite
## Install
pip install apple-news-to-sqlite
## Source code
[apple-news-to-sqlite](https://github.com/RhetTbull/apple-news-to-sqlite)
## Usage
apple-news-to-sqlite articles.db
apple-news-to-sqlite --dump
Your Terminal app will require full disk access in order to access the saved article database in the Apple News app sandbox.
## CLI help
<!-- [[[cog
import cog
from apple_news_to_sqlite.cli import cli
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(cli, ["--help"])
help = result.output.replace("Usage: cli", "Usage: apple-news-to-sqlite")
cog.out(
"```\n{}\n```".format(help)
)
]]] -->
```
Usage: apple-news-to-sqlite [OPTIONS] [DB_PATH]
Export your Apple News saved stories/articles to a SQLite database
Example usage:
apple-news-to-sqlite articles.db
This will populate articles.db with an "articles" table containing information
about your saved articles.
Notes:
The contents of the articles themselves are not stored in the database, only
metadata about the article such as title, author, url, etc.
The date the article was saved is in GMT.
Options:
--version Show the version and exit.
--dump, --json Print saved stories to standard output as JSON.
--all Process all saved articles; if not specified, only saved
articles that have not previously been stored in the database
are processed.
--schema Create database schema and exit.
--help Show this message and exit.
```
<!-- [[[end]]] -->
## Using apple-news-to-sqlite in your own Python code
`get_saved_articles()` returns a list of dictionaries, each representing a saved article with the
following keys:
* id: str
* date: datetime.datetime
* url: str
* title: str
* description: str
* image: str
* author: str
```pycon
>>> from apple_news_to_sqlite import get_saved_articles
>>> articles = get_saved_articles()
```
## How it works
Through reverse engineering, it was determined that the Apple News app stores
saved articles in a file called `reading-list` in the following directory:
`~/Library/Containers/com.apple.news/Data/Library/Application Support/com.apple.news/com.apple.news.public-com.apple.news.private-production/`
This format of this file is unknown but it is a binary file that contains two embedded
[binary plist](https://medium.com/@karaiskc/understanding-apples-binary-property-list-format-281e6da00dbd)
files. The first contains an [NSKeyedArchiver](https://developer.apple.com/documentation/foundation/nskeyedarchiver)
object which I have not yet inspected. The second bplist contains a list of saved article IDs along with the date
they were saved. The article IDs are used to look up the article data on Apple's News site and the article data
is extracted with [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/).
## Testing
The code is at the "it works on my machine" stage of testing. (M1 Mini, macOS Ventura 13.1)
I've also tested this on macOS Catalina 10.15.7 and it appears to work correctly.
If it doesn't work for you, please open an issue!
## Contributing
Contributions of all types are welcome! Fork the repo, make a branch, and submit a PR.
See [README_DEV.md](README_DEV.md) for developer notes.
## Thanks
Thanks to [Simon Willison](https://simonwillison.net/) who inspired this project
with his excellent "everything-to-sqlite" [dogsheep](https://github.com/dogsheep) project.
Thanks Simon also for the excellent tools
[sqlite-utils](https://github.com/simonw/sqlite-utils) and [Datasette](https://datasette.io).
Thanks also to [Dave Bullock](https://github.com/eecue) who inspired this project and helped
tremendously with the reverse engineering and initial code.
## License
MIT License
Raw data
{
"_id": null,
"home_page": "https://github.com/RhetTbull/apple-news-to-sqlite",
"name": "apple-news-to-sqlite",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<4.0",
"maintainer_email": "",
"keywords": "cli,mac,macos,sqlite,news,apple",
"author": "Rhet Turnbull",
"author_email": "rturnbull+git@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/16/57/330539f7c8632d7f06e3b3f660780c755b1a5138f9821fdfd00b0d54ecff/apple_news_to_sqlite-0.5.2.tar.gz",
"platform": null,
"description": "# apple-news-to-sqlite\n\nExport Apple News Saved Stories to SQLite\n\n## Install\n\n pip install apple-news-to-sqlite\n\n## Source code\n\n[apple-news-to-sqlite](https://github.com/RhetTbull/apple-news-to-sqlite)\n\n## Usage\n\n apple-news-to-sqlite articles.db\n \n apple-news-to-sqlite --dump\n\nYour Terminal app will require full disk access in order to access the saved article database in the Apple News app sandbox.\n\n## CLI help\n\n<!-- [[[cog\nimport cog\nfrom apple_news_to_sqlite.cli import cli\nfrom click.testing import CliRunner\nrunner = CliRunner()\nresult = runner.invoke(cli, [\"--help\"])\nhelp = result.output.replace(\"Usage: cli\", \"Usage: apple-news-to-sqlite\")\ncog.out(\n \"```\\n{}\\n```\".format(help)\n)\n]]] -->\n```\nUsage: apple-news-to-sqlite [OPTIONS] [DB_PATH]\n\n Export your Apple News saved stories/articles to a SQLite database\n\n Example usage:\n\n apple-news-to-sqlite articles.db\n\n This will populate articles.db with an \"articles\" table containing information\n about your saved articles.\n\n Notes:\n\n The contents of the articles themselves are not stored in the database, only\n metadata about the article such as title, author, url, etc.\n\n The date the article was saved is in GMT.\n\nOptions:\n --version Show the version and exit.\n --dump, --json Print saved stories to standard output as JSON.\n --all Process all saved articles; if not specified, only saved\n articles that have not previously been stored in the database\n are processed.\n --schema Create database schema and exit.\n --help Show this message and exit.\n\n```\n<!-- [[[end]]] -->\n\n## Using apple-news-to-sqlite in your own Python code\n\n`get_saved_articles()` returns a list of dictionaries, each representing a saved article with the\nfollowing keys:\n\n * id: str\n * date: datetime.datetime\n * url: str\n * title: str\n * description: str\n * image: str\n * author: str\n\n```pycon\n>>> from apple_news_to_sqlite import get_saved_articles\n>>> articles = get_saved_articles()\n```\n\n## How it works\n\nThrough reverse engineering, it was determined that the Apple News app stores\nsaved articles in a file called `reading-list` in the following directory:\n\n`~/Library/Containers/com.apple.news/Data/Library/Application Support/com.apple.news/com.apple.news.public-com.apple.news.private-production/`\n\nThis format of this file is unknown but it is a binary file that contains two embedded \n[binary plist](https://medium.com/@karaiskc/understanding-apples-binary-property-list-format-281e6da00dbd)\nfiles. The first contains an [NSKeyedArchiver](https://developer.apple.com/documentation/foundation/nskeyedarchiver)\nobject which I have not yet inspected. The second bplist contains a list of saved article IDs along with the date\nthey were saved. The article IDs are used to look up the article data on Apple's News site and the article data\nis extracted with [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/).\n\n## Testing\n\nThe code is at the \"it works on my machine\" stage of testing. (M1 Mini, macOS Ventura 13.1)\n\nI've also tested this on macOS Catalina 10.15.7 and it appears to work correctly.\n\nIf it doesn't work for you, please open an issue!\n\n## Contributing\n\nContributions of all types are welcome! Fork the repo, make a branch, and submit a PR.\n\nSee [README_DEV.md](README_DEV.md) for developer notes.\n\n## Thanks\n\nThanks to [Simon Willison](https://simonwillison.net/) who inspired this project\nwith his excellent \"everything-to-sqlite\" [dogsheep](https://github.com/dogsheep) project.\n\nThanks Simon also for the excellent tools\n[sqlite-utils](https://github.com/simonw/sqlite-utils) and [Datasette](https://datasette.io).\n\nThanks also to [Dave Bullock](https://github.com/eecue) who inspired this project and helped\ntremendously with the reverse engineering and initial code.\n\n## License\n\nMIT License\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Export \"Saved Stories\" from Apple News to SQLite",
"version": "0.5.2",
"split_keywords": [
"cli",
"mac",
"macos",
"sqlite",
"news",
"apple"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7d11a858ce4169ad4beb5e6261ed78bbde18c342bc65b612009480b3385e5a5c",
"md5": "8c3d0e131c829eef8753a76f7044df16",
"sha256": "996db85a87ce22ac2c2ccbc09d434d7e6d84f40cc2946640cd5483a94ae19d02"
},
"downloads": -1,
"filename": "apple_news_to_sqlite-0.5.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8c3d0e131c829eef8753a76f7044df16",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<4.0",
"size": 8508,
"upload_time": "2023-03-16T04:29:55",
"upload_time_iso_8601": "2023-03-16T04:29:55.885869Z",
"url": "https://files.pythonhosted.org/packages/7d/11/a858ce4169ad4beb5e6261ed78bbde18c342bc65b612009480b3385e5a5c/apple_news_to_sqlite-0.5.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1657330539f7c8632d7f06e3b3f660780c755b1a5138f9821fdfd00b0d54ecff",
"md5": "c0e1817e3186b506e6f28866b2108df8",
"sha256": "48a00bb9750fcbed8f13514238551ece65588eaedac811581134fac521f2d9e6"
},
"downloads": -1,
"filename": "apple_news_to_sqlite-0.5.2.tar.gz",
"has_sig": false,
"md5_digest": "c0e1817e3186b506e6f28866b2108df8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<4.0",
"size": 7644,
"upload_time": "2023-03-16T04:29:57",
"upload_time_iso_8601": "2023-03-16T04:29:57.617497Z",
"url": "https://files.pythonhosted.org/packages/16/57/330539f7c8632d7f06e3b3f660780c755b1a5138f9821fdfd00b0d54ecff/apple_news_to_sqlite-0.5.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-16 04:29:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "RhetTbull",
"github_project": "apple-news-to-sqlite",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "apple-news-to-sqlite"
}
JSON
