# User Guide
Cladetime is a wrapper around Nextstrain's GenBank-based SARS-CoV-2 genome
sequence data and the metadata that describes it. Included with the metadata
are the clades (variants) that each sequence is assigned to.
An advanced feature of Cladetime is the ability to perform custom clade
assignments using past reference trees. For example, you can use the
current set of sequence data and assign clades to it using the reference tree
as it existed three months ago.
Cladetime is designed for use with US-based sequences from Homo sapiens.
## Installation
Cladetime is written in Python and can be installed using pip:
```bash
pip install cladetime
```
## The CladeTime class
Most of Cladetime's features are accessible through the `CladeTime` class,
which accepts two optional parameters:
- `sequence_as_of`: access Nextstrain SARS-CoV-2 sequence data and metadata
files as they existing on this date (defaults to the current UTC datetime)
- `tree_as_of`: the date of the reference tree to use for clade assignments
(defaults to `sequence_as_of`)
> [!IMPORTANT]
> Using `tree_as_of` for custom clade assignments is an advanced feature
> and requires Docker.
```python
>>> from cladetime import CladeTime
# Create a CladeTime object that references the most recent available sequence
# data and metadata from Nextstrain
>>> ct = CladeTime()
```
## Accessing sequence data
Each `CladeTime` object has a link to the full set of Nextstrain's SARS-Cov-2
genomic sequences as they existed on the `sequence_as_of` date. This data
is in .fasta format, and most users won't need to download it directly.
```python
>>> from cladetime import CladeTime
>>> ct = CladeTime()
>>> ct.url_sequence
https://nextstrain-data.s3.amazonaws.com/files/ncov/open/sequences.fasta.xz?versionId=4Sv2PbA1NoEd.V_LOOQSBPkqBpdoj7s_'
```
More interesting to most users will be the [metadata that describes each
sequence](https://docs.nextstrain.org/projects/ncov/en/latest/reference/metadata-fields.html).
The `sequence_metadata` attribute of a `CladeTime` object is a Polars LazyFrame
that points to a copy of Nextstrain's sequence metadata.
You can apply your own filters and transformations to the LazyFrame, but
it's a good idea to start with the built-in `filter_metadata` function that
removes non-US and non-human sequences from the metadata.
A `collect()` operation will return the filtered metadata as an in-memory
Polars DataFrame.
```python
>>> import polars as pl
>>> from cladetime import CladeTime, sequence
>>> ct = CladeTime()
>>> filtered_metadata = sequence.filter_metadata(ct.sequence_metadata)
# Alternately, specify a sequence collection date range to the filter
>>> filtered_metadata = sequence.filter_metadata(
>>> ct.sequence_metadata,
>>> collection_min_date = "2024-10-01",collection_max_date ="2024-10-31"
>>> )
>>> metadata_df = filtered_metadata.collect(streaming=True)
# Pandas users can export Polars dataframes
>>> pandas_df = filtered_sequence_metadata.to_pandas()
```
## Past sequence data
Working with past sequence data and metadata is similar to the above examples.
Just pass in a `sequence_as_of` date when creating a `CladeTime` object.
The clades returned as part of the metadata will reflect the reference tree
in use when sequence metadata file was created.
```python
>>> from cladetime import CladeTime
# Create a CladeTime object for any date after May, 2023
>>> ct = CladeTime(sequence_as_of="2024-10-15")
```
## Custom clade assignments
You may want to assign sequence clades using a reference tree from a past date.
This feature is helpful when creating "source of truth" data to evaluate
models that predict clade proportions:
- create a `CladeTime` object using the `tree_as_of` parameter
- filter the sequence metadata to include only the sequences you want to assign
- pass the filtered metadata to the `assign_clades` method
CladeTime's `assign_clades` method returns two Polars LazyFrames:
- `detail`: a linefile of each sequence and its assigned clade
- `summary`: clade counts summarized by `country`, `location`, `date` and `host`
> [!WARNING]
> In addition to requiring Docker, assign_clades is resource-intensive,
> because the process requires downloading a full copy of SARS-CoV-2
> sequence data and then filtering it.
>
> The filtered sequences are then run through Nextclade's CLI for clade
> assignment, another resource-intensive process. We recommend not
> assigning more than 30 days worth of sequence collections at a time.
```python
>>> import polars as pl
>>> from cladetime import CladeTime, sequence
>>> ct = CladeTime(sequence_as_of="2024-11-15", tree_as_of="2024-09-01")
>>> filtered_metadata = sequence.filter_metadata(
>>> ct.sequence_metadata,
>>> collection_min_date = "2024-10-01",
>>> collection_max_date ="2024-10-31"
>>> )
>>> clade_assignments = ct.assign_clades(filtered_metadata)
# Summarized clade assignments
>>> clade_assignments.summary.collect().head()
shape: (5, 6)
┌──────────┬────────────┬──────────────┬──────────────────┬─────────┬───────┐
│ location ┆ date ┆ host ┆ clade_nextstrain ┆ country ┆ count │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ date ┆ str ┆ str ┆ str ┆ u32 │
╞══════════╪════════════╪══════════════╪══════════════════╪═════════╪═══════╡
│ IL ┆ 2024-10-28 ┆ Homo sapiens ┆ 24C ┆ USA ┆ 1 │
│ IL ┆ 2024-10-11 ┆ Homo sapiens ┆ 24C ┆ USA ┆ 5 │
│ NY ┆ 2024-10-08 ┆ Homo sapiens ┆ 24B ┆ USA ┆ 2 │
│ AZ ┆ 2024-10-15 ┆ Homo sapiens ┆ 24C ┆ USA ┆ 1 │
│ MN ┆ 2024-10-06 ┆ Homo sapiens ┆ 24A ┆ USA ┆ 2 │
└──────────┴────────────┴──────────────┴──────────────────┴─────────┴───────┘
# Detailed clade assignments
>>> clade_assignments.detail.collect().select(
>>> ["country", "location", "date", "strain", "clade_nextstrain"]
>>> ).head()
shape: (5, 5)
┌─────────┬──────────┬────────────┬─────────────────────┬──────────────────┐
│ country ┆ location ┆ date ┆ strain ┆ clade_nextstrain │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ str ┆ date ┆ str ┆ str │
╞═════════╪══════════╪════════════╪═════════════════════╪══════════════════╡
│ USA ┆ AZ ┆ 2024-10-01 ┆ USA/2024CV1711/2024 ┆ 24C │
│ USA ┆ AZ ┆ 2024-10-02 ┆ USA/2024CV1718/2024 ┆ 24C │
│ USA ┆ AZ ┆ 2024-10-04 ┆ USA/2024CV1719/2024 ┆ 24C │
│ USA ┆ AZ ┆ 2024-10-05 ┆ USA/2024CV1721/2024 ┆ 24C │
│ USA ┆ AZ ┆ 2024-10-06 ┆ USA/2024CV1722/2024 ┆ recombinant │
└─────────┴──────────┴────────────┴─────────────────────┴──────────────────┘
```
## Reproducibility
`CladeTime` objects have an `ncov_metadata` property with information needed to
reproduce the clade assignments in the object's sequence metadata.
In the example below, `ncov_metadata` shows that the
[Nextclade dataset](https://docs.nextstrain.org/projects/nextclade/en/stable/user/datasets.html)
used for clade assignment on 2024-09-22 was `2024-07-17--12-57-03Z`.
Each version of a SARS-CoV-2 Nextclade dataset contains a reference tree
that can be used as an input for clade assignments.
```python
>>> from cladetime import CladeTime
>>> ct = CladeTime(sequence_as_of='2024-09-22')
>>> ct.ncov_metadata.get('nextclade_dataset_name')
'SARS-CoV-2'
>>> ct.ncov_metadata.get('nextclade_dataset_version')
'2024-07-17--12-57-03Z'
```
Access to historical copies of `ncov_metadata` is what allows Cladetime to
access past reference trees for custom clade assignments. Cladetime retrieves
a separate set of `ncov_metadata` for the `tree_as_of` date and uses it to pass
the correct reference tree to the `assign_clades` method.
Raw data
{
"_id": null,
"home_page": null,
"name": "cladetime",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": "Becky Sweger <rsweger@umass.edu>, Evan Ray <elray@umass.edu>, Ben Rogers <bwrogers@umass.edu>",
"keywords": "biostatistics, clade, covid, epidemiology, genome, sequence",
"author": null,
"author_email": "\"Reich Lab @ University of Massachusetts\" <nick@umass.edu>",
"download_url": "https://files.pythonhosted.org/packages/5f/fb/34c2a105ed545032d66017d8a6b200d433336ced83d259c5729310281221/cladetime-0.3.0.tar.gz",
"platform": null,
"description": "\n\n# User Guide\n\nCladetime is a wrapper around Nextstrain's GenBank-based SARS-CoV-2 genome\nsequence data and the metadata that describes it. Included with the metadata\nare the clades (variants) that each sequence is assigned to.\n\nAn advanced feature of Cladetime is the ability to perform custom clade\nassignments using past reference trees. For example, you can use the\ncurrent set of sequence data and assign clades to it using the reference tree\nas it existed three months ago.\n\nCladetime is designed for use with US-based sequences from Homo sapiens.\n\n## Installation\n\nCladetime is written in Python and can be installed using pip:\n\n```bash\npip install cladetime\n```\n\n## The CladeTime class\n\nMost of Cladetime's features are accessible through the `CladeTime` class,\nwhich accepts two optional parameters:\n\n- `sequence_as_of`: access Nextstrain SARS-CoV-2 sequence data and metadata\nfiles as they existing on this date (defaults to the current UTC datetime)\n- `tree_as_of`: the date of the reference tree to use for clade assignments\n(defaults to `sequence_as_of`)\n\n> [!IMPORTANT]\n> Using `tree_as_of` for custom clade assignments is an advanced feature\n> and requires Docker.\n\n```python\n>>> from cladetime import CladeTime\n\n# Create a CladeTime object that references the most recent available sequence\n# data and metadata from Nextstrain\n>>> ct = CladeTime()\n```\n\n## Accessing sequence data\n\nEach `CladeTime` object has a link to the full set of Nextstrain's SARS-Cov-2\ngenomic sequences as they existed on the `sequence_as_of` date. This data\nis in .fasta format, and most users won't need to download it directly.\n\n```python\n>>> from cladetime import CladeTime\n>>> ct = CladeTime()\n>>> ct.url_sequence\nhttps://nextstrain-data.s3.amazonaws.com/files/ncov/open/sequences.fasta.xz?versionId=4Sv2PbA1NoEd.V_LOOQSBPkqBpdoj7s_'\n```\n\nMore interesting to most users will be the [metadata that describes each\nsequence](https://docs.nextstrain.org/projects/ncov/en/latest/reference/metadata-fields.html).\n\nThe `sequence_metadata` attribute of a `CladeTime` object is a Polars LazyFrame\nthat points to a copy of Nextstrain's sequence metadata.\n\nYou can apply your own filters and transformations to the LazyFrame, but\nit's a good idea to start with the built-in `filter_metadata` function that\nremoves non-US and non-human sequences from the metadata.\n\nA `collect()` operation will return the filtered metadata as an in-memory\nPolars DataFrame.\n\n```python\n>>> import polars as pl\n>>> from cladetime import CladeTime, sequence\n\n>>> ct = CladeTime()\n>>> filtered_metadata = sequence.filter_metadata(ct.sequence_metadata)\n\n# Alternately, specify a sequence collection date range to the filter\n>>> filtered_metadata = sequence.filter_metadata(\n>>> ct.sequence_metadata,\n>>> collection_min_date = \"2024-10-01\",collection_max_date =\"2024-10-31\"\n>>> )\n\n>>> metadata_df = filtered_metadata.collect(streaming=True)\n\n# Pandas users can export Polars dataframes\n>>> pandas_df = filtered_sequence_metadata.to_pandas()\n```\n\n## Past sequence data\n\nWorking with past sequence data and metadata is similar to the above examples.\nJust pass in a `sequence_as_of` date when creating a `CladeTime` object.\n\nThe clades returned as part of the metadata will reflect the reference tree\nin use when sequence metadata file was created.\n\n```python\n>>> from cladetime import CladeTime\n\n# Create a CladeTime object for any date after May, 2023\n>>> ct = CladeTime(sequence_as_of=\"2024-10-15\")\n```\n\n## Custom clade assignments\n\nYou may want to assign sequence clades using a reference tree from a past date.\nThis feature is helpful when creating \"source of truth\" data to evaluate\nmodels that predict clade proportions:\n\n- create a `CladeTime` object using the `tree_as_of` parameter\n- filter the sequence metadata to include only the sequences you want to assign\n- pass the filtered metadata to the `assign_clades` method\n\nCladeTime's `assign_clades` method returns two Polars LazyFrames:\n\n- `detail`: a linefile of each sequence and its assigned clade\n- `summary`: clade counts summarized by `country`, `location`, `date` and `host`\n\n> [!WARNING]\n> In addition to requiring Docker, assign_clades is resource-intensive,\n> because the process requires downloading a full copy of SARS-CoV-2\n> sequence data and then filtering it.\n>\n> The filtered sequences are then run through Nextclade's CLI for clade\n> assignment, another resource-intensive process. We recommend not\n> assigning more than 30 days worth of sequence collections at a time.\n\n```python\n>>> import polars as pl\n>>> from cladetime import CladeTime, sequence\n\n>>> ct = CladeTime(sequence_as_of=\"2024-11-15\", tree_as_of=\"2024-09-01\")\n>>> filtered_metadata = sequence.filter_metadata(\n>>> ct.sequence_metadata,\n>>> collection_min_date = \"2024-10-01\",\n>>> collection_max_date =\"2024-10-31\"\n>>> )\n>>> clade_assignments = ct.assign_clades(filtered_metadata)\n\n# Summarized clade assignments\n>>> clade_assignments.summary.collect().head()\nshape: (5, 6)\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 location \u2506 date \u2506 host \u2506 clade_nextstrain \u2506 country \u2506 count \u2502\n\u2502 --- \u2506 --- \u2506 --- \u2506 --- \u2506 --- \u2506 --- \u2502\n\u2502 str \u2506 date \u2506 str \u2506 str \u2506 str \u2506 u32 \u2502\n\u255e\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2561\n\u2502 IL \u2506 2024-10-28 \u2506 Homo sapiens \u2506 24C \u2506 USA \u2506 1 \u2502\n\u2502 IL \u2506 2024-10-11 \u2506 Homo sapiens \u2506 24C \u2506 USA \u2506 5 \u2502\n\u2502 NY \u2506 2024-10-08 \u2506 Homo sapiens \u2506 24B \u2506 USA \u2506 2 \u2502\n\u2502 AZ \u2506 2024-10-15 \u2506 Homo sapiens \u2506 24C \u2506 USA \u2506 1 \u2502\n\u2502 MN \u2506 2024-10-06 \u2506 Homo sapiens \u2506 24A \u2506 USA \u2506 2 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\n# Detailed clade assignments\n>>> clade_assignments.detail.collect().select(\n>>> [\"country\", \"location\", \"date\", \"strain\", \"clade_nextstrain\"]\n>>> ).head()\nshape: (5, 5)\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 country \u2506 location \u2506 date \u2506 strain \u2506 clade_nextstrain \u2502\n\u2502 --- \u2506 --- \u2506 --- \u2506 --- \u2506 --- \u2502\n\u2502 str \u2506 str \u2506 date \u2506 str \u2506 str \u2502\n\u255e\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2561\n\u2502 USA \u2506 AZ \u2506 2024-10-01 \u2506 USA/2024CV1711/2024 \u2506 24C \u2502\n\u2502 USA \u2506 AZ \u2506 2024-10-02 \u2506 USA/2024CV1718/2024 \u2506 24C \u2502\n\u2502 USA \u2506 AZ \u2506 2024-10-04 \u2506 USA/2024CV1719/2024 \u2506 24C \u2502\n\u2502 USA \u2506 AZ \u2506 2024-10-05 \u2506 USA/2024CV1721/2024 \u2506 24C \u2502\n\u2502 USA \u2506 AZ \u2506 2024-10-06 \u2506 USA/2024CV1722/2024 \u2506 recombinant \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n## Reproducibility\n\n`CladeTime` objects have an `ncov_metadata` property with information needed to\nreproduce the clade assignments in the object's sequence metadata.\n\nIn the example below, `ncov_metadata` shows that the\n[Nextclade dataset](https://docs.nextstrain.org/projects/nextclade/en/stable/user/datasets.html)\nused for clade assignment on 2024-09-22 was `2024-07-17--12-57-03Z`.\n\nEach version of a SARS-CoV-2 Nextclade dataset contains a reference tree\nthat can be used as an input for clade assignments.\n\n```python\n>>> from cladetime import CladeTime\n>>> ct = CladeTime(sequence_as_of='2024-09-22')\n\n>>> ct.ncov_metadata.get('nextclade_dataset_name')\n'SARS-CoV-2'\n>>> ct.ncov_metadata.get('nextclade_dataset_version')\n'2024-07-17--12-57-03Z'\n```\n\nAccess to historical copies of `ncov_metadata` is what allows Cladetime to\naccess past reference trees for custom clade assignments. Cladetime retrieves\na separate set of `ncov_metadata` for the `tree_as_of` date and uses it to pass\nthe correct reference tree to the `assign_clades` method.\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Assign clades to SARS-CoV-2 genome sequences at a point in time.",
"version": "0.3.0",
"project_urls": {
"Documentation": "https://cladetime.readthedocs.io/",
"Issues": "https://github.com/reichlab/cladetime/issues",
"Repository": "https://github.com/reichlab/cladetime.git"
},
"split_keywords": [
"biostatistics",
" clade",
" covid",
" epidemiology",
" genome",
" sequence"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "f4023b848bd300cc9bb242e916d46aece59bf1c29f32605c4eebc868b70d3a8c",
"md5": "b87553940ce1f57418a16174e65538cc",
"sha256": "8a7139c2add58e9dee7712da3e8eede59eeeaa9f98528bc1e62474d8c5e8ccab"
},
"downloads": -1,
"filename": "cladetime-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b87553940ce1f57418a16174e65538cc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 25644,
"upload_time": "2025-01-29T03:21:34",
"upload_time_iso_8601": "2025-01-29T03:21:34.025902Z",
"url": "https://files.pythonhosted.org/packages/f4/02/3b848bd300cc9bb242e916d46aece59bf1c29f32605c4eebc868b70d3a8c/cladetime-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "5ffb34c2a105ed545032d66017d8a6b200d433336ced83d259c5729310281221",
"md5": "76b31f431146022695672e1dfdcea570",
"sha256": "345864e53256506a925f2520638c2c1e0e46059627f510979cfba4537ee9ebbc"
},
"downloads": -1,
"filename": "cladetime-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "76b31f431146022695672e1dfdcea570",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 61200590,
"upload_time": "2025-01-29T03:21:39",
"upload_time_iso_8601": "2025-01-29T03:21:39.477340Z",
"url": "https://files.pythonhosted.org/packages/5f/fb/34c2a105ed545032d66017d8a6b200d433336ced83d259c5729310281221/cladetime-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-29 03:21:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "reichlab",
"github_project": "cladetime",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "cladetime"
}
JSON
