# doc-builder
This is the package we use to build the documentation of our Hugging Face repos.
## Table of Contents
- [doc-builder](#doc-builder)
* [Installation](#installation)
* [Previewing](#previewing)
* [Doc building](#doc-building)
* [Writing in notebooks](#writing-in-notebooks)
* [Templates for GitHub Actions](#templates-for-github-actions)
+ [Enabling multilingual documentation](#enabling-multilingual-documentation)
+ [Redirects](#redirects)
* [Fixing and testing doc-builder](#fixing-and-testing-doc-builder)
* [Writing documentation for Hugging Face libraries](#writing-documentation-for-hugging-face-libraries)
+ [Internal link to object](#internal-link-to-object)
+ [External link to object](#external-link-to-object)
+ [Tip](#tip)
+ [Framework Content](#framework-content)
+ [Options](#options)
+ [Anchor link](#anchor-link)
+ [LaTeX](#latex)
+ [Code Blocks](#code-blocks)
* [Writing API documentation (Python)](#writing-api-documentation-python)
+ [Autodoc](#autodoc)
+ [Code Blocks from file references](#code-blocks-from-file-references)
+ [Writing source documentation](#writing-source-documentation)
+ [Description](#description)
+ [Arguments](#arguments)
+ [Attributes](#attributes)
+ [Parmeter typing and default value](#parmeter-typing-and-default-value)
+ [Returns](#returns)
+ [Yields](#yields)
+ [Raises](#raises)
+ [Directives for Added, Changed, Deprecated](#directives-for-added-changed-deprecated)
* [Developing svelte locally](#developing-svelte-locally)
## Installation
You can install from PyPi with
```bash
pip install hf-doc-builder
```
To install from source, clone this repository then
```bash
cd doc-builder
pip install -e .
```
## Previewing
To preview the docs, use the following command:
```bash
doc-builder preview {package_name} {path_to_docs}
```
For example:
```bash
doc-builder preview datasets ~/Desktop/datasets/docs/source/
```
**`preview` command only works with existing doc files. When you add a completely new file, you need to update `_toctree.yml` & restart `preview` command (`ctrl-c` to stop it & call `doc-builder preview ...` again).
**`preview` command does not work with Windows.
## Doc building
To build the documentation of a given package, use the following command:
```bash
#Add --not_python_module if not building doc for a python lib
doc-builder build {package_name} {path_to_docs} --build_dir {build_dir}
```
For instance, here is how you can build the Datasets documentation (requires `pip install datasets[dev]`) if you have cloned the repo in `~/git/datasets`:
```bash
doc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build
```
This will generate MDX files that you can preview like any Markdown file in your favorite editor. To have a look at the documentation in HTML, you need to install node version 14 or higher. Then you can run (still with the example on Datasets)
```bash
doc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build --html
```
which will build HTML files in `~/tmp/test-build`. You can then inspect those files in your browser.
`doc-builder` can also automatically convert some of the documentation guides or tutorials into notebooks. This requires two steps:
- add `[[open-in-colab]]` in the tutorial for which you want to build a notebook
- add `--notebook_dir {path_to_notebook_folder}` to the build command.
## Writing in notebooks
You can write your docs in jupyter notebooks & use doc-builder to: turn jupyter notebooks into mdx files.
In some situations, such as course & tutorials, it makes more sense to write in jupyter notebooks (& use doc-builder converter) rather than writing in mdx files directly.
The process is:
1. In your `build_main_documentation.yml` & `build_pr_documentation.yml` enable the flag [convert_notebooks: true](https://github.com/huggingface/doc-builder/blob/main/.github/workflows/build_main_documentation.yml#L46-L48).
2. After this flag is enabled, doc-builder will convert all .ipynb files in [path_to_docs](https://github.com/huggingface/doc-builder/blob/main/.github/workflows/build_main_documentation.yml#L19-L20) to mdx files.
Moreover, you can locally convert .ipynb files into mdx files.
```bash
doc-builder notebook-to-mdx {path to notebook file or folder containing notebook files}
```
## Templates for GitHub Actions
`doc-builder` provides templates for GitHub Actions, so you can build your documentation with every pull request, push to some branch etc. To use them in your project, simply create the following three files in the `.github/workflows/` directory:
* `build_main_documentation.yml`: responsible for building the docs for the `main` branch, releases etc.
* `build_pr_documentation.yml`: responsible for building the docs on each PR.
* `upload_pr_documentation.yml`: responsible for uploading the PR artifacts to the Hugging Face Hub.
* `delete_doc_comment_trigger.yml`: responsible for removing the comments from the `HuggingFaceDocBuilder` bot that provides a URL to the PR docs.
Within each workflow, the main thing to include is a pointer from the `uses` field to the corresponding workflow in `doc-builder`. For example, this is what the PR workflow looks like in the `datasets` library:
```yaml
name: Build PR Documentation
on:
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
build:
uses: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml@main # Runs this doc-builder workflow
with:
commit_sha: ${{ github.event.pull_request.head.sha }}
pr_number: ${{ github.event.number }}
package: datasets # Replace this with your package name
```
Note the use of special arguments like `pr_number` and `package` under the `with` field. You can find the various options by inspecting each of the `doc-builder` [workflow files](https://github.com/huggingface/doc-builder/tree/main/.github/workflows).
### Enabling multilingual documentation
`doc-builder` can also convert documentation that's been translated from the English source into one or more languages. To enable the conversion, the documentation directories should be structured as follows:
```
doc_folder
├── en
│ ├── _toctree.yml
│ ├── _redirects.yml
│ ...
└── es
├── _toctree.yml
├── _redirects.yml
...
```
Note that each language directory has its own table of contents file `_toctree.yml` and that all languages are arranged under a single `doc_folder` directory - see the [`course`](https://github.com/huggingface/course/tree/main/chapters) repo for an example. You can then build the individual language subsets as follows:
```bash
doc-builder build {package_name} {path_to_docs} --build_dir {build_dir} --language {lang_id}
```
To automatically build the documentation for all languages via the GitHub Actions templates, simply provide the `languages` argument to your workflow, with a space-separated list of the languages you wish to build, e.g. `languages: en es`.
### Redirects
You can optionally provide `_redirects.yml` for "old links". The yml file should look like:
```yml
how_to: getting_started
package_reference/classes: package_reference/main_classes
# old_local: new_local
```
## Fixing and testing doc-builder
If you are working on a fix or an update of the doc-builder tool itself, you will eventually want to test it in the CI of another repository (transformers, diffusers, courses, etc.). To do so you should set the `doc_builder_revision` argument in your workflow file to point to your branch. Here is an example of what it would look like in the [`transformers.js` project](https://github.com/xenova/transformers.js/blob/main/.github/workflows/pr-documentation.yml):
```yml
jobs:
build:
uses: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml@my-test-branch
with:
repo_owner: xenova
commit_sha: ${{ github.sha }}
pr_number: ${{ github.event.number }}
package: transformers.js
path_to_docs: transformers.js/docs/source
pre_command: cd transformers.js && npm install && npm run docs-api
additional_args: --not_python_module
doc_builder_revision: my-test-branch # <- add this line
```
Once the docs build is complete in your project, you can drop that change.
## Writing documentation for Hugging Face libraries
`doc-builder` expects Markdown so you should write any new documentation in `".mdx"` files for tutorials, guides, API documentations. For docstrings, we follow the [Google format](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) with the main difference that you should use Markdown instead of restructured text (hopefully, that will be easier!)
Values that should be put in `code` should either be surrounded by backticks: \`like so\`. Note that argument names
and objects like True, None or any strings should usually be put in `code`.
Multi-line code blocks can be useful for displaying examples. They are done between two lines of three backticks as usual in Markdown:
````
```
# first line of code
# second line
# etc
```
````
We follow the [doctest](https://docs.python.org/3/library/doctest.html) syntax for the examples to automatically test
the results stay consistent with the library.
### Internal link to object
Syntax:
```html
[`XXXClass`] or [~`XXXClass`] // for class
[`XXXClass.method`] or [~`XXXClass.method`] // for method
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/sew-d.md?plain=1#L39) & [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/examples/research_projects/performer/modeling_flax_performer.py#L48) (as used inside docstring).
When mentioning a class, function or method, it is recommended to use the following syntax for internal links so that our tool
automatically adds a link to its documentation: \[\`XXXClass\`\] or \[\`function\`\]. This requires the class or
function to be in the main package.
If you want to create a link to some internal class or function, you need to
provide its path. For instance, in the Transformers documentation \[\`file_utils.ModelOutput\`\] will create a link to the documentation of `ModelOutput`. This link will have `file_utils.ModelOutput` in the description. To get rid of the path and only keep the name of the object you are
linking to in the description, add a ~: \[\`~file_utils.ModelOutput\`\] will generate a link with `ModelOutput` in the description.
The same works for methods, so you can either use \[\`XXXClass.method\`\] or \[~\`XXXClass.method\`\].
### External link to object
Syntax:
```html
[`XXXLibrary.XXXClass`] or [~`XXXLibrary.XXXClass`] // for class
[`XXXLibrary.XXXClass.method`] or [~`XXXLibrary.XXXClass.method`] // for method
```
Example: [here](https://github.com/huggingface/transformers/blob/0f0e1a2c2bff68541a5b9770d78e0fb6feb7de72/docs/source/en/accelerate.md?plain=1#L29) linking object from `accelerate` inside `transformers`.
### Tip
To write a block that you'd like to see highlighted as a note or warning, place your content between the following
markers.
Syntax:
```md
> [!TIP]
> Here is a tip. Go to this url [website](www.tip.com)
>
> Second line
```
or
```html
<Tip>
Write your note here
</Tip>
```
Example: [here](https://github.com/huggingface/transformers/blob/0f0e1a2c2bff68541a5b9770d78e0fb6feb7de72/docs/source/en/create_a_model.md#L282-L286)
For warnings, change the introduction to:
Syntax:
```md
> [!WARNING]
```
or
```html
`<Tip warning={true}>`
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/de/autoclass_tutorial.md#L102-L108)
### Framework Content
If your documentation has a block that is framework-dependent (PyTorch vs TensorFlow vs Flax), you can use the
following syntax:
Syntax:
```html
<frameworkcontent>
<pt>
PyTorch content goes here
</pt>
<tf>
TensorFlow content goes here
</tf>
<flax>
Flax content goes here
</flax>
</frameworkcontent>
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/de/autoclass_tutorial.md#L84-L131)
Note: all frameworks are optional (you can write a PyTorch-only block for instance) and the order does not matter.
### Options
Show alternatives (let's say code blocks for different version of a library etc.) in a way where a user can select an option and see the selected option content:
Syntax:
```html
<hfoptions id="some id">
<hfoption id="id for option 1">
{YOUR MARKDOWN}
</hfoption>
<hfoption id="id for option 2">
{YOUR MARKDOWN}
</hfoption>
... however many <hfoption> tags
</hfoptions>
```
Example: [here](https://github.com/huggingface/diffusers/blob/75ea54a1512ac443d517ab35cb9bf45f8d6f326e/docs/source/en/using-diffusers/kandinsky.md?plain=1#L30-L81)
Note: for multiple `<hfoptions id="some id">` in a same page, you may consider using same id so that when a user selects one option it affects all other hfoptions blocks. If you don't want this behaviour, use different ids.
### Anchor link
Anchor links for markdown headings are generated automatically (with the following rule: 1. lowercase, 2. replace space with dash `-`, 3. strip [^a-z0-9-]):
Syntax:
```
## My awesome section
// the anchor link is: `my-awesome-section`
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md#L132)
Moreover, there is a way to customize the anchor link.
Syntax:
```
## My awesome section[[some-section]]
// the anchor link is: `some-section`
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_summary.md#L79)
### LaTeX
Latex display mode. `$$...$$`
Syntax:
```
$$Y = X * \textbf{dequantize}(W); \text{quantize}(W)$$
```
Example: [here](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/rwkv.md?plain=1#L107)
Latex inline mode. `\\( ... )\\`
Syntax:
```
\\( Y = X * \textbf{dequantize}(W); \text{quantize}(W) )\\
```
Example: [here](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/rwkv.md?plain=1#L93)
### Code Blocks
Code blocks are written using a regular markdown syntax ```. However, there is a special flag you can put in your mdx files to change the wrapping style of the resulting html from overflow/scrollbar to wrap.
Syntax:
```
<!-- WRAP CODE BLOCKS -->
```
Example: [here](https://github.com/huggingface/text-generation-inference/blob/724199aaf172590c3658018c0e6bc6152cda4c2f/docs/source/basic_tutorials/launcher.md?plain=1#L3)
## Writing API documentation (Python)
### Autodoc
To show the full documentation of any object of the python library you are documenting, use the `[[autodoc]]` marker.
Syntax:
```
[[autodoc]] SomeObject
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md?plain=1#L142)
If the object is a class, this will include every public method of it that is documented. If for some reason you wish for a method
not to be displayed in the documentation, you can do so by specifying which methods should be in the docs, here is an example:
Syntax:
```
[[autodoc]] XXXTokenizer
- build_inputs_with_special_tokens
- get_special_tokens_mask
- create_token_type_ids_from_sequences
- save_vocabulary
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md?plain=1#L158-L159)
If you just want to add a method that is not documented (for instance magic method like `__call__` are not documented
by default) you can put the list of methods to add in a list that contains `all`:
Syntax:
```
## XXXTokenizer
[[autodoc]] XXXTokenizer
- all
- __call__
```
Example: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md?plain=1#L258-L259)
### Code Blocks from file references
You can create a code-block by referencing a file excerpt with `<literalinclude>` (sphinx-inspired) syntax.
There should be json between `<literalinclude>` open & close tags.
Syntax:
```
<literalinclude>
{"path": "./data/convert_literalinclude_dummy.txt", # relative path
"language": "python", # defaults to " (empty str)
"start-after": "START python_import", # defaults to start of file
"end-before": "END python_import", # defaults to end of file
"dedent": 7 # defaults to 0
}
</literalinclude>
```
### Writing source documentation
### Description
For a class or function description string, use markdown with [all the custom syntax of doc-builder](#writing-documentation-for-hugging-face-libraries).
Example: [here](https://github.com/huggingface/transformers/blob/910faa3e1f1c566b23a0318f78f5caf5bda8d3b2/examples/flax/language-modeling/run_t5_mlm_flax.py#L257-L267)
### Arguments
Arguments of a function/class/method should be defined with the `Args:` (or `Arguments:` or `Parameters:`) prefix, followed by a line return and
an indentation. The argument should be followed by its type, with its shape if it is a tensor, a colon, and its
description:
Syntax:
```
Args:
n_layers (`int`): The number of layers of the model.
```
Example: [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/src/transformers/models/bert/tokenization_bert_fast.py#L168-L198)
If the description is too long to fit in one line, another indentation is necessary before writing the description
after the argument.
Syntax:
```
Args:
input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
Indices of input sequence tokens in the vocabulary.
Indices can be obtained using [`AlbertTokenizer`]. See [`~PreTrainedTokenizer.encode`] and
[`~PreTrainedTokenizer.__call__`] for details.
[What are input IDs?](../glossary#input-ids)
```
Example: [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/src/transformers/models/bert/tokenization_bert_fast.py#L173-L175)
You can check the full example it comes from [here](https://github.com/huggingface/transformers/blob/v4.17.0/src/transformers/models/bert/modeling_bert.py#L794-L841)
### Attributes
If a class is similar to that of a dataclass but the parameters do not align to the available attributes of the class, such as in the below example, `Attributes` instance should be rewritten as `**Attributes**` in order to have the documentation properly render these. Otherwise it will assume that `Attributes` is synonymous to `Parameters`.
Syntax:
```diff
class SomeClass:
"""
Docstring
- Attributes:
+ **Attributes**:
- **attr_a** (`type_a`) -- Doc a
- **attr_b** (`type_b`) -- Doc b
"""
def __init__(self, param_a, param_b):
...
```
### Parmeter typing and default value
For optional arguments or arguments with defaults we follow the following syntax. Imagine we have a function with the
following signature:
```
def my_function(x: str = None, a: float = 1):
```
then its documentation should look like this:
Syntax:
```
Args:
x (`str`, *optional*):
This argument controls ...
a (`float`, *optional*, defaults to 1):
This argument is used to ...
```
Example: [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/src/transformers/models/bert/tokenization_bert_fast.py#L176)
Note that we always omit the "defaults to \`None\`" when None is the default for any argument. Also note that even
if the first line describing your argument type and its default gets long, you can't break it on several lines. You can
however write as many lines as you want in the indented description (see the example above with `input_ids`).
If your argument has for type a class defined in the package, you can use the syntax we saw earlier to link to its
documentation:
```
Args:
config ([`BertConfig`]):
Model configuration class with all the parameters of the model.
Initializing with a config file does not load the weights associated with the model, only the
configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights.
```
### Returns
The return block should be introduced with the `Returns:` prefix, followed by a line return and an indentation.
The first line should be the type of the return, followed by a line return. No need to indent further for the elements
building the return.
Here's an example for a single value return:
Syntax:
```
Returns:
`List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.
```
Example: [here](https://github.com/huggingface/transformers/blob/910faa3e1f1c566b23a0318f78f5caf5bda8d3b2/examples/flax/language-modeling/run_t5_mlm_flax.py#L273-L275)
Here's an example for tuple return, comprising several objects:
Syntax:
```
Returns:
`tuple(torch.FloatTensor)` comprising various elements depending on the configuration ([`BertConfig`]) and inputs:
- ** loss** (*optional*, returned when `masked_lm_labels` is provided) `torch.FloatTensor` of shape `(1,)` --
Total loss as the sum of the masked language modeling loss and the next sequence prediction (classification) loss.
- **prediction_scores** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) --
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).
```
Example: [here](https://github.com/huggingface/transformers/blob/003a0cf8cc4d78e47ef9debfb1e93a5c1197ca9a/examples/research_projects/bert-loses-patience/pabee/modeling_pabee_albert.py#L107-L130)
### Yields
Similarly, `Yields` is also supported.
Syntax:
```
Yields:
`tuple[str, io.BufferedReader]`:
2-tuple (path_within_archive, file_object).
File object is opened in binary mode.
```
Example: [here](https://github.com/huggingface/datasets/blob/f56fd9d6c877ffa6fb44fb832c13b61227c9cc5b/src/datasets/download/download_manager.py#L459-L462C17)
### Raises
You can also document `Raises`.
Syntax:
```
Args:
config ([`BertConfig`]):
Model configuration class with all the parameters of the model.
Initializing with a config file does not load the weights associated with the model, only the
configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights.
Raises:
`pa.ArrowInvalidError`: if the arrow data casting fails
TypeError: if the target type is not supported according, e.g.
- point1
- point2
[`HTTPError`](https://2.python-requests.org/en/master/api/#requests.HTTPError) if credentials are invalid
[`HTTPError`](https://2.python-requests.org/en/master/api/#requests.HTTPError) if connection got lost
Returns:
`List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.
```
Example: [here](https://github.com/huggingface/transformers/blob/1b2381c46b834a89e447f7a01f0961c4e940d117/src/transformers/models/mask2former/image_processing_mask2former.py#L167-L168)
### Directives for Added, Changed, Deprecated
There are directives for `Added`, `Changed`, & `Deprecated`.
Syntax:
```
Args:
cache_dir (`str`, *optional*): Directory to cache data.
config_name (`str`, *optional*): Name of the dataset configuration.
It affects the data generated on disk: different configurations will have their own subdirectories and
versions.
If not provided, the default configuration is used (if it exists).
<Added version="2.3.0">
`name` was renamed to `config_name`.
</Added>
name (`str`): Configuration name for the dataset.
<Deprecated version="2.3.0">
Use `config_name` instead.
</Deprecated>
```
Example: [here](https://github.com/huggingface/datasets/blob/a1e1867e932f14233244fb25713f3c94c46ff50a/src/datasets/combine.py#L53)
## Developing svelte locally
We use svelte components for doc UI ([Tip component](https://github.com/huggingface/doc-builder/blob/890df105f4173fb8dc299ad6ba3e4db378d2e53d/kit/src/lib/Tip.svelte), [Docstring component](https://github.com/huggingface/doc-builder/blob/a9598feb5a681a3817e58ef8d792349e85a30d1e/kit/src/lib/Docstring.svelte), etc.).
Follow these steps to develop svelte locally:
1. Create this file if it doesn't already exist: `doc-builder/kit/src/routes/_toctree.yml`. Contents should be:
```
- sections:
- local: index
title: Index page
title: Index page
```
2. Create this file if it doesn't already exist: `doc-builder/kit/src/routes/index.mdx`. Contents should be whatever you'd like to test. For example:
```
<script lang="ts">
import Tip from "$lib/Tip.svelte";
import Youtube from "$lib/Youtube.svelte";
import Docstring from "$lib/Docstring.svelte";
import CodeBlock from "$lib/CodeBlock.svelte";
import CodeBlockFw from "$lib/CodeBlockFw.svelte";
</script>
<Tip>
[Here](https://myurl.com)
</Tip>
## Some heading
And some text [Here](https://myurl.com)
Physics is the natural science that studies matter,[a] its fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force.[2] Physics is one of the most fundamental scientific disciplines, with its main goal being to understand how the universe behaves.[b][3][4][5] A scientist who specializes in the field of physics is called a physicist.
```
3. Install dependencies & run dev mode
```bash
cd doc-builder/kit
npm ci
npm run dev -- --open
```
4. Start developing. See svelte files in `doc-builder/kit/src/lib` for reference. The flow should be:
1. Create a svelte component in `doc-builder/kit/src/lib`
2. Import it & test it in `doc-builder/kit/src/routes/index.mdx`
Raw data
{
"_id": null,
"home_page": "https://github.com/huggingface/doc-builder",
"name": "hf-doc-builder",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "doc documentation doc-builder huggingface hugging face",
"author": "Hugging Face, Inc.",
"author_email": "docs@huggingface.co",
"download_url": "https://files.pythonhosted.org/packages/97/f7/3a4e5528b79891159cd428d6c6d09bd47ca6b4c26315b9bb77828b162586/hf-doc-builder-0.5.0.tar.gz",
"platform": null,
"description": "# doc-builder\n\nThis is the package we use to build the documentation of our Hugging Face repos.\n\n## Table of Contents\n\n- [doc-builder](#doc-builder)\n * [Installation](#installation)\n * [Previewing](#previewing)\n * [Doc building](#doc-building)\n * [Writing in notebooks](#writing-in-notebooks)\n * [Templates for GitHub Actions](#templates-for-github-actions)\n + [Enabling multilingual documentation](#enabling-multilingual-documentation)\n + [Redirects](#redirects)\n * [Fixing and testing doc-builder](#fixing-and-testing-doc-builder)\n * [Writing documentation for Hugging Face libraries](#writing-documentation-for-hugging-face-libraries)\n + [Internal link to object](#internal-link-to-object)\n + [External link to object](#external-link-to-object)\n + [Tip](#tip)\n + [Framework Content](#framework-content)\n + [Options](#options)\n + [Anchor link](#anchor-link)\n + [LaTeX](#latex)\n + [Code Blocks](#code-blocks)\n * [Writing API documentation (Python)](#writing-api-documentation-python)\n + [Autodoc](#autodoc)\n + [Code Blocks from file references](#code-blocks-from-file-references)\n + [Writing source documentation](#writing-source-documentation)\n + [Description](#description)\n + [Arguments](#arguments)\n + [Attributes](#attributes)\n + [Parmeter typing and default value](#parmeter-typing-and-default-value)\n + [Returns](#returns)\n + [Yields](#yields)\n + [Raises](#raises)\n + [Directives for Added, Changed, Deprecated](#directives-for-added-changed-deprecated)\n * [Developing svelte locally](#developing-svelte-locally)\n\n## Installation\n\nYou can install from PyPi with\n\n```bash\npip install hf-doc-builder\n```\n\nTo install from source, clone this repository then\n\n```bash\ncd doc-builder\npip install -e .\n```\n\n## Previewing\n\nTo preview the docs, use the following command:\n\n```bash\ndoc-builder preview {package_name} {path_to_docs}\n```\n\nFor example:\n```bash\ndoc-builder preview datasets ~/Desktop/datasets/docs/source/\n```\n\n**`preview` command only works with existing doc files. When you add a completely new file, you need to update `_toctree.yml` & restart `preview` command (`ctrl-c` to stop it & call `doc-builder preview ...` again).\n\n**`preview` command does not work with Windows.\n## Doc building\n\nTo build the documentation of a given package, use the following command:\n\n```bash\n#Add --not_python_module if not building doc for a python lib\ndoc-builder build {package_name} {path_to_docs} --build_dir {build_dir}\n```\n\nFor instance, here is how you can build the Datasets documentation (requires `pip install datasets[dev]`) if you have cloned the repo in `~/git/datasets`:\n\n```bash\ndoc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build\n```\n\nThis will generate MDX files that you can preview like any Markdown file in your favorite editor. To have a look at the documentation in HTML, you need to install node version 14 or higher. Then you can run (still with the example on Datasets)\n\n```bash\ndoc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build --html\n```\nwhich will build HTML files in `~/tmp/test-build`. You can then inspect those files in your browser.\n\n`doc-builder` can also automatically convert some of the documentation guides or tutorials into notebooks. This requires two steps:\n- add `[[open-in-colab]]` in the tutorial for which you want to build a notebook\n- add `--notebook_dir {path_to_notebook_folder}` to the build command.\n\n## Writing in notebooks\n\nYou can write your docs in jupyter notebooks & use doc-builder to: turn jupyter notebooks into mdx files.\n\nIn some situations, such as course & tutorials, it makes more sense to write in jupyter notebooks (& use doc-builder converter) rather than writing in mdx files directly.\n\nThe process is:\n1. In your `build_main_documentation.yml` & `build_pr_documentation.yml` enable the flag [convert_notebooks: true](https://github.com/huggingface/doc-builder/blob/main/.github/workflows/build_main_documentation.yml#L46-L48). \n2. After this flag is enabled, doc-builder will convert all .ipynb files in [path_to_docs](https://github.com/huggingface/doc-builder/blob/main/.github/workflows/build_main_documentation.yml#L19-L20) to mdx files.\n\nMoreover, you can locally convert .ipynb files into mdx files.\n```bash\ndoc-builder notebook-to-mdx {path to notebook file or folder containing notebook files}\n```\n\n## Templates for GitHub Actions\n\n`doc-builder` provides templates for GitHub Actions, so you can build your documentation with every pull request, push to some branch etc. To use them in your project, simply create the following three files in the `.github/workflows/` directory:\n\n* `build_main_documentation.yml`: responsible for building the docs for the `main` branch, releases etc.\n* `build_pr_documentation.yml`: responsible for building the docs on each PR.\n* `upload_pr_documentation.yml`: responsible for uploading the PR artifacts to the Hugging Face Hub.\n* `delete_doc_comment_trigger.yml`: responsible for removing the comments from the `HuggingFaceDocBuilder` bot that provides a URL to the PR docs.\n\nWithin each workflow, the main thing to include is a pointer from the `uses` field to the corresponding workflow in `doc-builder`. For example, this is what the PR workflow looks like in the `datasets` library:\n\n```yaml\nname: Build PR Documentation\n\non:\n pull_request:\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}\n cancel-in-progress: true\n\njobs:\n build:\n uses: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml@main # Runs this doc-builder workflow\n with:\n commit_sha: ${{ github.event.pull_request.head.sha }}\n pr_number: ${{ github.event.number }}\n package: datasets # Replace this with your package name\n```\n\nNote the use of special arguments like `pr_number` and `package` under the `with` field. You can find the various options by inspecting each of the `doc-builder` [workflow files](https://github.com/huggingface/doc-builder/tree/main/.github/workflows).\n\n### Enabling multilingual documentation\n\n`doc-builder` can also convert documentation that's been translated from the English source into one or more languages. To enable the conversion, the documentation directories should be structured as follows:\n\n```\ndoc_folder\n\u251c\u2500\u2500 en\n\u2502 \u251c\u2500\u2500 _toctree.yml\n\u2502 \u251c\u2500\u2500 _redirects.yml\n\u2502 ...\n\u2514\u2500\u2500 es\n \u251c\u2500\u2500 _toctree.yml\n \u251c\u2500\u2500 _redirects.yml\n ...\n```\n\nNote that each language directory has its own table of contents file `_toctree.yml` and that all languages are arranged under a single `doc_folder` directory - see the [`course`](https://github.com/huggingface/course/tree/main/chapters) repo for an example. You can then build the individual language subsets as follows:\n\n```bash\ndoc-builder build {package_name} {path_to_docs} --build_dir {build_dir} --language {lang_id}\n```\n\nTo automatically build the documentation for all languages via the GitHub Actions templates, simply provide the `languages` argument to your workflow, with a space-separated list of the languages you wish to build, e.g. `languages: en es`.\n\n### Redirects\n\nYou can optionally provide `_redirects.yml` for \"old links\". The yml file should look like:\n\n```yml\nhow_to: getting_started\npackage_reference/classes: package_reference/main_classes\n# old_local: new_local\n```\n\n## Fixing and testing doc-builder\n\nIf you are working on a fix or an update of the doc-builder tool itself, you will eventually want to test it in the CI of another repository (transformers, diffusers, courses, etc.). To do so you should set the `doc_builder_revision` argument in your workflow file to point to your branch. Here is an example of what it would look like in the [`transformers.js` project](https://github.com/xenova/transformers.js/blob/main/.github/workflows/pr-documentation.yml):\n\n```yml\njobs:\n build:\n uses: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml@my-test-branch\n with:\n repo_owner: xenova\n commit_sha: ${{ github.sha }}\n pr_number: ${{ github.event.number }}\n package: transformers.js\n path_to_docs: transformers.js/docs/source\n pre_command: cd transformers.js && npm install && npm run docs-api\n additional_args: --not_python_module\n doc_builder_revision: my-test-branch # <- add this line\n```\n\nOnce the docs build is complete in your project, you can drop that change.\n\n## Writing documentation for Hugging Face libraries\n\n`doc-builder` expects Markdown so you should write any new documentation in `\".mdx\"` files for tutorials, guides, API documentations. For docstrings, we follow the [Google format](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) with the main difference that you should use Markdown instead of restructured text (hopefully, that will be easier!)\n\nValues that should be put in `code` should either be surrounded by backticks: \\`like so\\`. Note that argument names\nand objects like True, None or any strings should usually be put in `code`.\n\nMulti-line code blocks can be useful for displaying examples. They are done between two lines of three backticks as usual in Markdown:\n\n````\n```\n# first line of code\n# second line\n# etc\n```\n````\n\nWe follow the [doctest](https://docs.python.org/3/library/doctest.html) syntax for the examples to automatically test\nthe results stay consistent with the library.\n\n### Internal link to object\n\nSyntax:\n```html\n[`XXXClass`] or [~`XXXClass`] // for class\n[`XXXClass.method`] or [~`XXXClass.method`] // for method\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/sew-d.md?plain=1#L39) & [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/examples/research_projects/performer/modeling_flax_performer.py#L48) (as used inside docstring).\n\nWhen mentioning a class, function or method, it is recommended to use the following syntax for internal links so that our tool\nautomatically adds a link to its documentation: \\[\\`XXXClass\\`\\] or \\[\\`function\\`\\]. This requires the class or \nfunction to be in the main package.\n\nIf you want to create a link to some internal class or function, you need to\nprovide its path. For instance, in the Transformers documentation \\[\\`file_utils.ModelOutput\\`\\] will create a link to the documentation of `ModelOutput`. This link will have `file_utils.ModelOutput` in the description. To get rid of the path and only keep the name of the object you are\nlinking to in the description, add a ~: \\[\\`~file_utils.ModelOutput\\`\\] will generate a link with `ModelOutput` in the description.\n\nThe same works for methods, so you can either use \\[\\`XXXClass.method\\`\\] or \\[~\\`XXXClass.method\\`\\].\n\n### External link to object\n\nSyntax:\n```html\n[`XXXLibrary.XXXClass`] or [~`XXXLibrary.XXXClass`] // for class\n[`XXXLibrary.XXXClass.method`] or [~`XXXLibrary.XXXClass.method`] // for method\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/0f0e1a2c2bff68541a5b9770d78e0fb6feb7de72/docs/source/en/accelerate.md?plain=1#L29) linking object from `accelerate` inside `transformers`.\n\n### Tip\n\nTo write a block that you'd like to see highlighted as a note or warning, place your content between the following\nmarkers.\n\nSyntax:\n\n```md\n> [!TIP]\n> Here is a tip. Go to this url [website](www.tip.com)\n> \n> Second line\n```\n\nor\n\n```html\n<Tip>\n\nWrite your note here\n\n</Tip>\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/0f0e1a2c2bff68541a5b9770d78e0fb6feb7de72/docs/source/en/create_a_model.md#L282-L286)\n\nFor warnings, change the introduction to:\n\nSyntax:\n\n```md\n> [!WARNING]\n```\n\nor\n\n```html\n`<Tip warning={true}>`\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/de/autoclass_tutorial.md#L102-L108)\n\n### Framework Content\n\nIf your documentation has a block that is framework-dependent (PyTorch vs TensorFlow vs Flax), you can use the\nfollowing syntax:\n\nSyntax:\n\n```html\n<frameworkcontent>\n<pt>\nPyTorch content goes here\n</pt>\n<tf>\nTensorFlow content goes here\n</tf>\n<flax>\nFlax content goes here\n</flax>\n</frameworkcontent>\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/de/autoclass_tutorial.md#L84-L131)\n\nNote: all frameworks are optional (you can write a PyTorch-only block for instance) and the order does not matter.\n\n### Options\n\nShow alternatives (let's say code blocks for different version of a library etc.) in a way where a user can select an option and see the selected option content:\n\nSyntax:\n\n```html\n<hfoptions id=\"some id\">\n<hfoption id=\"id for option 1\">\n{YOUR MARKDOWN}\n</hfoption>\n<hfoption id=\"id for option 2\">\n{YOUR MARKDOWN}\n</hfoption>\n... however many <hfoption> tags\n</hfoptions>\n```\n\nExample: [here](https://github.com/huggingface/diffusers/blob/75ea54a1512ac443d517ab35cb9bf45f8d6f326e/docs/source/en/using-diffusers/kandinsky.md?plain=1#L30-L81)\n\nNote: for multiple `<hfoptions id=\"some id\">` in a same page, you may consider using same id so that when a user selects one option it affects all other hfoptions blocks. If you don't want this behaviour, use different ids.\n\n### Anchor link\n\nAnchor links for markdown headings are generated automatically (with the following rule: 1. lowercase, 2. replace space with dash `-`, 3. strip [^a-z0-9-]):\n\nSyntax:\n```\n## My awesome section\n// the anchor link is: `my-awesome-section`\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md#L132)\n\nMoreover, there is a way to customize the anchor link.\n\nSyntax:\n```\n## My awesome section[[some-section]]\n// the anchor link is: `some-section`\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_summary.md#L79)\n\n### LaTeX\n\nLatex display mode. `$$...$$`\n\nSyntax:\n\n```\n$$Y = X * \\textbf{dequantize}(W); \\text{quantize}(W)$$\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/rwkv.md?plain=1#L107)\n\nLatex inline mode. `\\\\( ... )\\\\`\n\nSyntax:\n\n```\n\\\\( Y = X * \\textbf{dequantize}(W); \\text{quantize}(W) )\\\\\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/rwkv.md?plain=1#L93)\n\n### Code Blocks\n\nCode blocks are written using a regular markdown syntax ```. However, there is a special flag you can put in your mdx files to change the wrapping style of the resulting html from overflow/scrollbar to wrap.\n\nSyntax:\n```\n<!-- WRAP CODE BLOCKS -->\n```\n\nExample: [here](https://github.com/huggingface/text-generation-inference/blob/724199aaf172590c3658018c0e6bc6152cda4c2f/docs/source/basic_tutorials/launcher.md?plain=1#L3)\n\n## Writing API documentation (Python)\n\n### Autodoc\n\nTo show the full documentation of any object of the python library you are documenting, use the `[[autodoc]]` marker.\n\nSyntax:\n\n```\n[[autodoc]] SomeObject\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md?plain=1#L142)\n\nIf the object is a class, this will include every public method of it that is documented. If for some reason you wish for a method\nnot to be displayed in the documentation, you can do so by specifying which methods should be in the docs, here is an example:\n\nSyntax:\n\n```\n[[autodoc]] XXXTokenizer\n - build_inputs_with_special_tokens\n - get_special_tokens_mask\n - create_token_type_ids_from_sequences\n - save_vocabulary\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md?plain=1#L158-L159)\n\nIf you just want to add a method that is not documented (for instance magic method like `__call__` are not documented\nby default) you can put the list of methods to add in a list that contains `all`:\n\nSyntax:\n\n```\n## XXXTokenizer\n\n[[autodoc]] XXXTokenizer\n - all\n - __call__\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/eb849f6604c7dcc0e96d68f4851e52e253b9f0e5/docs/source/en/model_doc/bert.md?plain=1#L258-L259)\n\n### Code Blocks from file references\n\nYou can create a code-block by referencing a file excerpt with `<literalinclude>` (sphinx-inspired) syntax. \nThere should be json between `<literalinclude>` open & close tags.\n\nSyntax:\n\n```\n<literalinclude>\n{\"path\": \"./data/convert_literalinclude_dummy.txt\", # relative path\n\"language\": \"python\", # defaults to \" (empty str)\n\"start-after\": \"START python_import\", # defaults to start of file\n\"end-before\": \"END python_import\", # defaults to end of file\n\"dedent\": 7 # defaults to 0\n}\n</literalinclude>\n```\n\n### Writing source documentation\n\n### Description\n\nFor a class or function description string, use markdown with [all the custom syntax of doc-builder](#writing-documentation-for-hugging-face-libraries).\n\nExample: [here](https://github.com/huggingface/transformers/blob/910faa3e1f1c566b23a0318f78f5caf5bda8d3b2/examples/flax/language-modeling/run_t5_mlm_flax.py#L257-L267)\n\n### Arguments\n\nArguments of a function/class/method should be defined with the `Args:` (or `Arguments:` or `Parameters:`) prefix, followed by a line return and\nan indentation. The argument should be followed by its type, with its shape if it is a tensor, a colon, and its\ndescription:\n\nSyntax:\n\n```\n Args:\n n_layers (`int`): The number of layers of the model.\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/src/transformers/models/bert/tokenization_bert_fast.py#L168-L198)\n\nIf the description is too long to fit in one line, another indentation is necessary before writing the description\nafter the argument.\n\nSyntax:\n\n```\n Args:\n input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):\n Indices of input sequence tokens in the vocabulary.\n\n Indices can be obtained using [`AlbertTokenizer`]. See [`~PreTrainedTokenizer.encode`] and\n [`~PreTrainedTokenizer.__call__`] for details.\n\n [What are input IDs?](../glossary#input-ids)\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/src/transformers/models/bert/tokenization_bert_fast.py#L173-L175)\n\nYou can check the full example it comes from [here](https://github.com/huggingface/transformers/blob/v4.17.0/src/transformers/models/bert/modeling_bert.py#L794-L841)\n\n### Attributes\n\nIf a class is similar to that of a dataclass but the parameters do not align to the available attributes of the class, such as in the below example, `Attributes` instance should be rewritten as `**Attributes**` in order to have the documentation properly render these. Otherwise it will assume that `Attributes` is synonymous to `Parameters`.\n\nSyntax:\n\n```diff\n class SomeClass:\n \"\"\"\n Docstring\n- Attributes:\n+ **Attributes**:\n - **attr_a** (`type_a`) -- Doc a\n - **attr_b** (`type_b`) -- Doc b\n \"\"\"\n def __init__(self, param_a, param_b):\n ...\n```\n\n### Parmeter typing and default value\n\nFor optional arguments or arguments with defaults we follow the following syntax. Imagine we have a function with the\nfollowing signature:\n\n```\ndef my_function(x: str = None, a: float = 1):\n```\n\nthen its documentation should look like this:\n\nSyntax: \n\n```\n Args:\n x (`str`, *optional*):\n This argument controls ...\n a (`float`, *optional*, defaults to 1):\n This argument is used to ...\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/src/transformers/models/bert/tokenization_bert_fast.py#L176)\n\nNote that we always omit the \"defaults to \\`None\\`\" when None is the default for any argument. Also note that even\nif the first line describing your argument type and its default gets long, you can't break it on several lines. You can\nhowever write as many lines as you want in the indented description (see the example above with `input_ids`).\n\nIf your argument has for type a class defined in the package, you can use the syntax we saw earlier to link to its\ndocumentation:\n\n```\n Args:\n config ([`BertConfig`]):\n Model configuration class with all the parameters of the model.\n\n Initializing with a config file does not load the weights associated with the model, only the\n configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights.\n```\n\n### Returns\n\nThe return block should be introduced with the `Returns:` prefix, followed by a line return and an indentation.\nThe first line should be the type of the return, followed by a line return. No need to indent further for the elements\nbuilding the return.\n\nHere's an example for a single value return:\n\nSyntax:\n\n```\n Returns:\n `List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/910faa3e1f1c566b23a0318f78f5caf5bda8d3b2/examples/flax/language-modeling/run_t5_mlm_flax.py#L273-L275)\n\nHere's an example for tuple return, comprising several objects:\n\nSyntax:\n\n```\n Returns:\n `tuple(torch.FloatTensor)` comprising various elements depending on the configuration ([`BertConfig`]) and inputs:\n - ** loss** (*optional*, returned when `masked_lm_labels` is provided) `torch.FloatTensor` of shape `(1,)` --\n Total loss as the sum of the masked language modeling loss and the next sequence prediction (classification) loss.\n - **prediction_scores** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) --\n Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/003a0cf8cc4d78e47ef9debfb1e93a5c1197ca9a/examples/research_projects/bert-loses-patience/pabee/modeling_pabee_albert.py#L107-L130)\n\n### Yields\n\nSimilarly, `Yields` is also supported.\n\nSyntax:\n\n```\nYields:\n `tuple[str, io.BufferedReader]`:\n 2-tuple (path_within_archive, file_object).\n File object is opened in binary mode.\n```\n\nExample: [here](https://github.com/huggingface/datasets/blob/f56fd9d6c877ffa6fb44fb832c13b61227c9cc5b/src/datasets/download/download_manager.py#L459-L462C17)\n\n### Raises\n\nYou can also document `Raises`.\n\nSyntax:\n```\n Args:\n config ([`BertConfig`]):\n Model configuration class with all the parameters of the model.\n\n Initializing with a config file does not load the weights associated with the model, only the\n configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights.\n\n Raises:\n `pa.ArrowInvalidError`: if the arrow data casting fails\n TypeError: if the target type is not supported according, e.g.\n - point1\n - point2\n [`HTTPError`](https://2.python-requests.org/en/master/api/#requests.HTTPError) if credentials are invalid\n [`HTTPError`](https://2.python-requests.org/en/master/api/#requests.HTTPError) if connection got lost\n\n Returns:\n `List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.\n```\n\nExample: [here](https://github.com/huggingface/transformers/blob/1b2381c46b834a89e447f7a01f0961c4e940d117/src/transformers/models/mask2former/image_processing_mask2former.py#L167-L168)\n\n### Directives for Added, Changed, Deprecated\n\nThere are directives for `Added`, `Changed`, & `Deprecated`.\nSyntax:\n```\n Args:\n cache_dir (`str`, *optional*): Directory to cache data.\n config_name (`str`, *optional*): Name of the dataset configuration.\n It affects the data generated on disk: different configurations will have their own subdirectories and\n versions.\n If not provided, the default configuration is used (if it exists).\n\n <Added version=\"2.3.0\">\n\n `name` was renamed to `config_name`.\n\n </Added>\n name (`str`): Configuration name for the dataset.\n\n <Deprecated version=\"2.3.0\">\n\n Use `config_name` instead.\n\n </Deprecated>\n```\n\nExample: [here](https://github.com/huggingface/datasets/blob/a1e1867e932f14233244fb25713f3c94c46ff50a/src/datasets/combine.py#L53)\n\n## Developing svelte locally\n\nWe use svelte components for doc UI ([Tip component](https://github.com/huggingface/doc-builder/blob/890df105f4173fb8dc299ad6ba3e4db378d2e53d/kit/src/lib/Tip.svelte), [Docstring component](https://github.com/huggingface/doc-builder/blob/a9598feb5a681a3817e58ef8d792349e85a30d1e/kit/src/lib/Docstring.svelte), etc.).\n\nFollow these steps to develop svelte locally:\n1. Create this file if it doesn't already exist: `doc-builder/kit/src/routes/_toctree.yml`. Contents should be:\n```\n- sections: \n - local: index\n title: Index page\n title: Index page\n```\n2. Create this file if it doesn't already exist: `doc-builder/kit/src/routes/index.mdx`. Contents should be whatever you'd like to test. For example:\n```\n<script lang=\"ts\">\nimport Tip from \"$lib/Tip.svelte\";\nimport Youtube from \"$lib/Youtube.svelte\";\nimport Docstring from \"$lib/Docstring.svelte\";\nimport CodeBlock from \"$lib/CodeBlock.svelte\";\nimport CodeBlockFw from \"$lib/CodeBlockFw.svelte\";\n</script>\n\n<Tip>\n\n [Here](https://myurl.com)\n\n</Tip>\n\n## Some heading\nAnd some text [Here](https://myurl.com)\n\nPhysics is the natural science that studies matter,[a] its fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force.[2] Physics is one of the most fundamental scientific disciplines, with its main goal being to understand how the universe behaves.[b][3][4][5] A scientist who specializes in the field of physics is called a physicist.\n```\n3. Install dependencies & run dev mode\n```bash\ncd doc-builder/kit\nnpm ci\nnpm run dev -- --open\n```\n4. Start developing. See svelte files in `doc-builder/kit/src/lib` for reference. The flow should be:\n 1. Create a svelte component in `doc-builder/kit/src/lib`\n 2. Import it & test it in `doc-builder/kit/src/routes/index.mdx`\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "Doc building utility",
"version": "0.5.0",
"project_urls": {
"Homepage": "https://github.com/huggingface/doc-builder"
},
"split_keywords": [
"doc",
"documentation",
"doc-builder",
"huggingface",
"hugging",
"face"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4089b794a9f2708f2926d3050132541952bc44eeeb1e560dc34fd00bace47655",
"md5": "10c74814809dd615215a615d5dcb06d7",
"sha256": "bdceb44a26b7eb90a344afb3560306e23e95f0676f94fbd8a8111cf7f096e2ea"
},
"downloads": -1,
"filename": "hf_doc_builder-0.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "10c74814809dd615215a615d5dcb06d7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 67780,
"upload_time": "2024-03-11T13:40:10",
"upload_time_iso_8601": "2024-03-11T13:40:10.107302Z",
"url": "https://files.pythonhosted.org/packages/40/89/b794a9f2708f2926d3050132541952bc44eeeb1e560dc34fd00bace47655/hf_doc_builder-0.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "97f73a4e5528b79891159cd428d6c6d09bd47ca6b4c26315b9bb77828b162586",
"md5": "326cc01a61ab9e41d73e7ae7163b14ef",
"sha256": "e557660f76d1d90ac79e96d7b17642eca83f16a0a89ceabde180f0437d28179b"
},
"downloads": -1,
"filename": "hf-doc-builder-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "326cc01a61ab9e41d73e7ae7163b14ef",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 152415,
"upload_time": "2024-03-11T13:40:11",
"upload_time_iso_8601": "2024-03-11T13:40:11.597853Z",
"url": "https://files.pythonhosted.org/packages/97/f7/3a4e5528b79891159cd428d6c6d09bd47ca6b4c26315b9bb77828b162586/hf-doc-builder-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-11 13:40:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "huggingface",
"github_project": "doc-builder",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "hf-doc-builder"
}
JSON
