| Name | spil JSON |
| Version |
0.2.0
JSON |
| download |
| home_page | |
| Summary | The Simple Pipeline Lib. |
| upload_time | 2024-02-13 21:35:35 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | GNU LESSER GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. This version of the GNU Lesser General Public License incorporates the terms and conditions of version 3 of the GNU General Public License, supplemented by the additional permissions listed below. 0. Additional Definitions. As used herein, "this License" refers to version 3 of the GNU Lesser General Public License, and the "GNU GPL" refers to version 3 of the GNU General Public License. "The Library" refers to a covered work governed by this License, other than an Application or a Combined Work as defined below. An "Application" is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. Defining a subclass of a class defined by the Library is deemed a mode of using an interface provided by the Library. A "Combined Work" is a work produced by combining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the "Linked Version". The "Minimal Corresponding Source" for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version. The "Corresponding Application Code" for a Combined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Combined Work from the Application, but excluding the System Libraries of the Combined Work. 1. Exception to Section 3 of the GNU GPL. You may convey a covered work under sections 3 and 4 of this License without being bound by section 3 of the GNU GPL. 2. Conveying Modified Versions. If you modify a copy of the Library, and, in your modifications, a facility refers to a function or data to be supplied by an Application that uses the facility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modified version: a) under this License, provided that you make a good faith effort to ensure that, in the event an Application does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or b) under the GNU GPL, with none of the additional permissions of this License applicable to that copy. 3. Object Code Incorporating Material from Library Header Files. The object code form of an Application may incorporate material from a header file that is part of the Library. You may convey such object code under terms of your choice, provided that, if the incorporated material is not limited to numerical parameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the following: a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the object code with a copy of the GNU GPL and this license document. 4. Combined Works. You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following: a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the Combined Work with a copy of the GNU GPL and this license document. c) For a Combined Work that displays copyright notices during execution, include the copyright notice for the Library among these notices, as well as a reference directing the user to the copies of the GNU GPL and this license document. d) Do one of the following: 0) Convey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source. 1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Library already present on the user's computer system, and (b) will operate properly with a modified version of the Library that is interface-compatible with the Linked Version. e) Provide Installation Information, but only if you would otherwise be required to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modified version of the Combined Work produced by recombining or relinking the Application with a modified version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.) 5. Combined Libraries. You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following: a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License. b) Give prominent notice with the combined library that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work. 6. Revised Versions of the GNU Lesser General Public License. The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation. If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library. |
| keywords |
cg
cg-pipeline
path templates
vfx
vfx-pipeline
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Spil, the Simple Pipeline lib.
[](https://github.com/MichaelHaussmann/spil)
Spil provides a simple, hierarchical, path-like, unique identifier for every entity or file of a CG production pipeline.
An intuitive API is built around this identifier, including glob-like search, CRUD data access and path resolving.
## Motivation
Spil was created to:
- uniquely and intuitively identify all entities of a pipeline.
- aggregate different data sources (file systems, asset manager, DCCs, etc.)
- have a universal, versatile and lightweight "Entity" object for pipeline operations, at a higher abstraction level than a file path.
- propose an easy and intuitive API, to empower TDs and technical artists to connect to the pipeline.
## Usage
Full documentation: [spil.readthedocs.io](https://spil.readthedocs.io).
### Unique Hierarchical Identifier
The identifier is called the "**Sid**" - for "Scene Identifier".
Examples:
- Sid of sequence 30 shot 10 in the project "Hamlet":
```
"hamlet/s/sq030/sh0010"
```
- Sid for the maya mb file of character Ophelia's modeling, published version v002:
```
"hamlet/a/char/ophelia/modeling/v002/p/mb"
```
### Data Dictionary with a Type
Once resolved, the Sid is a dictionary associated to a type.
Keys and types are fully configurable.
Examples:
- type `shot__task` :
```
{
'project': 'hamlet',
'type': 's',
'sequence': 'sq030',
'shot': 'sh0010',
'task': 'animation'
}
```
- type `asset__version` :
```
{
'project': 'hamlet',
'type': 'a',
'cat': 'props',
'name': 'skull',
'task': 'modeling',
'version': 'v008'
}
```
### Intuitive API
Sid creation and manipulation
```python
from spil import Sid
# create a Sid from scratch
task = Sid("hamlet/s/sq030/sh0010/render")
# a task sid: hamlet/s/sq030/sh0010/render
# create a Sid by changing values
anim_task = task.get_with(task="anim")
# task sid: hamlet/s/sq030/sh0010/anim
# create a Sid from a Sids hierarchy
sequence = task.get_as('sequence')
# sequence sid: hamlet/s/sq030
# another way
shot = task.parent
# shot sid: hamlet/s/sq030/sh0010
```
Creation with a uri-query or dictionary
```python
from spil import Sid
seq = Sid(query="project=hamlet&type=s&sequence=sq010") # uri-query
seq = Sid(fields={'project': 'hamlet', 'type': 's', 'sequence': 'sq010'}) # dict
```
Data can be accessed in multiple ways: by key, as a complete dictionary, as string or uri-query.
```python
from spil import Sid
shot = Sid("hamlet/s/sq030/sh0010")
# get a field of the sid by key
shot.get("sequence")
# sq030
# as a dictionary
shot.fields
# { 'project': 'hamlet', 'type': 's', 'sequence': 'sq030', 'shot': 'sh0010' }
# as a Query
shot.as_query()
# "project=hamlet&type=s&seq=sq030&shot=sh0010"
# "uri": type and string
shot.uri
# "shot__shot:hamlet/s/sq030/sh0010"
```
### Path Resolver
The Sid can be resolved to and from paths.
Multiple configurations can co-exist.
For example "local", "server", "linux", etc. paths.
```python
from spil import Sid
# creating a Sid from path
scene = Sid(path="/projects/hamlet/chars/ophelia/modeling/v002/publish/ophelia_model.mb")
print(scene)
# "hamlet/a/chars/ophelia/modeling/v002/p/mb"
# returning default path
path = scene.path()
# "/projects/hamlet/chars/ophelia/modeling/v002/publish/ophelia_model.mb"
# returning path from "server" configuration
path = scene.path("server")
# "/server/projects/hamlet/chars/ophelia/modeling/v002/publish/ophelia_model.mb"
```
Example in maya, with an opened scene file:
```python
import maya.cmds as cmds
from spil import Sid
# Get the current scene's path
scene_path = cmds.file(query=True, sceneName=True)
# build the Sid
scene = Sid(path=scene_path)
if scene: # A sid that is not resolvable (not conform), has no type, and evaluates to False.
print(scene.get('project')) # hamlet
print(scene.get('version')) # "v002"
else:
print("opened scene is not a pipeline scene")
```
### Data access and Pipeline workflows
Sid wraps common requests, that are delegated to configurable data sources (Finder and Getter).
```python
from spil import Sid
task_sid = Sid("hamlet/s/sq030/sh0010/layout")
task_sid.exists() # True
task_sid.get_last('version') # "hamlet/s/sq030/sh0010/layout/v003"
```
Sids API can intuitively express common pipeline workflows.
```python
from spil import Sid
task_sid = Sid("hamlet/s/sq030/sh0010/layout")
if task_sid.exists():
print( task_sid.get_last('version').get_attr('comment') ) # "Changed camera angle."
# match() is handy for hooks and action overrides
if task_sid.match('hamlet/s/*/*/layout/**/maya'):
# do something specific for hamlet maya layouts
...
```
### Finding Sids: glob-like search syntax
Considering the Sid a middleware and data abstraction layer, Spil proposes an intuitive glob-like search syntax.
It is string based, and uses operators:
- \* : star search
- ** : recursive star search
- \> : last
- \, : "or"
- configurable aliases ("movie" -> "mov,avi,mp4", "maya" -> "ma,mb")
#### Search Examples
- "All the Shots of sequence sq030" ?
```
"hamlet/s/sq030/*"
```
- "All the published maya files of Ophelias modeling" ?
```
"hamlet/a/chars/ophelia/model/*/p/maya"
```
- "Last published render movies for the project hamlet" ?
```
"hamlet/s/**/render/>/p/movie"
```
- "All cache files for hamlet's sequence 30, shot 10" ?
```
"hamlet/s/sq030/sh010/**/cache"
```
#### Query in search
Query Syntax can be used to add search filters on yet untyped searches
- "All published Movie files for hamlet's sequence 30" ?
```
"hamlet/s/sq030/**/movie?state=p"
```
- "All published hip files for hamlet's sequence 30, animation or layout" ?
```
"hamlet/s/sq030/**/hip?state=p&task=animation,layout"
```
### Finders
To launch search queries, Spil implements `Finder` classes that access different data sources.
- **FindInPaths**: to search the file system
- **FindInList**: to search a list
- **FindInCache**: to search a cache
- **FindInAll**: to search other Finders, depending on a configuration
- **FindInShotgrid**: to search Shotgrid
All Finders implement `find()`, `find_one()` and `exists()`.
```python
# Look up the last published versions for given shot, on the File System
from spil import FindInPaths as Finder
for sid in Finder().find("hamlet/s/sq010/sh0010/**/>/p/movie"):
print(sid)
# "hamlet/s/sq010/sh0010/layout/v012/p/mov"
# "hamlet/s/sq010/sh0010/animation/v003/p/avi"
# "hamlet/s/sq010/sh0010/render/v001/p/mov"
# ...
```
The `FindInShotgrid` is an example Finder implementation.
It may need to be adapted, depending on the production's Shotgrid usage.
It is planned to implement other Finders, for example for MongoDB and CGWire kitsu.
## UI
Spil can be used with the spil_ui.browser.
[](https://github.com/MichaelHaussmann/spil_ui)
**Spil_UI** is a Qt browser UI, built on top of QtPy (PySide2/PySide6).
Navigating through the columns builds a **"Search Sid"** and calls a **Finder**.
It is possible to run actions on the currently selected Sid.
**spil_ui** is a separate repository (in the process of being released).
## REST API
Spil can run server side.
*(REST API and docker under development)*
## Flexible and configurable
Spil is a library, not a framework.<br>
It is fully configurable. It adopts your naming conventions, and does not enforce specific workflows.
It easily integrates and connects onto existing pipelines.
The Sid is based on the [resolva](https://github.com/MichaelHaussmann/resolva) resolver.
Sid config example:
```
'asset__file': '{project}/{type:a}/{assettype}/{asset}/{task}/{version}/{state}/{ext:scenes}',
'shot__file': '{project}/{type:s}/{sequence}/{shot}/{task}/{version}/{state}/{ext:scenes}',
'shot__movie_file': '{project}/{type:s}/{sequence}/{shot}/{task}/{version}/{state}/{ext:movies}',
```
Paths config example:
```
'asset__file': '{@project_root}/{project}/PROD/{type:ASSETS}/{assettype}/{asset}/{task}/{version}/{assettype}_{asset}_{task}_{state}_{version}.{ext:scenes}',
'shot__file': '{@project_root}/{project}/PROD/{type:SHOTS}/{sequence}/{sequence}_{shot}/{task}/{version}/{sequence}_{shot}_{task}_{state}_{version}.{ext:scenes}',
'shot__movie_file': '{@project_root}/{project}/PROD/{type:SHOTS}/{sequence}/{sequence}_{shot}/{task}/{version}/EXPORT/{sequence}_{shot}_{task}_{state}_{version}.{ext:movies}',
```
The Data Source is configurable depending on the given Sid or Sid type.
```
'project': finder_projects,
'shot_task': FindInShotgrid(),
'default': FindInPaths(),
```
## Installation
Spil works in Python >=3.7.
Spil is available on pypi and can be installed using `pip install spil`,
or from github `pip install git+https://github.com/MichaelHaussmann/spil.git`
More about installation, configuration and testing: [spil.readthedocs.io](https://spil.readthedocs.io).
## Performance
Spil thrives to be used interactively.
It's performance depends on the data sources that are used.
- Spil uses a `FindInConstants` class to handle configurable data that mostly doesn't change (types, asset types)
- Spil ships with a configurable `FindInCache` class to handle data that changes rarely (projects, sequences, assets, etc.).
(not production ready in current release)
- String / Sid Resolves are internally stored in a custom lru_cache
- `Finders` use generators
## Concepts
Spil builds upon general concepts, as well as production proven CG pipeline concepts.
### General concepts
- Unique Identifier - Human readable Identifier - "Natural Key"
[dzone.com/articles/7-strategies-for-assigning-ids-to-microservices](https://dzone.com/articles/7-strategies-for-assigning-ids-to-microservices)
[medium.com/blue-sky-tech-blog/a-rose-by-any-other-name-4b569309b575](https://medium.com/blue-sky-tech-blog/a-rose-by-any-other-name-4b569309b575)
- Python File system path
[www.python.org/dev/peps/pep-0428](https://www.python.org/dev/peps/pep-0428)
- Query by Example
A query technique where "example" entities, with search values, are used to retrieve "matching" results.
[en.wikipedia.org/wiki/Query_by_Example](https://en.wikipedia.org/wiki/Query_by_Example#As_a_general_technique)
- URI / URL
[en.wikipedia.org/wiki/Uniform_Resource_Identifier](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier)
- Node tree & hierarchy
### Pipeline concepts
- Unique Identifier & Resource Locator
Examples: "SPREF" (Sony Pictures), or the "Pipeline Resource Identifier - PRI" (Blue Sky)
[medium.com/blue-sky-tech-blog/conduit-pipeline-resource-identifiers](https://medium.com/blue-sky-tech-blog/conduit-pipeline-resource-identifiers-4432776da6ab)
Also OpenAssetIO's [Entity Reference](https://openassetio.github.io/OpenAssetIO/glossary.html#entity_reference)
- Resource description and "Context" (Shotgrid Toolkit)
[developer.shotgridsoftware.com/tk-core/core.html#context](https://developer.shotgridsoftware.com/tk-core/core.html#context)
- the "TypedContext", an entity for hierarchical types in Ftrack
- Template based path resolving
As implemented in Shotgrid Toolkit:
[github.com/shotgunsoftware/tk-config-default/blob/master/core/templates.yml](https://github.com/shotgunsoftware/tk-config-default/blob/master/core/templates.yml)
By CGWire's kitsu [zou.cg-wire.com/file_trees](https://zou.cg-wire.com/file_trees)
By [Lucidity](https://gitlab.com/4degrees/lucidity) or [resolva](https://github.com/MichaelHaussmann/resolva)
- Middleware between Asset consumers or producers
[OpenAssetIO](https://github.com/OpenAssetIO/OpenAssetIO)
[Katana Asset API](https://learn.foundry.com/katana/4.0/Content/tg/asset_management_system_plugin_api/concepts.html)
- Asset Resolution - ArResolver - in USD
[https://openusd.org/release/api/ar_page_front.html](https://openusd.org/release/api/ar_page_front.html)
- The Sid itself
The Sid has been used in general and fx pipelines since 2011, in various implementations and at various degrees.
## Philosophy
Spil aims to be : flexible, pragmatic, simple - and reliable.
- flexible
Spil is a library, and not a framework.
It can be plugged to existing pipelines. It easily blends in, to be used only where it is needed.
It can also be planned at a pipelines core - and be a central part of it.
<br>
- pragmatic
It all starts as files. So does Spil.
YAGNI meets WYSIWYG.
<br>
- simple
Complexity costs money, at all levels of a pipeline.
Spil aims at simplicity, even at the price of some universality or adaptability.
Usage is intuitive: it is obvious that `hamlet/a/char` is an asset category,
and `hamlet/a/char/ophelia/modeling` is a modeling task.
Producers have an overview, artists see clearly, TDs are empowered.
That is the goal of Spil.
<br>
- reliable
This part is yet to prove.
"In the face of ambiguity, refuse the temptation to guess."
But who are you to have read this far anyway?
## Limitations
- The configuration is tricky
For complex projects, creating the config is not simple, and is lacking tools to help.
Complex configurations may not work out of the box
- Beta stage
The core concepts have been around for a while, and different versions of the Sid are and have been used in production pipelines for some time now.
But this version of "Spil" is a rewrite. It is currently used in production, but is still young.
- Needs optimisation
The resolver is fast (using caches and memoization), but would benefit from a faster rust implementation.
File sequence support (eg. image sequences using fileseq) is still very slow.
## Plans and ongoing development
The priority is to make the current feature set more robust, efficient, and easy to deploy.
- tools to help create and verify the configuration files
- more testing and profiling
- rust implementation of [resolva](https://github.com/MichaelHaussmann/resolva)
To take profit from the Sids universality, we plan on building reusable open source bricks and pipeline tools.
For example:
- protocol for pipeline actions, for example `sid://play?hamlet/s/sq030/**/>/p/movie`
- connectors to Shotgrid, CGWire Kitsu, Ftrack and Databases
- using the sid as a USD Asset Resolver / In a USD pipeline
- GraphQL and/or rest API
- file system style navigation and context handling
For example `cd hamlet/s/sq010`
## Interested ?
We'd love to hear from you.
We are interested in any kind of feedback: comments, questions, issues, pull requests.
Spil is released under LGPL and is usable in closed source commercial applications.
Other licensing is possible, please get in touch.
Don't hesitate to contact us : [spil@xeo.info](mailto:spil@xeo.info).
We will be happy to respond.
<br>
Raw data
{
"_id": null,
"home_page": "",
"name": "spil",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "cg,cg-pipeline,path templates,vfx,vfx-pipeline",
"author": "",
"author_email": "Michael Haussmann <spil@xeo.info>",
"download_url": "https://files.pythonhosted.org/packages/fa/2b/8fdfc59e21ccd3ceb8bca34389fd3445f5a66b89d4ce933982e2730d865b/spil-0.2.0.tar.gz",
"platform": null,
"description": "# Spil, the Simple Pipeline lib.\n\n[](https://github.com/MichaelHaussmann/spil)\n\nSpil provides a simple, hierarchical, path-like, unique identifier for every entity or file of a CG production pipeline. \nAn intuitive API is built around this identifier, including glob-like search, CRUD data access and path resolving. \n\n## Motivation\n\nSpil was created to:\n- uniquely and intuitively identify all entities of a pipeline.\n- aggregate different data sources (file systems, asset manager, DCCs, etc.)\n- have a universal, versatile and lightweight \"Entity\" object for pipeline operations, at a higher abstraction level than a file path. \n- propose an easy and intuitive API, to empower TDs and technical artists to connect to the pipeline. \n\n\n## Usage\n\nFull documentation: [spil.readthedocs.io](https://spil.readthedocs.io).\n\n### Unique Hierarchical Identifier\n\nThe identifier is called the \"**Sid**\" - for \"Scene Identifier\".\n\nExamples: \n\n- Sid of sequence 30 shot 10 in the project \"Hamlet\":\n```\n\"hamlet/s/sq030/sh0010\"\n```\n\n- Sid for the maya mb file of character Ophelia's modeling, published version v002:\n```\n\"hamlet/a/char/ophelia/modeling/v002/p/mb\" \n```\n\n### Data Dictionary with a Type \n\nOnce resolved, the Sid is a dictionary associated to a type.\nKeys and types are fully configurable.\n\nExamples:\n\n- type `shot__task` :\n```\n{ \n 'project': 'hamlet', \n 'type': 's', \n 'sequence': 'sq030', \n 'shot': 'sh0010',\n 'task': 'animation' \n}\n```\n\n- type `asset__version` :\n```\n{ \n 'project': 'hamlet', \n 'type': 'a', \n 'cat': 'props', \n 'name': 'skull',\n 'task': 'modeling',\n 'version': 'v008' \n}\n```\n\n### Intuitive API\n\nSid creation and manipulation\n\n```python\nfrom spil import Sid\n\n# create a Sid from scratch\ntask = Sid(\"hamlet/s/sq030/sh0010/render\")\n# a task sid: hamlet/s/sq030/sh0010/render\n\n# create a Sid by changing values\nanim_task = task.get_with(task=\"anim\") \n# task sid: hamlet/s/sq030/sh0010/anim \n\n# create a Sid from a Sids hierarchy \nsequence = task.get_as('sequence') \n# sequence sid: hamlet/s/sq030\n\n# another way\nshot = task.parent \n# shot sid: hamlet/s/sq030/sh0010 \n```\n\nCreation with a uri-query or dictionary\n```python\nfrom spil import Sid\nseq = Sid(query=\"project=hamlet&type=s&sequence=sq010\") # uri-query \nseq = Sid(fields={'project': 'hamlet', 'type': 's', 'sequence': 'sq010'}) # dict\n```\n\nData can be accessed in multiple ways: by key, as a complete dictionary, as string or uri-query.\n```python\nfrom spil import Sid \nshot = Sid(\"hamlet/s/sq030/sh0010\")\n\n# get a field of the sid by key\nshot.get(\"sequence\") \n# sq030\n\n# as a dictionary\nshot.fields \n# { 'project': 'hamlet', 'type': 's', 'sequence': 'sq030', 'shot': 'sh0010' }\n\n# as a Query\nshot.as_query() \n# \"project=hamlet&type=s&seq=sq030&shot=sh0010\"\n\n# \"uri\": type and string\nshot.uri \n# \"shot__shot:hamlet/s/sq030/sh0010\" \n```\n\n### Path Resolver\n\nThe Sid can be resolved to and from paths. \nMultiple configurations can co-exist. \nFor example \"local\", \"server\", \"linux\", etc. paths. \n\n```python\nfrom spil import Sid\n\n# creating a Sid from path\nscene = Sid(path=\"/projects/hamlet/chars/ophelia/modeling/v002/publish/ophelia_model.mb\")\n\nprint(scene) \n# \"hamlet/a/chars/ophelia/modeling/v002/p/mb\"\n\n# returning default path\npath = scene.path() \n# \"/projects/hamlet/chars/ophelia/modeling/v002/publish/ophelia_model.mb\"\n\n# returning path from \"server\" configuration\npath = scene.path(\"server\") \n# \"/server/projects/hamlet/chars/ophelia/modeling/v002/publish/ophelia_model.mb\"\n```\n\nExample in maya, with an opened scene file:\n```python\nimport maya.cmds as cmds\nfrom spil import Sid\n\n# Get the current scene's path\nscene_path = cmds.file(query=True, sceneName=True)\n\n# build the Sid\nscene = Sid(path=scene_path)\n\nif scene: # A sid that is not resolvable (not conform), has no type, and evaluates to False.\n print(scene.get('project')) # hamlet\n print(scene.get('version')) # \"v002\"\nelse:\n print(\"opened scene is not a pipeline scene\")\n```\n\n### Data access and Pipeline workflows\n\nSid wraps common requests, that are delegated to configurable data sources (Finder and Getter).\n\n```python\nfrom spil import Sid\ntask_sid = Sid(\"hamlet/s/sq030/sh0010/layout\") \n\ntask_sid.exists() # True\ntask_sid.get_last('version') # \"hamlet/s/sq030/sh0010/layout/v003\"\n```\n\nSids API can intuitively express common pipeline workflows. \n```python\nfrom spil import Sid\ntask_sid = Sid(\"hamlet/s/sq030/sh0010/layout\") \n\nif task_sid.exists():\n print( task_sid.get_last('version').get_attr('comment') ) # \"Changed camera angle.\"\n\n# match() is handy for hooks and action overrides\nif task_sid.match('hamlet/s/*/*/layout/**/maya'): \n # do something specific for hamlet maya layouts\n ... \n```\n\n### Finding Sids: glob-like search syntax \n\nConsidering the Sid a middleware and data abstraction layer, Spil proposes an intuitive glob-like search syntax.\n \nIt is string based, and uses operators:\n- \\* : star search\n- ** : recursive star search\n- \\> : last\n- \\, : \"or\" \n- configurable aliases (\"movie\" -> \"mov,avi,mp4\", \"maya\" -> \"ma,mb\") \n \n#### Search Examples\n\n- \"All the Shots of sequence sq030\" ?\n```\n\"hamlet/s/sq030/*\"\n```\n\n- \"All the published maya files of Ophelias modeling\" ?\n```\n\"hamlet/a/chars/ophelia/model/*/p/maya\"\n```\n\n- \"Last published render movies for the project hamlet\" ?\n```\n\"hamlet/s/**/render/>/p/movie\"\n```\n\n- \"All cache files for hamlet's sequence 30, shot 10\" ?\n```\n\"hamlet/s/sq030/sh010/**/cache\"\n```\n\n#### Query in search \n\nQuery Syntax can be used to add search filters on yet untyped searches\n\n- \"All published Movie files for hamlet's sequence 30\" ?\n```\n\"hamlet/s/sq030/**/movie?state=p\"\n```\n\n- \"All published hip files for hamlet's sequence 30, animation or layout\" ?\n```\n\"hamlet/s/sq030/**/hip?state=p&task=animation,layout\"\n```\n\n \n### Finders\n\nTo launch search queries, Spil implements `Finder` classes that access different data sources. \n\n- **FindInPaths**: to search the file system\n- **FindInList**: to search a list\n- **FindInCache**: to search a cache\n- **FindInAll**: to search other Finders, depending on a configuration\n- **FindInShotgrid**: to search Shotgrid\n\nAll Finders implement `find()`, `find_one()` and `exists()`.\n\n```python\n# Look up the last published versions for given shot, on the File System \nfrom spil import FindInPaths as Finder\nfor sid in Finder().find(\"hamlet/s/sq010/sh0010/**/>/p/movie\"):\n print(sid)\n\n# \"hamlet/s/sq010/sh0010/layout/v012/p/mov\"\n# \"hamlet/s/sq010/sh0010/animation/v003/p/avi\"\n# \"hamlet/s/sq010/sh0010/render/v001/p/mov\"\n# ...\n```\n \nThe `FindInShotgrid` is an example Finder implementation. \nIt may need to be adapted, depending on the production's Shotgrid usage. \n\nIt is planned to implement other Finders, for example for MongoDB and CGWire kitsu. \n\n \n## UI\n \nSpil can be used with the spil_ui.browser. \n\n[](https://github.com/MichaelHaussmann/spil_ui)\n \n**Spil_UI** is a Qt browser UI, built on top of QtPy (PySide2/PySide6). \nNavigating through the columns builds a **\"Search Sid\"** and calls a **Finder**. \nIt is possible to run actions on the currently selected Sid. \n \n**spil_ui** is a separate repository (in the process of being released). \n\n## REST API\n\nSpil can run server side.\n\n\n\n*(REST API and docker under development)*\n\n## Flexible and configurable\n\nSpil is a library, not a framework.<br> \nIt is fully configurable. It adopts your naming conventions, and does not enforce specific workflows. \nIt easily integrates and connects onto existing pipelines. \n\nThe Sid is based on the [resolva](https://github.com/MichaelHaussmann/resolva) resolver. \n\nSid config example:\n```\n'asset__file': '{project}/{type:a}/{assettype}/{asset}/{task}/{version}/{state}/{ext:scenes}',\n'shot__file': '{project}/{type:s}/{sequence}/{shot}/{task}/{version}/{state}/{ext:scenes}',\n'shot__movie_file': '{project}/{type:s}/{sequence}/{shot}/{task}/{version}/{state}/{ext:movies}',\n```\n\nPaths config example:\n```\n'asset__file': '{@project_root}/{project}/PROD/{type:ASSETS}/{assettype}/{asset}/{task}/{version}/{assettype}_{asset}_{task}_{state}_{version}.{ext:scenes}',\n'shot__file': '{@project_root}/{project}/PROD/{type:SHOTS}/{sequence}/{sequence}_{shot}/{task}/{version}/{sequence}_{shot}_{task}_{state}_{version}.{ext:scenes}',\n'shot__movie_file': '{@project_root}/{project}/PROD/{type:SHOTS}/{sequence}/{sequence}_{shot}/{task}/{version}/EXPORT/{sequence}_{shot}_{task}_{state}_{version}.{ext:movies}', \n```\n\nThe Data Source is configurable depending on the given Sid or Sid type.\n```\n'project': finder_projects,\n'shot_task': FindInShotgrid(),\n'default': FindInPaths(),\n```\n\n## Installation\n\nSpil works in Python >=3.7. \nSpil is available on pypi and can be installed using `pip install spil`, \nor from github `pip install git+https://github.com/MichaelHaussmann/spil.git`\n\nMore about installation, configuration and testing: [spil.readthedocs.io](https://spil.readthedocs.io).\n\n\n## Performance\n\nSpil thrives to be used interactively. \nIt's performance depends on the data sources that are used.\n\n- Spil uses a `FindInConstants` class to handle configurable data that mostly doesn't change (types, asset types)\n- Spil ships with a configurable `FindInCache` class to handle data that changes rarely (projects, sequences, assets, etc.).\n (not production ready in current release)\n- String / Sid Resolves are internally stored in a custom lru_cache\n- `Finders` use generators\n\n## Concepts \n\nSpil builds upon general concepts, as well as production proven CG pipeline concepts. \n\n### General concepts\n\n- Unique Identifier - Human readable Identifier - \"Natural Key\" \n [dzone.com/articles/7-strategies-for-assigning-ids-to-microservices](https://dzone.com/articles/7-strategies-for-assigning-ids-to-microservices) \n [medium.com/blue-sky-tech-blog/a-rose-by-any-other-name-4b569309b575](https://medium.com/blue-sky-tech-blog/a-rose-by-any-other-name-4b569309b575)\n \n- Python File system path \n [www.python.org/dev/peps/pep-0428](https://www.python.org/dev/peps/pep-0428)\n \n- Query by Example \n A query technique where \"example\" entities, with search values, are used to retrieve \"matching\" results. \n [en.wikipedia.org/wiki/Query_by_Example](https://en.wikipedia.org/wiki/Query_by_Example#As_a_general_technique)\n \n- URI / URL \n [en.wikipedia.org/wiki/Uniform_Resource_Identifier](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier)\n \n- Node tree & hierarchy\n\n\n### Pipeline concepts\n\n- Unique Identifier & Resource Locator \n Examples: \"SPREF\" (Sony Pictures), or the \"Pipeline Resource Identifier - PRI\" (Blue Sky) \n [medium.com/blue-sky-tech-blog/conduit-pipeline-resource-identifiers](https://medium.com/blue-sky-tech-blog/conduit-pipeline-resource-identifiers-4432776da6ab) \n Also OpenAssetIO's [Entity Reference](https://openassetio.github.io/OpenAssetIO/glossary.html#entity_reference) \n \n- Resource description and \"Context\" (Shotgrid Toolkit) \n [developer.shotgridsoftware.com/tk-core/core.html#context](https://developer.shotgridsoftware.com/tk-core/core.html#context) \n \n- the \"TypedContext\", an entity for hierarchical types in Ftrack\n \n- Template based path resolving \n As implemented in Shotgrid Toolkit: \n [github.com/shotgunsoftware/tk-config-default/blob/master/core/templates.yml](https://github.com/shotgunsoftware/tk-config-default/blob/master/core/templates.yml) \n By CGWire's kitsu [zou.cg-wire.com/file_trees](https://zou.cg-wire.com/file_trees) \n By [Lucidity](https://gitlab.com/4degrees/lucidity) or [resolva](https://github.com/MichaelHaussmann/resolva)\n \n- Middleware between Asset consumers or producers \n [OpenAssetIO](https://github.com/OpenAssetIO/OpenAssetIO) \n [Katana Asset API](https://learn.foundry.com/katana/4.0/Content/tg/asset_management_system_plugin_api/concepts.html) \n\n- Asset Resolution - ArResolver - in USD \n [https://openusd.org/release/api/ar_page_front.html](https://openusd.org/release/api/ar_page_front.html) \n \n- The Sid itself \n The Sid has been used in general and fx pipelines since 2011, in various implementations and at various degrees. \n\n## Philosophy\n\nSpil aims to be : flexible, pragmatic, simple - and reliable. \n \n- flexible \n Spil is a library, and not a framework. \n It can be plugged to existing pipelines. It easily blends in, to be used only where it is needed. \n It can also be planned at a pipelines core - and be a central part of it. \n <br> \n- pragmatic \n It all starts as files. So does Spil. \n YAGNI meets WYSIWYG. \n <br> \n- simple \n Complexity costs money, at all levels of a pipeline. \n Spil aims at simplicity, even at the price of some universality or adaptability. \n Usage is intuitive: it is obvious that `hamlet/a/char` is an asset category, \n and `hamlet/a/char/ophelia/modeling` is a modeling task. \n Producers have an overview, artists see clearly, TDs are empowered. \n That is the goal of Spil. \n <br>\n- reliable \n This part is yet to prove. \n \"In the face of ambiguity, refuse the temptation to guess.\" \n But who are you to have read this far anyway? \n\n## Limitations\n\n- The configuration is tricky \n For complex projects, creating the config is not simple, and is lacking tools to help. \n Complex configurations may not work out of the box \n \n- Beta stage \n The core concepts have been around for a while, and different versions of the Sid are and have been used in production pipelines for some time now. \n But this version of \"Spil\" is a rewrite. It is currently used in production, but is still young.\n \n- Needs optimisation \n The resolver is fast (using caches and memoization), but would benefit from a faster rust implementation. \n File sequence support (eg. image sequences using fileseq) is still very slow. \n\n \n## Plans and ongoing development\n\nThe priority is to make the current feature set more robust, efficient, and easy to deploy.\n- tools to help create and verify the configuration files\n- more testing and profiling\n- rust implementation of [resolva](https://github.com/MichaelHaussmann/resolva)\n\nTo take profit from the Sids universality, we plan on building reusable open source bricks and pipeline tools.\n\nFor example:\n- protocol for pipeline actions, for example `sid://play?hamlet/s/sq030/**/>/p/movie`\n- connectors to Shotgrid, CGWire Kitsu, Ftrack and Databases\n- using the sid as a USD Asset Resolver / In a USD pipeline\n- GraphQL and/or rest API \n- file system style navigation and context handling \nFor example `cd hamlet/s/sq010`\n\n\n## Interested ?\n\nWe'd love to hear from you. \nWe are interested in any kind of feedback: comments, questions, issues, pull requests. \n\nSpil is released under LGPL and is usable in closed source commercial applications.\nOther licensing is possible, please get in touch.\n\nDon't hesitate to contact us : [spil@xeo.info](mailto:spil@xeo.info). \nWe will be happy to respond. \n<br>\n \n\n\n\n",
"bugtrack_url": null,
"license": "GNU LESSER GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. This version of the GNU Lesser General Public License incorporates the terms and conditions of version 3 of the GNU General Public License, supplemented by the additional permissions listed below. 0. Additional Definitions. As used herein, \"this License\" refers to version 3 of the GNU Lesser General Public License, and the \"GNU GPL\" refers to version 3 of the GNU General Public License. \"The Library\" refers to a covered work governed by this License, other than an Application or a Combined Work as defined below. An \"Application\" is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. Defining a subclass of a class defined by the Library is deemed a mode of using an interface provided by the Library. A \"Combined Work\" is a work produced by combining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the \"Linked Version\". The \"Minimal Corresponding Source\" for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version. The \"Corresponding Application Code\" for a Combined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Combined Work from the Application, but excluding the System Libraries of the Combined Work. 1. Exception to Section 3 of the GNU GPL. You may convey a covered work under sections 3 and 4 of this License without being bound by section 3 of the GNU GPL. 2. Conveying Modified Versions. If you modify a copy of the Library, and, in your modifications, a facility refers to a function or data to be supplied by an Application that uses the facility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modified version: a) under this License, provided that you make a good faith effort to ensure that, in the event an Application does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or b) under the GNU GPL, with none of the additional permissions of this License applicable to that copy. 3. Object Code Incorporating Material from Library Header Files. The object code form of an Application may incorporate material from a header file that is part of the Library. You may convey such object code under terms of your choice, provided that, if the incorporated material is not limited to numerical parameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the following: a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the object code with a copy of the GNU GPL and this license document. 4. Combined Works. You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following: a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the Combined Work with a copy of the GNU GPL and this license document. c) For a Combined Work that displays copyright notices during execution, include the copyright notice for the Library among these notices, as well as a reference directing the user to the copies of the GNU GPL and this license document. d) Do one of the following: 0) Convey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source. 1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Library already present on the user's computer system, and (b) will operate properly with a modified version of the Library that is interface-compatible with the Linked Version. e) Provide Installation Information, but only if you would otherwise be required to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modified version of the Combined Work produced by recombining or relinking the Application with a modified version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.) 5. Combined Libraries. You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following: a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License. b) Give prominent notice with the combined library that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work. 6. Revised Versions of the GNU Lesser General Public License. The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License \"or any later version\" applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation. If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library.",
"summary": "The Simple Pipeline Lib.",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://spil.readthedocs.io",
"Homepage": "https://github.com/MichaelHaussmann/spil"
},
"split_keywords": [
"cg",
"cg-pipeline",
"path templates",
"vfx",
"vfx-pipeline"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "53fe94ce443222d6e6403efad59c9dcb58a68c3b3966c3c1b2db0c9e558bbf59",
"md5": "9e2a56394450748222483e3021aefaf1",
"sha256": "8414388547388b64b817848daddc658dec4e5e8df4821c94303e57f37188b304"
},
"downloads": -1,
"filename": "spil-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9e2a56394450748222483e3021aefaf1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 147743,
"upload_time": "2024-02-13T21:35:33",
"upload_time_iso_8601": "2024-02-13T21:35:33.593759Z",
"url": "https://files.pythonhosted.org/packages/53/fe/94ce443222d6e6403efad59c9dcb58a68c3b3966c3c1b2db0c9e558bbf59/spil-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fa2b8fdfc59e21ccd3ceb8bca34389fd3445f5a66b89d4ce933982e2730d865b",
"md5": "6807218bb7a391457d72dd8862b0e5ea",
"sha256": "2495b6654e6e08da44e5db05598f76628e7d13b1ca5f51cd01b5e9d8816fb21b"
},
"downloads": -1,
"filename": "spil-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "6807218bb7a391457d72dd8862b0e5ea",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 87422,
"upload_time": "2024-02-13T21:35:35",
"upload_time_iso_8601": "2024-02-13T21:35:35.657208Z",
"url": "https://files.pythonhosted.org/packages/fa/2b/8fdfc59e21ccd3ceb8bca34389fd3445f5a66b89d4ce933982e2730d865b/spil-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-13 21:35:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MichaelHaussmann",
"github_project": "spil",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "spil"
}
