| Name | typed-D3DShot JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | None |
| Summary | Extremely Fast, Robust, and Typed, Screen Capture on Windows with the Desktop Duplication API |
| upload_time | 2024-10-09 20:43:29 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT License Copyright (c) 2019 Serpent.AI / Nicholas Brochu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
computer vision
d3d
desktop duplication
direct3d
screen capture
screenshot
windows
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# typed-D3DShot (D3DShot but statically typed!)
This fork of the [now archived D3DShot](https://github.com/SerpentAI/D3DShot) aims to add static type-checking support by adding type annotations and making certain classes generic. With [typed-D3DShot](https://pypi.org/project/typed-D3DShot/), you don't need to install Typeshed's defunct [types-D3DShot](https://pypi.org/project/types-D3DShot/). `typed-D3DShot`'s own code is type-checked using [mypy](/mypy.ini) and [pyright](/pyrightconfig.json).
Note that I do not intend on maintaining this library outside typing support and what I need for [AutoSplit](https://github.com/Toufool/AutoSplit).
---
_D3DShot_ is a pure Python implementation of the [Windows Desktop Duplication API](https://docs.microsoft.com/en-us/windows/desktop/direct3ddxgi/desktop-dup-api). It leverages DXGI and Direct3D system libraries to enable extremely fast and robust screen capture functionality for your Python scripts and applications on Windows.
**D3DShot:**
* Is by far the fastest way to capture the screen with Python on Windows 8.1+
* Is very easy to use. If you can remember 10-ish methods, you know the entire thing.
* Covers all common scenarios and use cases:
* Screenshot to memory
* Screenshot to disk
* Screenshot to memory buffer every X seconds (threaded; non-blocking)
* Screenshot to disk every X seconds (threaded; non-blocking)
* High-speed capture to memory buffer (threaded; non-blocking)
* Output options to capture to PIL Images, NumPy or PyTorch.
* Detects displays in just about any configuration: Single monitor, multiple monitors on one adapter, multiple monitors on multiple adapters.
* Handles display rotation and scaling for you
* Supports capturing specific regions of the screen
* Is robust and very stable. You can run it for hours / days without performance degradation
* Is even able to capture DirectX 11 / 12 exclusive fullscreen applications and games!
### TL;DR Quick Code Samples
**Screenshot to Memory**
```python
import d3dshot
d = d3dshot.create()
d.screenshot()
```
```python
Out[1]: <PIL.Image.Image image mode=RGB size=2560x1440 at 0x1AA7ECB5C88>
```
**Screenshot to Disk**
```python
import d3dshot
d = d3dshot.create()
d.screenshot_to_disk()
```
```python
Out[1]: './1554298682.5632973.png'
```
**Screen Capture for 5 Seconds and Grab the Latest Frame**
```python
import d3dshot
import time
d = d3dshot.create()
d.capture()
time.sleep(5) # Capture is non-blocking so we wait explicitly
d.stop()
d.get_latest_frame()
```
```python
Out[1]: <PIL.Image.Image image mode=RGB size=2560x1440 at 0x1AA044BCF60>
```
**Screen Capture the Second Monitor as NumPy Arrays for 3 Seconds and Grab the 4 Latest Frames as a Stack**
```python
import d3dshot
import time
d = d3dshot.create(capture_output="numpy")
d.display = d.displays[1]
d.capture()
time.sleep(3) # Capture is non-blocking so we wait explicitly
d.stop()
frame_stack = d.get_frame_stack((0, 1, 2, 3), stack_dimension="last")
frame_stack.shape
```
```python
Out[1]: (1080, 1920, 3, 4)
```
This is barely scratching the surface... Keep reading!
## Requirements
* Windows 8.1+ (64-bit)
* Python 3.8+ (64-bit)
## Installation
_D3DShot_ leverages DLLs that are already available on your system so the dependencies are very light. Namely:
* [_comtypes_](https://github.com/enthought/comtypes): Internal use. To preserve developer sanity while working with COM interfaces.
These dependencies will automatically be installed alongside _D3DShot_; No need to worry about them!
Some more dependencies are required depending on your preferred capture output:
* [_Pillow_](https://github.com/python-pillow/Pillow): Default Capture Output. Also used to save to disk as PNG and JPG.
* [_NumPy_](https://github.com/numpy/numpy): Works extremely fast with multidimensional arrays, which images are.
* [_PyTorch_](https://github.com/numpy/numpy): Offers Tensor computation with strong GPU acceleration.
Optional dependencies (also known as "extras") are provided to make it easy to install all needed dependencies.
```shell
# For the NumPy capture output
pip install d3dshot[numpy]
# For the Pillow capture output
pip install d3dshot[PIL]
# For the PyTorch capture output
pip install d3dshot[torch]
# You can mix if needed
pip install d3dshot[numpy][PIL]
```
##### Extra Step: Laptop Users
Windows has a quirk when using Desktop Duplication on hybrid-GPU systems. Please see the [wiki article](https://github.com/SerpentAI/D3DShot/wiki/Installation-Note:-Laptops) before attempting to use _D3DShot_ on your system.
## Concepts
### Capture Outputs
The desired _Capture Output_ is defined when creating a _D3DShot_ instance. It defines the type of all captured images. By default, all captures will return PIL.Image objects. This is a good option if you mostly intend to take screenshots.
```python
# Captures will be PIL.Image in RGB mode
d = d3dshot.create()
d = d3dshot.create(capture_output="pil")
```
_D3DShot_ is however quite flexible! As your environment meets certain optional sets of requirements, more options become available.
**If _NumPy_ is available**
```python
# Captures will be np.ndarray of dtype uint8 with values in range (0, 255)
d = d3dshot.create(capture_output="numpy")
# Captures will be np.ndarray of dtype float64 with normalized values in range (0.0, 1.0)
d = d3dshot.create(capture_output="numpy_float")
```
**If _NumPy_ and _PyTorch_ are available**
```python
# Captures will be torch.Tensor of dtype uint8 with values in range (0, 255)
d = d3dshot.create(capture_output="pytorch")
# Captures will be torch.Tensor of dtype float64 with normalized values in range (0.0, 1.0)
d = d3dshot.create(capture_output="pytorch_float")
```
**If _NumPy_ and _PyTorch_ are available + _CUDA_ is installed and _torch.cuda.is_available()_**
```python
# Captures will be torch.Tensor of dtype uint8 with values in range (0, 255) on device cuda:0
d = d3dshot.create(capture_output="pytorch_gpu")
# Captures will be torch.Tensor of dtype float64 with normalized values in range (0.0, 1.0) on device cuda:0
d = d3dshot.create(capture_output="pytorch_float_gpu")
```
Trying to use a Capture Output for which your environment does not meet the requirements will result in an error.
### Singleton
Windows only allows 1 instance of Desktop Duplication per process. To make sure we fall in line with that limitation to avoid issues, the _D3DShot_ class acts as a singleton. Any subsequent calls to `d3dshot.create()` will always return the existing instance.
```python
d = d3dshot.create(capture_output="numpy")
# Attempting to create a second instance
d2 = d3dshot.create(capture_output="pil")
# Only 1 instance of D3DShot is allowed per process! Returning the existing instance...
# Capture output remains 'numpy'
d2.capture_output.backend
# Out[1]: <d3dshot.capture_outputs.numpy_capture_output.NumpyCaptureOutput at 0x2672be3b8e0>
d == d2
# Out[2]: True
```
### Frame Buffer
When you create a _D3DShot_ instance, a frame buffer is also initialized. It is meant as a thread-safe, first-in, first-out way to hold a certain quantity of captures and is implemented as a `collections.deque`.
By default, the size of the frame buffer is set to 60. You can customize it when creating your _D3DShot_ object.
```python
d = d3dshot.create(frame_buffer_size=100)
```
Be mindful of RAM usage with larger values; You will be dealing with uncompressed images which use up to 100 MB each depending on the resolution.
The frame buffer can be accessed directly with `d.frame_buffer` but the usage of the utility methods instead is recommended.
The buffer is used by the following methods:
* `d.capture()`
* `d.screenshot_every()`
It is always automatically cleared before starting one of these operations.
### Displays
When you create a _D3DShot_ instance, your available displays will automatically be detected along with all their relevant properties.
```python
d.displays
```
```python
Out[1]:
[<Display name=BenQ XL2730Z (DisplayPort) adapter=NVIDIA GeForce GTX 1080 Ti resolution=2560x1440 rotation=0 scale_factor=1.0 primary=True>,
<Display name=BenQ XL2430T (HDMI) adapter=Intel(R) UHD Graphics 630 resolution=1920x1080 rotation=0 scale_factor=1.0 primary=False>]
```
By default, your primary display will be selected. At all times you can verify which display is set to be used for capture.
```python
d.display
```
```python
Out[1]: <Display name=BenQ XL2730Z (DisplayPort) adapter=NVIDIA GeForce GTX 1080 Ti resolution=2560x1440 rotation=0 scale_factor=1.0 primary=True>
```
Selecting another display for capture is as simple as setting `d.display` to another value from `d.displays`
```python
d.display = d.displays[1]
d.display
```
```python
Out[1]: <Display name=BenQ XL2430T (HDMI) adapter=Intel(R) UHD Graphics 630 resolution=1080x1920 rotation=90 scale_factor=1.0 primary=False>
```
Display rotation and scaling is detected and handled for you by _D3DShot_:
* Captures on rotated displays will always be in the correct orientation (i.e. matching what you see on your physical displays)
* Captures on scaled displays will always be in full, non-scaled resolution (e.g. 1280x720 at 200% scaling will yield 2560x1440 captures)
### Regions
All capture methods (screenshots included) accept an optional `region` kwarg. The expected value is a 4-length tuple of integers that is to be structured like this:
```python
(left, top, right, bottom) # values represent pixels
```
For example, if you want to only capture a 200px by 200px region offset by 100px from both the left and top, you would do:
```python
d.screenshot(region=(100, 100, 300, 300))
```
If you are capturing a scaled display, the region will be computed against the full, non-scaled resolution.
If you go through the source code, you will notice that the region cropping happens after a full display capture. That might seem sub-optimal but testing has revealed that copying a region of the GPU _D3D11Texture2D_ to the destination CPU _D3D11Texture2D_ using _CopySubresourceRegion_ is only faster when the region is very small. In fact, it doesn't take long for larger regions to actually start becoming slower than the full display capture using this method. To make things worse, it adds a lot of complexity by having the surface pitch not match the buffer size and treating rotated displays differently. It was therefore decided that it made more sense to stick to _CopyResource_ in all cases and crop after the fact.
## Usage
**Create a D3DShot instance**
```python
import d3dshot
d = d3dshot.create()
```
`create` accepts 2 optional kwargs:
* `capture_output`: Which capture output to use. See the _Capture Outputs_ section under _Concepts_
* `frame_buffer_size`: The maximum size the frame buffer can grow to. See the _Frame Buffer_ section under _Concepts_
Do NOT import the _D3DShot_ class directly and attempt to initialize it yourself! The `create` helper function initializes and validates a bunch of things for you behind the scenes.
Once you have a _D3DShot_ instance in scope, we can start doing stuff with it!
**List the detected displays**
```python
d.displays
```
**Select a display for capture**
Your primary display is selected by default but if you have a multi-monitor setup, you can select another entry in `d.displays`
```python
d.display = d.displays[1]
```
**Take a screenshot**
```python
d.screenshot()
```
`screenshot` accepts 1 optional kwarg:
* `region`: A region tuple. See the _Regions_ section under _Concepts_
_Returns_: A screenshot with a format that matches the capture output you selected when creating your _D3DShot_ object
**Take a screenshot and save it to disk**
```python
d.screenshot_to_disk()
```
`screenshot_to_disk` accepts 3 optional kwargs:
* `directory`: The path / directory where to write the file. If omitted, the working directory of the program will be used
* `file_name`: The file name to use. Permitted extensions are: _.png_, _.jpg_. If omitted, the file name will be `<time.time()>.png`
* `region`: A region tuple. See the _Regions_ section under _Concepts_
_Returns_: A string representing the full path to the saved image file
**Take a screenshot every X seconds**
```python
d.screenshot_every(X) # Where X is a number representing seconds
```
This operation is threaded and non-blocking. It will keep running until `d.stop()` is called. Captures are pushed to the frame buffer.
`screenshot_every` accepts 1 optional kwarg:
* `region`: A region tuple. See the _Regions_ section under _Concepts_
_Returns_: A boolean indicating whether or not the capture thread was started
**Take a screenshot every X seconds and save it to disk**
```python
d.screenshot_to_disk_every(X) # Where X is a number representing seconds
```
This operation is threaded and non-blocking. It will keep running until `d.stop()` is called.
`screenshot_to_disk_every` accepts 2 optional kwargs:
* `directory`: The path / directory where to write the file. If omitted, the working directory of the program will be used
* `region`: A region tuple. See the _Regions_ section under _Concepts_
_Returns_: A boolean indicating whether or not the capture thread was started
**Start a high-speed screen capture**
```python
d.capture()
```
This operation is threaded and non-blocking. It will keep running until `d.stop()` is called. Captures are pushed to the frame buffer.
`capture` accepts 2 optional kwargs:
* `target_fps`: How many captures per second to aim for. The effective capture rate will go under if the system can't keep up but it will never go over this target. It is recommended to set this to a reasonable value for your use case in order not to waste system resources. Default is set to 60.
* `region`: A region tuple. See the _Regions_ section under _Concepts_
_Returns_: A boolean indicating whether or not the capture thread was started
**Grab the latest frame from the buffer**
```python
d.get_latest_frame()
```
_Returns_: A frame with a format that matches the capture output you selected when creating your _D3DShot_ object
**Grab a specific frame from the buffer**
```python
d.get_frame(X) # Where X is the index of the desired frame. Needs to be < len(d.frame_buffer)
```
_Returns_: A frame with a format that matches the capture output you selected when creating your _D3DShot_ object
**Grab specific frames from the buffer**
```python
d.get_frames([X, Y, Z, ...]) # Where X, Y, Z are valid indices to desired frames
```
_Returns_: A list of frames with a format that matches the capture output you selected when creating your _D3DShot_ object
**Grab specific frames from the buffer as a stack**
```python
d.get_frame_stack([X, Y, Z, ...], stack_dimension="first|last") # Where X, Y, Z are valid indices to desired frames
```
Only has an effect on NumPy and PyTorch capture outputs.
`get_frame_stack` accepts 1 optional kwarg:
* `stack_dimension`: One of _first_, _last_. Which axis / dimension to perform the stack on
_Returns_: A single array stacked on the specified dimension with a format that matches the capture output you selected when creating your _D3DShot_ object. If the capture output is not stackable, returns a list of frames.
**Dump the frame buffer to disk**
The files will be named according to this convention: `<frame buffer index>.png`
```python
d.frame_buffer_to_disk()
```
`frame_buffer_to_disk` accepts 1 optional kwarg:
* `directory`: The path / directory where to write the files. If omitted, the working directory of the program will be used
_Returns_: None
## Performance
Measuring the exact performance of the Windows Desktop Duplication API proves to be a little complicated because it will only return new texture data if the contents of the screen has changed. This is optimal for performance but it makes it difficult to express in terms of frames per second, the measurement people tend to expect for benchmarks. Ultimately the solution ended up being to run a high FPS video game on the display to capture to make sure the screen contents is different at all times while benchmarking.
As always, remember that benchmarks are inherently flawed and highly depend on your individual hardware configuration and other circumstances. Use the numbers below as a relative indication of what to expect from _D3DShot_, not as some sort of absolute truth.
| | 2560x1440 on _NVIDIA GTX 1080 Ti_ | 1920x1080 on _Intel UHD Graphics 630_ | 1080x1920 (vertical) on _Intel UHD Graphics 630_ |
|-------------------------|-----------------------------------|---------------------------------------|--------------------------------------------------|
| **"pil"** | 29.717 FPS | 47.75 FPS | 35.95 FPS |
| **"numpy"** | **57.667 FPS** | **58.1 FPS** | **58.033 FPS** |
| **"numpy_float"** | 18.783 FPS | 29.05 FPS | 27.517 FPS |
| **"pytorch"** | **57.867 FPS** | **58.1 FPS** | 34.817 FPS |
| **"pytorch_float"** | 18.767 FPS | 28.367 FPS | 27.017 FPS |
| **"pytorch_gpu"** | 27.333 FPS | 35.767 FPS | 34.8 FPS |
| **"pytorch_float_gpu"** | 27.267 FPS | 37.383 FPS | 35.033 FPS |
The absolute fastest capture outputs appear to be _"numpy"_ and unrotated _"pytorch"_; all averaging around 58 FPS. In Python land, this is FAST!
#### How is the "numpy" capture output performance _that_ good?
NumPy arrays have a ctypes interface that can give you their raw memory address (`X.ctypes.data`). If you have the memory address and size of another byte buffer, which is what we end up with by processing what returns from the Desktop Duplication API, you can use `ctypes.memmove` to copy that byte buffer directly to the NumPy structure, effectively bypassing as much Python as possible.
In practice it ends up looking like this:
```python
ctypes.memmove(np.empty((size,), dtype=np.uint8).ctypes.data, pointer, size)
```
This low-level operation is extremely fast, leaving everything else that would normally compete with NumPy in the dust.
#### Why is the "pytorch" capture output slower on rotated displays?
Don't tell anyone but the reason it can compete with NumPy in the first place is only because... _it is_ generated from a NumPy array built from the method above! If you sniff around the code, you will indeed find `torch.from_numpy()` scattered around. This pretty much matches the speed of the "numpy" capture output 1:1, except when dealing with a rotated display. Display rotation is handled by `np.rot90()` calls which yields negative strides on that array. Negative strides are understood and perform well under NumPy but are still unsupported in PyTorch at the time of writing. To address this, an additional copy operation is needed to bring it back to a contiguous array which imposes a performance penalty.
#### Why is the "pil" capture output, being the default, not the fastest?
PIL has no ctypes interface like NumPy so a bytearray needs to be read into Python first and then fed to `PIL.Image.frombytes()`. This is still fast in Python terms, but it just cannot match the speed of the low-level NumPy method.
It remains the default capture output because:
1) PIL Image objects tend to be familiar to Python users
2) It's a way lighter / simpler dependency for a library compared to NumPy or PyTorch
#### Why are the float versions of capture outputs slower?
The data of the Direct3D textures made accessible by the Desktop Duplication API is formatted as bytes. To represent this data as normalized floats instead, a type cast and element-wise division needs to be performed on the array holding those bytes. This imposes a major performance penalty. Interestingly, you can see this performance penalty mitigated on GPU PyTorch tensors since the element-wise division can be massively parallelized on the device.
## Acknowledgements
_Crafted with ❤ by Serpent.AI 🐍_
[Twitter](https://twitter.com/Serpent_AI) - [Twitch](https://www.twitch.tv/serpent_ai) - [GitHub](https://github.com/SerpentAI) - [@nbrochu](https://github.com/nbrochu)
Raw data
{
"_id": null,
"home_page": null,
"name": "typed-D3DShot",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Avasam <samuel.06@hotmail.com>",
"keywords": "Computer Vision, D3D, Desktop Duplication, Direct3D, Screen Capture, Screenshot, Windows",
"author": null,
"author_email": "Nicholas Brochu <nicholas@serpent.ai>",
"download_url": "https://files.pythonhosted.org/packages/91/05/faad24e43db0333df99ad93cf4472702d4bfcb2bc7d946186a312f1fe3b8/typed_d3dshot-1.0.1.tar.gz",
"platform": null,
"description": "# typed-D3DShot (D3DShot but statically typed!)\r\n\r\nThis fork of the [now archived D3DShot](https://github.com/SerpentAI/D3DShot) aims to add static type-checking support by adding type annotations and making certain classes generic. With [typed-D3DShot](https://pypi.org/project/typed-D3DShot/), you don't need to install Typeshed's defunct [types-D3DShot](https://pypi.org/project/types-D3DShot/). `typed-D3DShot`'s own code is type-checked using [mypy](/mypy.ini) and [pyright](/pyrightconfig.json).\r\n\r\nNote that I do not intend on maintaining this library outside typing support and what I need for [AutoSplit](https://github.com/Toufool/AutoSplit).\r\n\r\n---\r\n\r\n_D3DShot_ is a pure Python implementation of the [Windows Desktop Duplication API](https://docs.microsoft.com/en-us/windows/desktop/direct3ddxgi/desktop-dup-api). It leverages DXGI and Direct3D system libraries to enable extremely fast and robust screen capture functionality for your Python scripts and applications on Windows.\r\n\r\n**D3DShot:**\r\n\r\n* Is by far the fastest way to capture the screen with Python on Windows 8.1+\r\n* Is very easy to use. If you can remember 10-ish methods, you know the entire thing.\r\n* Covers all common scenarios and use cases:\r\n * Screenshot to memory\r\n * Screenshot to disk\r\n * Screenshot to memory buffer every X seconds (threaded; non-blocking)\r\n * Screenshot to disk every X seconds (threaded; non-blocking)\r\n * High-speed capture to memory buffer (threaded; non-blocking)\r\n* Output options to capture to PIL Images, NumPy or PyTorch.\r\n* Detects displays in just about any configuration: Single monitor, multiple monitors on one adapter, multiple monitors on multiple adapters.\r\n* Handles display rotation and scaling for you\r\n* Supports capturing specific regions of the screen\r\n* Is robust and very stable. You can run it for hours / days without performance degradation\r\n* Is even able to capture DirectX 11 / 12 exclusive fullscreen applications and games!\r\n\r\n### TL;DR Quick Code Samples\r\n\r\n**Screenshot to Memory**\r\n\r\n```python\r\nimport d3dshot\r\n\r\nd = d3dshot.create()\r\nd.screenshot()\r\n```\r\n\r\n```python\r\nOut[1]: <PIL.Image.Image image mode=RGB size=2560x1440 at 0x1AA7ECB5C88>\r\n```\r\n\r\n**Screenshot to Disk**\r\n\r\n```python\r\nimport d3dshot\r\n\r\nd = d3dshot.create()\r\nd.screenshot_to_disk()\r\n```\r\n\r\n```python\r\nOut[1]: './1554298682.5632973.png'\r\n```\r\n\r\n**Screen Capture for 5 Seconds and Grab the Latest Frame**\r\n\r\n```python\r\nimport d3dshot\r\nimport time\r\n\r\nd = d3dshot.create()\r\n\r\nd.capture()\r\ntime.sleep(5) # Capture is non-blocking so we wait explicitly\r\nd.stop()\r\n\r\nd.get_latest_frame()\r\n```\r\n\r\n```python\r\nOut[1]: <PIL.Image.Image image mode=RGB size=2560x1440 at 0x1AA044BCF60>\r\n```\r\n\r\n**Screen Capture the Second Monitor as NumPy Arrays for 3 Seconds and Grab the 4 Latest Frames as a Stack**\r\n\r\n```python\r\nimport d3dshot\r\nimport time\r\n\r\nd = d3dshot.create(capture_output=\"numpy\")\r\n\r\nd.display = d.displays[1]\r\n\r\nd.capture()\r\ntime.sleep(3) # Capture is non-blocking so we wait explicitly\r\nd.stop()\r\n\r\nframe_stack = d.get_frame_stack((0, 1, 2, 3), stack_dimension=\"last\")\r\nframe_stack.shape\r\n```\r\n\r\n```python\r\nOut[1]: (1080, 1920, 3, 4)\r\n```\r\n\r\nThis is barely scratching the surface... Keep reading!\r\n\r\n## Requirements\r\n\r\n* Windows 8.1+ (64-bit)\r\n* Python 3.8+ (64-bit)\r\n\r\n## Installation\r\n\r\n_D3DShot_ leverages DLLs that are already available on your system so the dependencies are very light. Namely:\r\n\r\n* [_comtypes_](https://github.com/enthought/comtypes): Internal use. To preserve developer sanity while working with COM interfaces.\r\n\r\nThese dependencies will automatically be installed alongside _D3DShot_; No need to worry about them!\r\n\r\nSome more dependencies are required depending on your preferred capture output:\r\n\r\n* [_Pillow_](https://github.com/python-pillow/Pillow): Default Capture Output. Also used to save to disk as PNG and JPG.\r\n* [_NumPy_](https://github.com/numpy/numpy): Works extremely fast with multidimensional arrays, which images are.\r\n* [_PyTorch_](https://github.com/numpy/numpy): Offers Tensor computation with strong GPU acceleration.\r\n\r\nOptional dependencies (also known as \"extras\") are provided to make it easy to install all needed dependencies.\r\n\r\n```shell\r\n# For the NumPy capture output\r\npip install d3dshot[numpy]\r\n# For the Pillow capture output\r\npip install d3dshot[PIL]\r\n# For the PyTorch capture output\r\npip install d3dshot[torch]\r\n# You can mix if needed\r\npip install d3dshot[numpy][PIL]\r\n```\r\n\r\n##### Extra Step: Laptop Users\r\n\r\nWindows has a quirk when using Desktop Duplication on hybrid-GPU systems. Please see the [wiki article](https://github.com/SerpentAI/D3DShot/wiki/Installation-Note:-Laptops) before attempting to use _D3DShot_ on your system.\r\n\r\n## Concepts\r\n\r\n### Capture Outputs\r\n\r\nThe desired _Capture Output_ is defined when creating a _D3DShot_ instance. It defines the type of all captured images. By default, all captures will return PIL.Image objects. This is a good option if you mostly intend to take screenshots.\r\n\r\n```python\r\n# Captures will be PIL.Image in RGB mode\r\nd = d3dshot.create()\r\nd = d3dshot.create(capture_output=\"pil\")\r\n```\r\n\r\n_D3DShot_ is however quite flexible! As your environment meets certain optional sets of requirements, more options become available.\r\n\r\n**If _NumPy_ is available**\r\n\r\n```python\r\n# Captures will be np.ndarray of dtype uint8 with values in range (0, 255)\r\nd = d3dshot.create(capture_output=\"numpy\")\r\n\r\n# Captures will be np.ndarray of dtype float64 with normalized values in range (0.0, 1.0)\r\nd = d3dshot.create(capture_output=\"numpy_float\") \r\n```\r\n\r\n**If _NumPy_ and _PyTorch_ are available**\r\n\r\n```python\r\n# Captures will be torch.Tensor of dtype uint8 with values in range (0, 255)\r\nd = d3dshot.create(capture_output=\"pytorch\")\r\n\r\n# Captures will be torch.Tensor of dtype float64 with normalized values in range (0.0, 1.0)\r\nd = d3dshot.create(capture_output=\"pytorch_float\")\r\n```\r\n\r\n**If _NumPy_ and _PyTorch_ are available + _CUDA_ is installed and _torch.cuda.is_available()_**\r\n\r\n```python\r\n# Captures will be torch.Tensor of dtype uint8 with values in range (0, 255) on device cuda:0\r\nd = d3dshot.create(capture_output=\"pytorch_gpu\")\r\n\r\n# Captures will be torch.Tensor of dtype float64 with normalized values in range (0.0, 1.0) on device cuda:0\r\nd = d3dshot.create(capture_output=\"pytorch_float_gpu\")\r\n```\r\n\r\nTrying to use a Capture Output for which your environment does not meet the requirements will result in an error.\r\n\r\n### Singleton\r\n\r\nWindows only allows 1 instance of Desktop Duplication per process. To make sure we fall in line with that limitation to avoid issues, the _D3DShot_ class acts as a singleton. Any subsequent calls to `d3dshot.create()` will always return the existing instance.\r\n\r\n```python\r\nd = d3dshot.create(capture_output=\"numpy\")\r\n\r\n# Attempting to create a second instance\r\nd2 = d3dshot.create(capture_output=\"pil\")\r\n# Only 1 instance of D3DShot is allowed per process! Returning the existing instance...\r\n\r\n# Capture output remains 'numpy'\r\nd2.capture_output.backend\r\n# Out[1]: <d3dshot.capture_outputs.numpy_capture_output.NumpyCaptureOutput at 0x2672be3b8e0>\r\n\r\nd == d2\r\n# Out[2]: True\r\n``` \r\n\r\n### Frame Buffer\r\n\r\nWhen you create a _D3DShot_ instance, a frame buffer is also initialized. It is meant as a thread-safe, first-in, first-out way to hold a certain quantity of captures and is implemented as a `collections.deque`.\r\n\r\nBy default, the size of the frame buffer is set to 60. You can customize it when creating your _D3DShot_ object.\r\n\r\n```python\r\nd = d3dshot.create(frame_buffer_size=100)\r\n```\r\n\r\nBe mindful of RAM usage with larger values; You will be dealing with uncompressed images which use up to 100 MB each depending on the resolution.\r\n\r\nThe frame buffer can be accessed directly with `d.frame_buffer` but the usage of the utility methods instead is recommended.\r\n\r\nThe buffer is used by the following methods:\r\n\r\n* `d.capture()`\r\n* `d.screenshot_every()`\r\n\r\nIt is always automatically cleared before starting one of these operations.\r\n\r\n### Displays\r\n\r\nWhen you create a _D3DShot_ instance, your available displays will automatically be detected along with all their relevant properties.\r\n\r\n```python\r\nd.displays\r\n```\r\n\r\n```python\r\nOut[1]: \r\n[<Display name=BenQ XL2730Z (DisplayPort) adapter=NVIDIA GeForce GTX 1080 Ti resolution=2560x1440 rotation=0 scale_factor=1.0 primary=True>,\r\n <Display name=BenQ XL2430T (HDMI) adapter=Intel(R) UHD Graphics 630 resolution=1920x1080 rotation=0 scale_factor=1.0 primary=False>]\r\n```\r\n\r\nBy default, your primary display will be selected. At all times you can verify which display is set to be used for capture.\r\n\r\n```python\r\nd.display\r\n```\r\n\r\n```python\r\nOut[1]: <Display name=BenQ XL2730Z (DisplayPort) adapter=NVIDIA GeForce GTX 1080 Ti resolution=2560x1440 rotation=0 scale_factor=1.0 primary=True>\r\n```\r\n\r\nSelecting another display for capture is as simple as setting `d.display` to another value from `d.displays`\r\n\r\n```python\r\nd.display = d.displays[1]\r\nd.display\r\n```\r\n\r\n```python\r\nOut[1]: <Display name=BenQ XL2430T (HDMI) adapter=Intel(R) UHD Graphics 630 resolution=1080x1920 rotation=90 scale_factor=1.0 primary=False>\r\n```\r\n\r\nDisplay rotation and scaling is detected and handled for you by _D3DShot_:\r\n\r\n* Captures on rotated displays will always be in the correct orientation (i.e. matching what you see on your physical displays)\r\n* Captures on scaled displays will always be in full, non-scaled resolution (e.g. 1280x720 at 200% scaling will yield 2560x1440 captures)\r\n\r\n### Regions\r\n\r\nAll capture methods (screenshots included) accept an optional `region` kwarg. The expected value is a 4-length tuple of integers that is to be structured like this:\r\n\r\n```python\r\n(left, top, right, bottom) # values represent pixels\r\n```\r\n\r\nFor example, if you want to only capture a 200px by 200px region offset by 100px from both the left and top, you would do:\r\n\r\n```python\r\nd.screenshot(region=(100, 100, 300, 300))\r\n```\r\n\r\nIf you are capturing a scaled display, the region will be computed against the full, non-scaled resolution.\r\n\r\nIf you go through the source code, you will notice that the region cropping happens after a full display capture. That might seem sub-optimal but testing has revealed that copying a region of the GPU _D3D11Texture2D_ to the destination CPU _D3D11Texture2D_ using _CopySubresourceRegion_ is only faster when the region is very small. In fact, it doesn't take long for larger regions to actually start becoming slower than the full display capture using this method. To make things worse, it adds a lot of complexity by having the surface pitch not match the buffer size and treating rotated displays differently. It was therefore decided that it made more sense to stick to _CopyResource_ in all cases and crop after the fact.\r\n\r\n## Usage\r\n\r\n**Create a D3DShot instance**\r\n\r\n```python\r\nimport d3dshot\r\n\r\nd = d3dshot.create()\r\n```\r\n\r\n`create` accepts 2 optional kwargs:\r\n\r\n* `capture_output`: Which capture output to use. See the _Capture Outputs_ section under _Concepts_\r\n* `frame_buffer_size`: The maximum size the frame buffer can grow to. See the _Frame Buffer_ section under _Concepts_\r\n\r\nDo NOT import the _D3DShot_ class directly and attempt to initialize it yourself! The `create` helper function initializes and validates a bunch of things for you behind the scenes.\r\n\r\nOnce you have a _D3DShot_ instance in scope, we can start doing stuff with it!\r\n\r\n**List the detected displays**\r\n\r\n```python\r\nd.displays\r\n```\r\n\r\n**Select a display for capture**\r\n\r\nYour primary display is selected by default but if you have a multi-monitor setup, you can select another entry in `d.displays`\r\n\r\n```python\r\nd.display = d.displays[1]\r\n```\r\n\r\n**Take a screenshot**\r\n\r\n```python\r\nd.screenshot()\r\n```\r\n\r\n`screenshot` accepts 1 optional kwarg:\r\n\r\n* `region`: A region tuple. See the _Regions_ section under _Concepts_\r\n\r\n_Returns_: A screenshot with a format that matches the capture output you selected when creating your _D3DShot_ object\r\n\r\n**Take a screenshot and save it to disk**\r\n\r\n```python\r\nd.screenshot_to_disk()\r\n```\r\n\r\n`screenshot_to_disk` accepts 3 optional kwargs:\r\n\r\n* `directory`: The path / directory where to write the file. If omitted, the working directory of the program will be used\r\n* `file_name`: The file name to use. Permitted extensions are: _.png_, _.jpg_. If omitted, the file name will be `<time.time()>.png`\r\n* `region`: A region tuple. See the _Regions_ section under _Concepts_\r\n\r\n_Returns_: A string representing the full path to the saved image file\r\n\r\n**Take a screenshot every X seconds**\r\n\r\n```python\r\nd.screenshot_every(X) # Where X is a number representing seconds\r\n```\r\n\r\nThis operation is threaded and non-blocking. It will keep running until `d.stop()` is called. Captures are pushed to the frame buffer.\r\n\r\n`screenshot_every` accepts 1 optional kwarg:\r\n\r\n* `region`: A region tuple. See the _Regions_ section under _Concepts_\r\n\r\n_Returns_: A boolean indicating whether or not the capture thread was started\r\n\r\n**Take a screenshot every X seconds and save it to disk**\r\n\r\n```python\r\nd.screenshot_to_disk_every(X) # Where X is a number representing seconds\r\n```\r\n\r\nThis operation is threaded and non-blocking. It will keep running until `d.stop()` is called.\r\n\r\n`screenshot_to_disk_every` accepts 2 optional kwargs:\r\n\r\n* `directory`: The path / directory where to write the file. If omitted, the working directory of the program will be used\r\n* `region`: A region tuple. See the _Regions_ section under _Concepts_\r\n\r\n_Returns_: A boolean indicating whether or not the capture thread was started\r\n\r\n**Start a high-speed screen capture**\r\n\r\n```python\r\nd.capture()\r\n```\r\n\r\nThis operation is threaded and non-blocking. It will keep running until `d.stop()` is called. Captures are pushed to the frame buffer.\r\n\r\n`capture` accepts 2 optional kwargs:\r\n\r\n* `target_fps`: How many captures per second to aim for. The effective capture rate will go under if the system can't keep up but it will never go over this target. It is recommended to set this to a reasonable value for your use case in order not to waste system resources. Default is set to 60.\r\n* `region`: A region tuple. See the _Regions_ section under _Concepts_\r\n\r\n_Returns_: A boolean indicating whether or not the capture thread was started\r\n\r\n**Grab the latest frame from the buffer**\r\n\r\n```python\r\nd.get_latest_frame()\r\n```\r\n\r\n_Returns_: A frame with a format that matches the capture output you selected when creating your _D3DShot_ object\r\n\r\n**Grab a specific frame from the buffer**\r\n\r\n```python\r\nd.get_frame(X) # Where X is the index of the desired frame. Needs to be < len(d.frame_buffer)\r\n```\r\n\r\n_Returns_: A frame with a format that matches the capture output you selected when creating your _D3DShot_ object\r\n\r\n**Grab specific frames from the buffer**\r\n\r\n```python\r\nd.get_frames([X, Y, Z, ...]) # Where X, Y, Z are valid indices to desired frames\r\n```\r\n\r\n_Returns_: A list of frames with a format that matches the capture output you selected when creating your _D3DShot_ object\r\n\r\n**Grab specific frames from the buffer as a stack**\r\n\r\n```python\r\nd.get_frame_stack([X, Y, Z, ...], stack_dimension=\"first|last\") # Where X, Y, Z are valid indices to desired frames\r\n```\r\n\r\nOnly has an effect on NumPy and PyTorch capture outputs.\r\n\r\n`get_frame_stack` accepts 1 optional kwarg:\r\n\r\n* `stack_dimension`: One of _first_, _last_. Which axis / dimension to perform the stack on\r\n\r\n_Returns_: A single array stacked on the specified dimension with a format that matches the capture output you selected when creating your _D3DShot_ object. If the capture output is not stackable, returns a list of frames.\r\n\r\n**Dump the frame buffer to disk**\r\n\r\nThe files will be named according to this convention: `<frame buffer index>.png`\r\n\r\n```python\r\nd.frame_buffer_to_disk()\r\n```\r\n\r\n`frame_buffer_to_disk` accepts 1 optional kwarg:\r\n\r\n* `directory`: The path / directory where to write the files. If omitted, the working directory of the program will be used\r\n\r\n_Returns_: None\r\n\r\n## Performance\r\n\r\nMeasuring the exact performance of the Windows Desktop Duplication API proves to be a little complicated because it will only return new texture data if the contents of the screen has changed. This is optimal for performance but it makes it difficult to express in terms of frames per second, the measurement people tend to expect for benchmarks. Ultimately the solution ended up being to run a high FPS video game on the display to capture to make sure the screen contents is different at all times while benchmarking.\r\n\r\nAs always, remember that benchmarks are inherently flawed and highly depend on your individual hardware configuration and other circumstances. Use the numbers below as a relative indication of what to expect from _D3DShot_, not as some sort of absolute truth.\r\n\r\n| | 2560x1440 on _NVIDIA GTX 1080 Ti_ | 1920x1080 on _Intel UHD Graphics 630_ | 1080x1920 (vertical) on _Intel UHD Graphics 630_ |\r\n|-------------------------|-----------------------------------|---------------------------------------|--------------------------------------------------|\r\n| **\"pil\"** | 29.717 FPS | 47.75 FPS | 35.95 FPS |\r\n| **\"numpy\"** | **57.667 FPS** | **58.1 FPS** | **58.033 FPS** |\r\n| **\"numpy_float\"** | 18.783 FPS | 29.05 FPS | 27.517 FPS |\r\n| **\"pytorch\"** | **57.867 FPS** | **58.1 FPS** | 34.817 FPS |\r\n| **\"pytorch_float\"** | 18.767 FPS | 28.367 FPS | 27.017 FPS |\r\n| **\"pytorch_gpu\"** | 27.333 FPS | 35.767 FPS | 34.8 FPS |\r\n| **\"pytorch_float_gpu\"** | 27.267 FPS | 37.383 FPS | 35.033 FPS |\r\n\r\nThe absolute fastest capture outputs appear to be _\"numpy\"_ and unrotated _\"pytorch\"_; all averaging around 58 FPS. In Python land, this is FAST!\r\n\r\n#### How is the \"numpy\" capture output performance _that_ good?\r\n\r\nNumPy arrays have a ctypes interface that can give you their raw memory address (`X.ctypes.data`). If you have the memory address and size of another byte buffer, which is what we end up with by processing what returns from the Desktop Duplication API, you can use `ctypes.memmove` to copy that byte buffer directly to the NumPy structure, effectively bypassing as much Python as possible.\r\n\r\nIn practice it ends up looking like this:\r\n\r\n```python\r\nctypes.memmove(np.empty((size,), dtype=np.uint8).ctypes.data, pointer, size)\r\n```\r\n\r\nThis low-level operation is extremely fast, leaving everything else that would normally compete with NumPy in the dust.\r\n\r\n#### Why is the \"pytorch\" capture output slower on rotated displays?\r\n\r\nDon't tell anyone but the reason it can compete with NumPy in the first place is only because... _it is_ generated from a NumPy array built from the method above! If you sniff around the code, you will indeed find `torch.from_numpy()` scattered around. This pretty much matches the speed of the \"numpy\" capture output 1:1, except when dealing with a rotated display. Display rotation is handled by `np.rot90()` calls which yields negative strides on that array. Negative strides are understood and perform well under NumPy but are still unsupported in PyTorch at the time of writing. To address this, an additional copy operation is needed to bring it back to a contiguous array which imposes a performance penalty.\r\n\r\n#### Why is the \"pil\" capture output, being the default, not the fastest?\r\n\r\nPIL has no ctypes interface like NumPy so a bytearray needs to be read into Python first and then fed to `PIL.Image.frombytes()`. This is still fast in Python terms, but it just cannot match the speed of the low-level NumPy method.\r\n\r\nIt remains the default capture output because:\r\n\r\n1) PIL Image objects tend to be familiar to Python users\r\n2) It's a way lighter / simpler dependency for a library compared to NumPy or PyTorch\r\n\r\n#### Why are the float versions of capture outputs slower?\r\n\r\nThe data of the Direct3D textures made accessible by the Desktop Duplication API is formatted as bytes. To represent this data as normalized floats instead, a type cast and element-wise division needs to be performed on the array holding those bytes. This imposes a major performance penalty. Interestingly, you can see this performance penalty mitigated on GPU PyTorch tensors since the element-wise division can be massively parallelized on the device.\r\n\r\n## Acknowledgements\r\n\r\n_Crafted with \u2764 by Serpent.AI \ud83d\udc0d_ \r\n[Twitter](https://twitter.com/Serpent_AI) - [Twitch](https://www.twitch.tv/serpent_ai) - [GitHub](https://github.com/SerpentAI) - [@nbrochu](https://github.com/nbrochu)\r\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2019 Serpent.AI / Nicholas Brochu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Extremely Fast, Robust, and Typed, Screen Capture on Windows with the Desktop Duplication API",
"version": "1.0.1",
"project_urls": {
"Changelog": "https://github.com/Avasam/D3DShot/blob/HEAD/CHANGELOG.md",
"Homepage": "https://github.com/Avasam/D3DShot",
"Issues": "https://github.com/Avasam/D3DShot/issues",
"Repository": "https://github.com/Avasam/D3DShot.git"
},
"split_keywords": [
"computer vision",
" d3d",
" desktop duplication",
" direct3d",
" screen capture",
" screenshot",
" windows"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "8923bafe1f74835ff57efc8b8bbdec95e3a5d5e4316f17c51616fb33aeaa24f2",
"md5": "3f792affb8c962ff2680d04d40c22489",
"sha256": "775cbb476535d98fb0bdc99b21d247af44957608a38bd1f6b5de7da74a2a991f"
},
"downloads": -1,
"filename": "typed_d3dshot-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3f792affb8c962ff2680d04d40c22489",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 33511,
"upload_time": "2024-10-09T20:43:27",
"upload_time_iso_8601": "2024-10-09T20:43:27.779478Z",
"url": "https://files.pythonhosted.org/packages/89/23/bafe1f74835ff57efc8b8bbdec95e3a5d5e4316f17c51616fb33aeaa24f2/typed_d3dshot-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9105faad24e43db0333df99ad93cf4472702d4bfcb2bc7d946186a312f1fe3b8",
"md5": "65608e2bde4baccbe2cd094c04392fbc",
"sha256": "7cf3fb6304bfde82d9a279c16fd5798100e035f3ac9f53b82a9bec718d84bb29"
},
"downloads": -1,
"filename": "typed_d3dshot-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "65608e2bde4baccbe2cd094c04392fbc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 33913,
"upload_time": "2024-10-09T20:43:29",
"upload_time_iso_8601": "2024-10-09T20:43:29.188723Z",
"url": "https://files.pythonhosted.org/packages/91/05/faad24e43db0333df99ad93cf4472702d4bfcb2bc7d946186a312f1fe3b8/typed_d3dshot-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-09 20:43:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Avasam",
"github_project": "D3DShot",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "typed-d3dshot"
}
