<p align="center">
<a href="https://github.com/mapillary/mapillary_tools/">
<img src="https://raw.githubusercontent.com/mapillary/mapillary_tools/main/docs/images/logo.png">
</a>
</p>
<p align="center">
<a href="https://pypi.org/project/mapillary_tools/"><img alt="PyPI" src="https://img.shields.io/pypi/v/mapillary_tools"></a>
<a href="https://github.com/mapillary/mapillary_tools/actions"><img alt="Actions Status" src="https://github.com/mapillary/mapillary_tools/actions/workflows/python-package.yml/badge.svg"></a>
<a href="https://github.com/mapillary/mapillary_tools/blob/main/LICENSE"><img alt="GitHub license" src="https://img.shields.io/github/license/mapillary/mapillary_tools"></a>
<a href="https://github.com/mapillary/mapillary_tools/stargazers"><img alt="GitHub stars" src="https://img.shields.io/github/stars/mapillary/mapillary_tools"></a>
<a href="https://pepy.tech/project/mapillary_tools"><img alt="Downloads" src="https://pepy.tech/badge/mapillary_tools"></a>
</p>
mapillary_tools is a command line tool that uploads geotagged images and videos to Mapillary.
```sh
# Install mapillary_tools
pip install mapillary_tools
# Process and upload images and videos in the directory
mapillary_tools process_and_upload MY_CAPTURE_DIR
# List all commands
mapillary_tools --help
```
<!--ts-->
- [Supported File Formats](#supported-file-formats)
- [Image Formats](#image-formats)
- [Video Formats](#video-formats)
- [Installation](#installation)
- [Standalone Executable](#standalone-executable)
- [Installing via pip](#installing-via-pip)
- [Installing on Android Devices](#installing-on-android-devices)
- [Usage](#usage)
- [Process and Upload](#process-and-upload)
- [Process](#process)
- [Upload](#upload)
- [Advanced Usage](#advanced-usage)
- [Local Video Processing](#local-video-processing)
- [Install FFmpeg](#install-ffmpeg)
- [Video Processing](#video-processing)
- [Geotagging with GPX](#geotagging-with-gpx)
- [New video geotagging features (experimental)](#new-video-geotagging-features-experimental)
- [Usage](#usage-1)
- [Examples](#examples)
- [Generic supported videos](#generic-supported-videos)
- [External GPX](#external-gpx)
- [Insta360 stitched videos](#insta360-stitched-videos)
- [Limitations of `--video_geotag_source`](#limitations-of---video_geotag_source)
- [Authenticate](#authenticate)
- [Examples](#examples-1)
- [Image Description](#image-description)
- [Zip Images](#zip-images)
- [Development](#development)
- [Setup](#setup)
- [Option 1: Using uv (Recommended)](#option-1-using-uv-recommended)
- [Option 2: Using pip with virtual environment](#option-2-using-pip-with-virtual-environment)
- [Running the code](#running-the-code)
- [Tests](#tests)
- [Code Quality](#code-quality)
- [Release and Build](#release-and-build)
<!--te-->
# Supported File Formats
mapillary_tools can upload both images and videos.
## Image Formats
mapillary_tools supports JPG/JEPG images (.jpg, .jpeg), with the following EXIF tags minimally required:
- GPS Longitude
- GPS Latitude
- Date/Time Original or GPS Date/Time
## Video Formats
mapillary_tools supports videos (.mp4, .360) that contain any of the following telemetry structures:
- [GPMF](https://github.com/gopro/gpmf-parser): mostly GoPro videos
- [GoPro HERO series](https://gopro.com/en/us/shop/cameras/hero11-black/CHDHX-111-master.html) (from 5 to 13)
- [GoPro MAX](https://gopro.com/en/us/shop/cameras/max/CHDHZ-202-master.html)
- [CAMM](https://developers.google.com/streetview/publish/camm-spec): an open-standard telemetry spec supported by a number of cameras
- [Insta360 Pro2](https://www.insta360.com/product/insta360-pro2/)
- [Insta360 Titan](https://www.insta360.com/product/insta360-titan)
- [Ricoh Theta X](https://theta360.com/en/about/theta/x.html)
- [Labpano](https://www.labpano.com/)
- and more...
- [BlackVue](https://blackvue.com/) videos
- [DR900S-1CH](https://shop.blackvue.com/product/dr900x-1ch-plus/)
- [DR900X Plus](https://shop.blackvue.com/product/dr900x-2ch-plus/)
# Installation
## Standalone Executable
1. Download the latest executable for your platform from the [releases](https://github.com/mapillary/mapillary_tools/releases).
2. Move the executable to your system `$PATH`
> **_NOTE:_** If you see the error "**mapillary_tools is damaged and can’t be opened**" on macOS, try to clear the extended attributes:
>
> ```
> xattr -c mapillary_tools
> ```
## Installing via pip
To install or upgrade to the latest stable version:
```sh
pip install --upgrade mapillary_tools
```
If you can't wait for the latest features in development, install it from GitHub:
```sh
pip install --upgrade git+https://github.com/mapillary/mapillary_tools
```
> **_NOTE:_** If you see "**Permission Denied**" error, try to run the command above with `sudo`, or install it in your
> local [virtualenv](#setup) (recommended).
### Installing on Android Devices
A command line program such as Termux is required. Installation can be done without root privileges. The following
commands will install Python 3, pip3, git, and all required libraries for mapillary_tools on Termux:
```sh
pkg install python git build-essential libgeos openssl libjpeg-turbo libexpat libexpat-static
pip install --upgrade pip wheel
pip install --upgrade mapillary_tools
```
Termux must access the device's internal storage to process and upload images. To do this, use the following command:
```sh
termux-setup-storage
```
Finally, on devices running Android 11, using a command line program, mapillary_tools will process images very slowly if
they are in shared internal storage during processing. It is advisable to first move images to the command line
program’s native directory before running mapillary_tools. For an example using Termux, if imagery is stored in the
folder `Internal storage/DCIM/mapillaryimages` the following command will move that folder from shared storage to
Termux:
```sh
mv -v storage/dcim/mapillaryimages mapillaryimages
```
# Usage
## Process and Upload
For most users, `process_and_upload` is the command to go:
```sh
# Process and upload all images and videos in MY_CAPTURE_DIR and its subfolders, and all videos under MY_VIDEO_DIR
mapillary_tools process_and_upload MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4
```
If any process error occurs, e.g. GPS not found in an image, mapillary_tools will exit with non-zero status code.
To ignore these errors and continue uploading the rest:
```sh
# Skip process errors and upload to the specified user and organization
mapillary_tools process_and_upload MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4 \
--skip_process_errors \
--user_name "my_username" \
--organization_key "my_organization_id"
```
The `process_and_upload` command will run the [`process`](#process) and the [`upload`](#upload) commands consecutively with combined required and optional arguments.
The command above is equivalent to:
```sh
mapillary_tools process MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4 \
--skip_process_errors \
--desc_path /tmp/mapillary_description_file.json
mapillary_tools upload MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4 \
--desc_path /tmp/mapillary_description_file.json \
--user_name "my_username" \
--organization_key "my_organization_id"
```
## Process
The `process` command is an intermediate step that extracts the metadata from images and videos,
and writes them in an [image description file](#image-description). Users should pass it to the [`upload`](#upload) command.
```sh
mapillary_tools process MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4
```
Duplicate check with custom distance and angle:
```sh
# Mark images that are 3 meters closer to its previous one as duplicates.
# Duplicates won't be uploaded
mapillary_tools process MY_CAPTURE_DIR \
--duplicate_distance 3 \
--duplicate_angle 360 # Set 360 to disable angle check
```
Split sequences with the custom cutoff distance or custom capture time gap:
```sh
# If two successive images are 100 meters apart,
# OR their capture times are 120 seconds apart,
# then split the sequence from there
mapillary_tools process MY_CAPTURE_DIR \
--offset_angle 90 \
--cutoff_distance 100 \
--cutoff_time 120 \
```
## Upload
After processing you should get the [image description file](#image-description). Pass it to the `upload` command to upload them:
```sh
# Upload processed images and videos to the specified user account and organization
mapillary_tools upload MY_CAPTURE_DIR \
--desc_path /tmp/mapillary_image_description.json \
--user_name "my_username" \
--organization_key "my_organization_id"
```
# Advanced Usage
## Local Video Processing
Local video processing samples a video into a sequence of sample images and ensures the images are geotagged and ready for uploading.
It gives users more control over the sampling process, for example, you can specify the sampling distance to control the density.
Also, the sample images have smaller file sizes than videos, hence saving bandwidth.
### Install FFmpeg
[FFmpeg](https://ffmpeg.org/) is required for local video processing.
You can download `ffmpeg` and `ffprobe` from [here](https://ffmpeg.org/download.html),
or install them with your favorite package manager.
### Video Processing
mapillary_tools first extracts the GPS track from the video's telemetry structure, and then locates video frames along the GPS track.
When all are located, it then extracts one frame (image) every 3 meters by default.
```sh
# Sample videos in MY_VIDEO_DIR and write the sample images in MY_SAMPLES with a custom sampling distance
mapillary_tools video_process MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5
# The command above is equivalent to
mapillary_tools sample_video MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5
mapillary_tools process MY_SAMPLES
```
To process and upload the sample images consecutively, run:
```sh
mapillary_tools video_process_and_upload MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5
# The command above is equivalent to
mapillary_tools video_process MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5 --desc_path=/tmp/mapillary_description.json
mapillary_tools upload MY_SAMPLES --desc_path=/tmp/mapillary_description.json
```
## Geotagging with GPX
If you use external GPS devices for mapping, you will need to geotag your captures with the external GPS tracks.
To geotag images with a GPX file, the capture time (extracted from EXIF tag "Date/Time Original" or "GPS Date/Time") is minimally required.
It is used to locate the images along the GPS tracks.
```sh
mapillary_tools process MY_IMAGE_DIR --geotag_source "gpx" --geotag_source_path MY_EXTERNAL_GPS.gpx
```
To geotag videos with a GPX file, video start time (video creation time minus video duration) is required to locate the sample images along the GPS tracks.
```sh
# Geotagging with GPX works with interval-based sampling only,
# the two options --video_sample_distance -1 --video_sample_interval 2 are therefore required
# to switch from the default distance-based sampling to the legacy interval-based sampling
mapillary_tools video_process MY_VIDEO_DIR \
--geotag_source "gpx" \
--geotag_source_path MY_EXTERNAL_GPS.gpx \
--video_sample_distance -1 --video_sample_interval 2
```
Ideally, the GPS device and the capture device should use the same clock to get the timestamps synchronized.
If not, as is often the case, the image locations will be shifted. To solve that, mapillary_tools provides an
option `--interpolation_offset_time N` that adds N seconds to image capture times for synchronizing the timestamps.
```sh
# The capture device's clock is 8 hours (i.e. -8 * 3600 = -28800 seconds) ahead of the GPS device's clock
mapillary_tools process MY_IMAGE_DIR \
--geotag_source "gpx" \
--geotag_source_path MY_EXTERNAL_GPS.gpx \
--interpolation_offset_time -28800
```
Another option `--interpolation_use_gpx_start_time` moves your images to align with the beginning of the GPS track.
This is useful when you can confirm that you start GPS recording and capturing at the same time, or with a known delay.
```sh
# Start capturing 2.5 seconds after start GPS recording
mapillary_tools video_process MY_VIDEO_DIR \
--geotag_source "gpx" \
--geotag_source_path MY_EXTERNAL_GPS.gpx \
--interpolation_use_gpx_start_time \
--interpolation_offset_time 2.5 \
--video_sample_distance -1 --video_sample_interval 2
```
## New video geotagging features (experimental)
As experimental features, mapillary_tools can now:
* Geotag videos from tracks recorded in GPX and NMEA files
* Invoke `exiftool` internally, if available on the system. `exiftool` can extract geolocation data from a wide
range of video formats, allowing us to support more cameras with less trouble for the end user.
* Try several geotagging sources sequentially, until proper data is found.
These features apply to the `process` command, which analyzes the video for direct upload instead of sampling it
into images. They are experimental and will be subject to change in future releases.
### Usage
The new video processing is triggered with the `--video_geotag_source SOURCE` option. It can be specified multiple times:
In this case, each source will be tried in turn, until one returns good quality data.
`SOURCE` can be:
1. the plain name of the source - one of `video, camm, gopro, blackvue, gpx, nmea, exiftool_xml, exiftool_runtime`
2. a JSON object that includes optional parameters
```json5
{
"source": "SOURCE",
"pattern": "FILENAME_PATTERN" // optional
}
```
PATTERN specifies how the data source file is named, starting from the video filename:
* `%f`: the full video filename
* `%g`: the video filename without extension
* `%e`: the video filename extension
Supported sources and their default pattern are:
* `video`: parse the video, in order, as `camm, gopro, blackvue`. Pattern: `%f`
* `camm`: parse the video looking for a CAMM track. Pattern: `%f`
* `gopro`: parse the video looking for geolocation in GoPro format. Pattern: `%f`
* `blackvue`: parse the video looking for geolocation in BlackVue format. Pattern: `%f`
* `gpx`: external GPX file. Pattern: `%g.gpx`
* `nmea`: external NMEA file. Pattern: `%g.nmea`
* `exiftool_xml`: external XML file generated by exiftool. Pattern: `%g.xml`
* `exiftool_runtime`: execute exiftool on the video file. Pattern: `%f`
Notes:
* `exiftool_runtime` only works if exiftool is installed on the system. You can find it at https://exiftool.org/ or through
your software manager. If exiftool is installed, but is not in the default execution path, it is
possible to specify its location by setting the environment variable `MAPILLARY_TOOLS_EXIFTOOL_PATH`.
* Pattern are case-sensitive or not depending on the filesystem - in Windows, `%g.gpx` will match both `basename.gpx`
and `basename.GPX`, in MacOs, Linux or other Unix systems no.
* If both `--video_geotag_source` and `--geotag_source` are specified, `--video_geotag_source` will apply to video files
and `--geotag_source` to image files.
### Examples
#### Generic supported videos
Process all videos in a directory, trying to parse them as CAMM, GoPro or BlackVue:
```sh
mapillary_tools process --video_geotag_source video VIDEO_DIR/
```
#### External GPX
Process all videos in a directory, taking geolocation data from GPX files. A video named `foo.mp4` will be associated
with a GPX file called `foo.gpx`.
```sh
mapillary_tools process --video_geotag_source gpx VIDEO_DIR/
```
#### Insta360 stitched videos
The videos to process have been stitched by Insta360 Studio; the geolocation data is in the original
videos in the parent directory, and there may be GPX files alongside the stitched video.
First look for GPX, then fallback to running exiftool against the original videos.
```sh
mapillary_tools process \
--video_geotag_source gpx \
--video_geotag_source '{"source": "exiftool_runtime", "pattern": "../%g.insv"}' \
VIDEO_DIR/
```
### Limitations of `--video_geotag_source`
**External geolocation sources will be aligned with the start of video, there is
currently no way of applying offsets or scaling the time.** This means, for instance, that GPX tracks must begin precisely
at the instant of the video, and that timelapse videos are supported only for sources `camm, gopro, blackvue`.
## Authenticate
The command `authenticate` will update the user credentials stored in the config file.
### Examples
Authenticate new user:
```sh
mapillary_tools authenticate
```
Authenticate for user `my_username`. If the user is already authenticated, it will update the credentials in the config:
```sh
mapillary_tools authenticate --user_name "my_username"
```
## Image Description
The output of the [`process`](#process) command is a JSON array of objects that describes metadata for each image or video.
The metadata is validated by the [image description schema](https://github.com/mapillary/mapillary_tools/blob/main/schema/image_description_schema.json).
Here is a minimal example:
```json
[
{
"MAPLatitude": 58.5927694,
"MAPLongitude": 16.1840944,
"MAPCaptureTime": "2021_02_13_13_24_41_140",
"filename": "/MY_IMAGE_DIR/IMG_0291.jpg"
},
{
"error": {
"type": "MapillaryGeoTaggingError",
"message": "Unable to extract GPS Longitude or GPS Latitude from the image"
},
"filename": "/MY_IMAGE_DIR/IMG_0292.jpg"
}
]
```
Users may create or manipulate the image description file before passing them to the [`upload`](#upload) command. Here are a few examples:
```sh
# Remove images outside the bounding box and map matching the rest images on the road network
mapillary_tools process MY_IMAGE_DIR | \
./filter_by_bbox.py 5.9559,45.818,10.4921,47.8084 | \
./map_match.py > /tmp/mapillary_image_description.json
# Upload the processed images
mapillary_tools upload MY_IMAGE_DIR --desc_path /tmp/mapillary_image_description.json
```
```sh
# Converts captures.csv to an image description file
./custom_csv_to_description.sh captures.csv | mapillary_tools upload MY_IMAGE_DIR --desc_path -
```
## Zip Images
When [uploading](#upload) an image directory, internally the `upload` command will zip sequences in the temporary
directory (`TMPDIR`) and then upload these zip files.
mapillary_tools provides `zip` command that allows users to specify where to store the zip files, usually somewhere with
faster IO or more free space.
```sh
# Zip processed images in MY_IMAGE_DIR and write zip files in MY_ZIPFILES
mapillary_tools zip MY_IMAGE_DIR MY_ZIPFILES
# Upload all the zip files (*.zip) in MY_ZIPFILES:
mapillary_tools upload --file_types zip MY_ZIPFILES
```
# Development
## Setup
Clone the repository:
```sh
git clone git@github.com:mapillary/mapillary_tools.git
cd mapillary_tools
```
### Option 1: Using uv (Recommended)
Use [uv](https://docs.astral.sh/uv/) - a fast Python package manager.
Install the project in development mode with all dependencies:
```sh
# Install the project and development dependencies
uv sync --group dev
# Activate the virtual environment
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
### Option 2: Using pip with virtual environment
Set up a virtual environment (recommended):
```sh
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
Install the project in development mode:
```sh
# Install the project and all dependencies in editable mode
pip install -e .
# Install development dependencies
pip install --group dev
```
## Running the code
Run the code from the repository:
```sh
# If you have mapillary_tools installed in editable mode
mapillary_tools --version
# Alternatively
python -m mapillary_tools.commands --version
```
## Tests
Run tests:
```sh
# Test all cases
pytest -s -vv tests
# Or test a single case specifically
pytest -s -vv tests/unit/test_camm_parser.py::test_build_and_parse
```
## Code Quality
Run code formatting and linting:
```sh
# Format code with ruff
ruff format mapillary_tools tests
# Lint code with ruff
ruff check mapillary_tools tests
# Sort imports with usort
usort format mapillary_tools tests
# Type checking with mypy
mypy mapillary_tools
```
## Release and Build
```sh
# Assume you are releasing v0.9.1a2 (alpha2)
# Tag your local branch
# Use -f here to replace the existing one
git tag -f v0.9.1a2
# Push the tagged commit first if it is not there yet
git push origin
# Push ALL local tags (TODO: How to push a specific tag?)
# Use -f here to replace the existing tags in the remote repo
git push origin --tags -f
# The last step will trigger CI to publish a draft release with binaries built
# in https://github.com/mapillary/mapillary_tools/releases
```
Raw data
{
"_id": null,
"home_page": null,
"name": "mapillary-tools",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "mapillary, gis, computer vision, street view",
"author": null,
"author_email": "Mapillary <support@mapillary.com>",
"download_url": "https://files.pythonhosted.org/packages/59/f1/988830d31a2bf182322996b38bffbee21d9dd5fa5ee60d4a6062e90ee0e6/mapillary_tools-0.14.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <a href=\"https://github.com/mapillary/mapillary_tools/\">\n <img src=\"https://raw.githubusercontent.com/mapillary/mapillary_tools/main/docs/images/logo.png\">\n </a>\n</p>\n\n<p align=\"center\">\n<a href=\"https://pypi.org/project/mapillary_tools/\"><img alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/mapillary_tools\"></a>\n<a href=\"https://github.com/mapillary/mapillary_tools/actions\"><img alt=\"Actions Status\" src=\"https://github.com/mapillary/mapillary_tools/actions/workflows/python-package.yml/badge.svg\"></a>\n<a href=\"https://github.com/mapillary/mapillary_tools/blob/main/LICENSE\"><img alt=\"GitHub license\" src=\"https://img.shields.io/github/license/mapillary/mapillary_tools\"></a>\n<a href=\"https://github.com/mapillary/mapillary_tools/stargazers\"><img alt=\"GitHub stars\" src=\"https://img.shields.io/github/stars/mapillary/mapillary_tools\"></a>\n<a href=\"https://pepy.tech/project/mapillary_tools\"><img alt=\"Downloads\" src=\"https://pepy.tech/badge/mapillary_tools\"></a>\n</p>\n\nmapillary_tools is a command line tool that uploads geotagged images and videos to Mapillary.\n\n```sh\n# Install mapillary_tools\npip install mapillary_tools\n\n# Process and upload images and videos in the directory\nmapillary_tools process_and_upload MY_CAPTURE_DIR\n\n# List all commands\nmapillary_tools --help\n```\n\n<!--ts-->\n\n- [Supported File Formats](#supported-file-formats)\n - [Image Formats](#image-formats)\n - [Video Formats](#video-formats)\n- [Installation](#installation)\n - [Standalone Executable](#standalone-executable)\n - [Installing via pip](#installing-via-pip)\n - [Installing on Android Devices](#installing-on-android-devices)\n- [Usage](#usage)\n - [Process and Upload](#process-and-upload)\n - [Process](#process)\n - [Upload](#upload)\n- [Advanced Usage](#advanced-usage)\n - [Local Video Processing](#local-video-processing)\n - [Install FFmpeg](#install-ffmpeg)\n - [Video Processing](#video-processing)\n - [Geotagging with GPX](#geotagging-with-gpx)\n - [New video geotagging features (experimental)](#new-video-geotagging-features-experimental)\n - [Usage](#usage-1)\n - [Examples](#examples)\n - [Generic supported videos](#generic-supported-videos)\n - [External GPX](#external-gpx)\n - [Insta360 stitched videos](#insta360-stitched-videos)\n - [Limitations of `--video_geotag_source`](#limitations-of---video_geotag_source)\n - [Authenticate](#authenticate)\n - [Examples](#examples-1)\n - [Image Description](#image-description)\n - [Zip Images](#zip-images)\n- [Development](#development)\n - [Setup](#setup)\n - [Option 1: Using uv (Recommended)](#option-1-using-uv-recommended)\n - [Option 2: Using pip with virtual environment](#option-2-using-pip-with-virtual-environment)\n - [Running the code](#running-the-code)\n - [Tests](#tests)\n - [Code Quality](#code-quality)\n - [Release and Build](#release-and-build)\n\n<!--te-->\n\n# Supported File Formats\n\nmapillary_tools can upload both images and videos.\n\n## Image Formats\n\nmapillary_tools supports JPG/JEPG images (.jpg, .jpeg), with the following EXIF tags minimally required:\n\n- GPS Longitude\n- GPS Latitude\n- Date/Time Original or GPS Date/Time\n\n## Video Formats\n\nmapillary_tools supports videos (.mp4, .360) that contain any of the following telemetry structures:\n\n- [GPMF](https://github.com/gopro/gpmf-parser): mostly GoPro videos\n - [GoPro HERO series](https://gopro.com/en/us/shop/cameras/hero11-black/CHDHX-111-master.html) (from 5 to 13)\n - [GoPro MAX](https://gopro.com/en/us/shop/cameras/max/CHDHZ-202-master.html)\n- [CAMM](https://developers.google.com/streetview/publish/camm-spec): an open-standard telemetry spec supported by a number of cameras\n - [Insta360 Pro2](https://www.insta360.com/product/insta360-pro2/)\n - [Insta360 Titan](https://www.insta360.com/product/insta360-titan)\n - [Ricoh Theta X](https://theta360.com/en/about/theta/x.html)\n - [Labpano](https://www.labpano.com/)\n - and more...\n- [BlackVue](https://blackvue.com/) videos\n - [DR900S-1CH](https://shop.blackvue.com/product/dr900x-1ch-plus/)\n - [DR900X Plus](https://shop.blackvue.com/product/dr900x-2ch-plus/)\n\n# Installation\n\n## Standalone Executable\n\n1. Download the latest executable for your platform from the [releases](https://github.com/mapillary/mapillary_tools/releases).\n2. Move the executable to your system `$PATH`\n\n> **_NOTE:_** If you see the error \"**mapillary_tools is damaged and can\u2019t be opened**\" on macOS, try to clear the extended attributes:\n>\n> ```\n> xattr -c mapillary_tools\n> ```\n\n## Installing via pip\n\nTo install or upgrade to the latest stable version:\n\n```sh\npip install --upgrade mapillary_tools\n```\n\nIf you can't wait for the latest features in development, install it from GitHub:\n\n```sh\npip install --upgrade git+https://github.com/mapillary/mapillary_tools\n```\n\n> **_NOTE:_** If you see \"**Permission Denied**\" error, try to run the command above with `sudo`, or install it in your\n> local [virtualenv](#setup) (recommended).\n\n### Installing on Android Devices\n\nA command line program such as Termux is required. Installation can be done without root privileges. The following\ncommands will install Python 3, pip3, git, and all required libraries for mapillary_tools on Termux:\n\n```sh\npkg install python git build-essential libgeos openssl libjpeg-turbo libexpat libexpat-static\npip install --upgrade pip wheel\npip install --upgrade mapillary_tools\n```\n\nTermux must access the device's internal storage to process and upload images. To do this, use the following command:\n\n```sh\ntermux-setup-storage\n```\n\nFinally, on devices running Android 11, using a command line program, mapillary_tools will process images very slowly if\nthey are in shared internal storage during processing. It is advisable to first move images to the command line\nprogram\u2019s native directory before running mapillary_tools. For an example using Termux, if imagery is stored in the\nfolder `Internal storage/DCIM/mapillaryimages` the following command will move that folder from shared storage to\nTermux:\n\n```sh\nmv -v storage/dcim/mapillaryimages mapillaryimages\n```\n\n# Usage\n\n## Process and Upload\n\nFor most users, `process_and_upload` is the command to go:\n\n```sh\n# Process and upload all images and videos in MY_CAPTURE_DIR and its subfolders, and all videos under MY_VIDEO_DIR\nmapillary_tools process_and_upload MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4\n```\n\nIf any process error occurs, e.g. GPS not found in an image, mapillary_tools will exit with non-zero status code.\nTo ignore these errors and continue uploading the rest:\n\n```sh\n# Skip process errors and upload to the specified user and organization\nmapillary_tools process_and_upload MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4 \\\n --skip_process_errors \\\n --user_name \"my_username\" \\\n --organization_key \"my_organization_id\"\n```\n\nThe `process_and_upload` command will run the [`process`](#process) and the [`upload`](#upload) commands consecutively with combined required and optional arguments.\nThe command above is equivalent to:\n\n```sh\nmapillary_tools process MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4 \\\n --skip_process_errors \\\n --desc_path /tmp/mapillary_description_file.json\n\nmapillary_tools upload MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4 \\\n --desc_path /tmp/mapillary_description_file.json \\\n --user_name \"my_username\" \\\n --organization_key \"my_organization_id\"\n```\n\n## Process\n\nThe `process` command is an intermediate step that extracts the metadata from images and videos,\nand writes them in an [image description file](#image-description). Users should pass it to the [`upload`](#upload) command.\n\n```sh\nmapillary_tools process MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4\n```\n\nDuplicate check with custom distance and angle:\n\n```sh\n# Mark images that are 3 meters closer to its previous one as duplicates.\n# Duplicates won't be uploaded\nmapillary_tools process MY_CAPTURE_DIR \\\n --duplicate_distance 3 \\\n --duplicate_angle 360 # Set 360 to disable angle check\n```\n\nSplit sequences with the custom cutoff distance or custom capture time gap:\n\n```sh\n# If two successive images are 100 meters apart,\n# OR their capture times are 120 seconds apart,\n# then split the sequence from there\nmapillary_tools process MY_CAPTURE_DIR \\\n --offset_angle 90 \\\n --cutoff_distance 100 \\\n --cutoff_time 120 \\\n```\n\n## Upload\n\nAfter processing you should get the [image description file](#image-description). Pass it to the `upload` command to upload them:\n\n```sh\n# Upload processed images and videos to the specified user account and organization\nmapillary_tools upload MY_CAPTURE_DIR \\\n --desc_path /tmp/mapillary_image_description.json \\\n --user_name \"my_username\" \\\n --organization_key \"my_organization_id\"\n```\n\n# Advanced Usage\n\n## Local Video Processing\n\nLocal video processing samples a video into a sequence of sample images and ensures the images are geotagged and ready for uploading.\nIt gives users more control over the sampling process, for example, you can specify the sampling distance to control the density.\nAlso, the sample images have smaller file sizes than videos, hence saving bandwidth.\n\n### Install FFmpeg\n\n[FFmpeg](https://ffmpeg.org/) is required for local video processing.\nYou can download `ffmpeg` and `ffprobe` from [here](https://ffmpeg.org/download.html),\nor install them with your favorite package manager.\n\n### Video Processing\n\nmapillary_tools first extracts the GPS track from the video's telemetry structure, and then locates video frames along the GPS track.\nWhen all are located, it then extracts one frame (image) every 3 meters by default.\n\n```sh\n# Sample videos in MY_VIDEO_DIR and write the sample images in MY_SAMPLES with a custom sampling distance\nmapillary_tools video_process MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5\n# The command above is equivalent to\nmapillary_tools sample_video MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5\nmapillary_tools process MY_SAMPLES\n```\n\nTo process and upload the sample images consecutively, run:\n\n```sh\nmapillary_tools video_process_and_upload MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5\n# The command above is equivalent to\nmapillary_tools video_process MY_VIDEO_DIR MY_SAMPLES --video_sample_distance 5 --desc_path=/tmp/mapillary_description.json\nmapillary_tools upload MY_SAMPLES --desc_path=/tmp/mapillary_description.json\n```\n\n## Geotagging with GPX\n\nIf you use external GPS devices for mapping, you will need to geotag your captures with the external GPS tracks.\n\nTo geotag images with a GPX file, the capture time (extracted from EXIF tag \"Date/Time Original\" or \"GPS Date/Time\") is minimally required.\nIt is used to locate the images along the GPS tracks.\n\n```sh\nmapillary_tools process MY_IMAGE_DIR --geotag_source \"gpx\" --geotag_source_path MY_EXTERNAL_GPS.gpx\n```\n\nTo geotag videos with a GPX file, video start time (video creation time minus video duration) is required to locate the sample images along the GPS tracks.\n\n```sh\n# Geotagging with GPX works with interval-based sampling only,\n# the two options --video_sample_distance -1 --video_sample_interval 2 are therefore required\n# to switch from the default distance-based sampling to the legacy interval-based sampling\nmapillary_tools video_process MY_VIDEO_DIR \\\n --geotag_source \"gpx\" \\\n --geotag_source_path MY_EXTERNAL_GPS.gpx \\\n --video_sample_distance -1 --video_sample_interval 2\n```\n\nIdeally, the GPS device and the capture device should use the same clock to get the timestamps synchronized.\nIf not, as is often the case, the image locations will be shifted. To solve that, mapillary_tools provides an\noption `--interpolation_offset_time N` that adds N seconds to image capture times for synchronizing the timestamps.\n\n```sh\n# The capture device's clock is 8 hours (i.e. -8 * 3600 = -28800 seconds) ahead of the GPS device's clock\nmapillary_tools process MY_IMAGE_DIR \\\n --geotag_source \"gpx\" \\\n --geotag_source_path MY_EXTERNAL_GPS.gpx \\\n --interpolation_offset_time -28800\n```\n\nAnother option `--interpolation_use_gpx_start_time` moves your images to align with the beginning of the GPS track.\nThis is useful when you can confirm that you start GPS recording and capturing at the same time, or with a known delay.\n\n```sh\n# Start capturing 2.5 seconds after start GPS recording\nmapillary_tools video_process MY_VIDEO_DIR \\\n --geotag_source \"gpx\" \\\n --geotag_source_path MY_EXTERNAL_GPS.gpx \\\n --interpolation_use_gpx_start_time \\\n --interpolation_offset_time 2.5 \\\n --video_sample_distance -1 --video_sample_interval 2\n```\n\n## New video geotagging features (experimental)\n\nAs experimental features, mapillary_tools can now:\n* Geotag videos from tracks recorded in GPX and NMEA files\n* Invoke `exiftool` internally, if available on the system. `exiftool` can extract geolocation data from a wide\nrange of video formats, allowing us to support more cameras with less trouble for the end user.\n* Try several geotagging sources sequentially, until proper data is found.\n\nThese features apply to the `process` command, which analyzes the video for direct upload instead of sampling it\ninto images. They are experimental and will be subject to change in future releases.\n\n### Usage\n\nThe new video processing is triggered with the `--video_geotag_source SOURCE` option. It can be specified multiple times:\nIn this case, each source will be tried in turn, until one returns good quality data.\n\n`SOURCE` can be:\n1. the plain name of the source - one of `video, camm, gopro, blackvue, gpx, nmea, exiftool_xml, exiftool_runtime`\n2. a JSON object that includes optional parameters\n```json5\n{\n \"source\": \"SOURCE\",\n \"pattern\": \"FILENAME_PATTERN\" // optional\n}\n```\n\nPATTERN specifies how the data source file is named, starting from the video filename:\n* `%f`: the full video filename\n* `%g`: the video filename without extension\n* `%e`: the video filename extension\n\nSupported sources and their default pattern are:\n\n* `video`: parse the video, in order, as `camm, gopro, blackvue`. Pattern: `%f`\n* `camm`: parse the video looking for a CAMM track. Pattern: `%f`\n* `gopro`: parse the video looking for geolocation in GoPro format. Pattern: `%f`\n* `blackvue`: parse the video looking for geolocation in BlackVue format. Pattern: `%f`\n* `gpx`: external GPX file. Pattern: `%g.gpx`\n* `nmea`: external NMEA file. Pattern: `%g.nmea`\n* `exiftool_xml`: external XML file generated by exiftool. Pattern: `%g.xml`\n* `exiftool_runtime`: execute exiftool on the video file. Pattern: `%f`\n\nNotes:\n* `exiftool_runtime` only works if exiftool is installed on the system. You can find it at https://exiftool.org/ or through\nyour software manager. If exiftool is installed, but is not in the default execution path, it is\npossible to specify its location by setting the environment variable `MAPILLARY_TOOLS_EXIFTOOL_PATH`.\n* Pattern are case-sensitive or not depending on the filesystem - in Windows, `%g.gpx` will match both `basename.gpx`\nand `basename.GPX`, in MacOs, Linux or other Unix systems no.\n* If both `--video_geotag_source` and `--geotag_source` are specified, `--video_geotag_source` will apply to video files\nand `--geotag_source` to image files.\n\n### Examples\n\n#### Generic supported videos\n\nProcess all videos in a directory, trying to parse them as CAMM, GoPro or BlackVue:\n```sh\nmapillary_tools process --video_geotag_source video VIDEO_DIR/\n```\n\n#### External GPX\n\nProcess all videos in a directory, taking geolocation data from GPX files. A video named `foo.mp4` will be associated\nwith a GPX file called `foo.gpx`.\n```sh\nmapillary_tools process --video_geotag_source gpx VIDEO_DIR/\n```\n\n#### Insta360 stitched videos\n\nThe videos to process have been stitched by Insta360 Studio; the geolocation data is in the original\nvideos in the parent directory, and there may be GPX files alongside the stitched video.\nFirst look for GPX, then fallback to running exiftool against the original videos.\n```sh\nmapillary_tools process \\\n--video_geotag_source gpx \\\n--video_geotag_source '{\"source\": \"exiftool_runtime\", \"pattern\": \"../%g.insv\"}' \\\nVIDEO_DIR/\n```\n\n### Limitations of `--video_geotag_source`\n\n**External geolocation sources will be aligned with the start of video, there is\ncurrently no way of applying offsets or scaling the time.** This means, for instance, that GPX tracks must begin precisely\nat the instant of the video, and that timelapse videos are supported only for sources `camm, gopro, blackvue`.\n\n\n## Authenticate\n\nThe command `authenticate` will update the user credentials stored in the config file.\n\n### Examples\n\nAuthenticate new user:\n\n```sh\nmapillary_tools authenticate\n```\n\nAuthenticate for user `my_username`. If the user is already authenticated, it will update the credentials in the config:\n\n```sh\nmapillary_tools authenticate --user_name \"my_username\"\n```\n\n## Image Description\n\nThe output of the [`process`](#process) command is a JSON array of objects that describes metadata for each image or video.\nThe metadata is validated by the [image description schema](https://github.com/mapillary/mapillary_tools/blob/main/schema/image_description_schema.json).\nHere is a minimal example:\n\n```json\n[\n {\n \"MAPLatitude\": 58.5927694,\n \"MAPLongitude\": 16.1840944,\n \"MAPCaptureTime\": \"2021_02_13_13_24_41_140\",\n \"filename\": \"/MY_IMAGE_DIR/IMG_0291.jpg\"\n },\n {\n \"error\": {\n \"type\": \"MapillaryGeoTaggingError\",\n \"message\": \"Unable to extract GPS Longitude or GPS Latitude from the image\"\n },\n \"filename\": \"/MY_IMAGE_DIR/IMG_0292.jpg\"\n }\n]\n```\n\nUsers may create or manipulate the image description file before passing them to the [`upload`](#upload) command. Here are a few examples:\n\n```sh\n# Remove images outside the bounding box and map matching the rest images on the road network\nmapillary_tools process MY_IMAGE_DIR | \\\n ./filter_by_bbox.py 5.9559,45.818,10.4921,47.8084 | \\\n ./map_match.py > /tmp/mapillary_image_description.json\n\n# Upload the processed images\nmapillary_tools upload MY_IMAGE_DIR --desc_path /tmp/mapillary_image_description.json\n```\n\n```sh\n# Converts captures.csv to an image description file\n./custom_csv_to_description.sh captures.csv | mapillary_tools upload MY_IMAGE_DIR --desc_path -\n```\n\n## Zip Images\n\nWhen [uploading](#upload) an image directory, internally the `upload` command will zip sequences in the temporary\ndirectory (`TMPDIR`) and then upload these zip files.\n\nmapillary_tools provides `zip` command that allows users to specify where to store the zip files, usually somewhere with\nfaster IO or more free space.\n\n```sh\n# Zip processed images in MY_IMAGE_DIR and write zip files in MY_ZIPFILES\nmapillary_tools zip MY_IMAGE_DIR MY_ZIPFILES\n\n# Upload all the zip files (*.zip) in MY_ZIPFILES:\nmapillary_tools upload --file_types zip MY_ZIPFILES\n```\n\n# Development\n\n## Setup\n\nClone the repository:\n\n```sh\ngit clone git@github.com:mapillary/mapillary_tools.git\ncd mapillary_tools\n```\n\n### Option 1: Using uv (Recommended)\n\nUse [uv](https://docs.astral.sh/uv/) - a fast Python package manager.\n\nInstall the project in development mode with all dependencies:\n\n```sh\n# Install the project and development dependencies\nuv sync --group dev\n\n# Activate the virtual environment\nsource .venv/bin/activate # On Windows: .venv\\Scripts\\activate\n```\n\n### Option 2: Using pip with virtual environment\n\nSet up a virtual environment (recommended):\n\n```sh\npython -m venv .venv\nsource .venv/bin/activate # On Windows: .venv\\Scripts\\activate\n```\n\nInstall the project in development mode:\n\n```sh\n# Install the project and all dependencies in editable mode\npip install -e .\n\n# Install development dependencies\npip install --group dev\n```\n\n## Running the code\n\nRun the code from the repository:\n\n```sh\n# If you have mapillary_tools installed in editable mode\nmapillary_tools --version\n\n# Alternatively\npython -m mapillary_tools.commands --version\n```\n\n## Tests\n\nRun tests:\n\n```sh\n# Test all cases\npytest -s -vv tests\n# Or test a single case specifically\npytest -s -vv tests/unit/test_camm_parser.py::test_build_and_parse\n```\n\n## Code Quality\n\nRun code formatting and linting:\n\n```sh\n# Format code with ruff\nruff format mapillary_tools tests\n\n# Lint code with ruff\nruff check mapillary_tools tests\n\n# Sort imports with usort\nusort format mapillary_tools tests\n\n# Type checking with mypy\nmypy mapillary_tools\n```\n\n## Release and Build\n\n```sh\n# Assume you are releasing v0.9.1a2 (alpha2)\n\n# Tag your local branch\n# Use -f here to replace the existing one\ngit tag -f v0.9.1a2\n\n# Push the tagged commit first if it is not there yet\ngit push origin\n\n# Push ALL local tags (TODO: How to push a specific tag?)\n# Use -f here to replace the existing tags in the remote repo\ngit push origin --tags -f\n\n# The last step will trigger CI to publish a draft release with binaries built\n# in https://github.com/mapillary/mapillary_tools/releases\n```\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "Mapillary Image/Video Import Pipeline",
"version": "0.14.0",
"project_urls": {
"Homepage": "https://github.com/mapillary/mapillary_tools",
"Issues": "https://github.com/mapillary/mapillary_tools/issues",
"Repository": "https://github.com/mapillary/mapillary_tools"
},
"split_keywords": [
"mapillary",
" gis",
" computer vision",
" street view"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "a9dfbeae11a5d2b0a83840bf8ced11ab574cd304f625858111b63a5b5ae54ab1",
"md5": "e724acd99ed3e7d1c533250b96e8eb28",
"sha256": "997445da31e9bcbbbec947b806b2b284981d2661452a74de7d1ff64665d2e6a3"
},
"downloads": -1,
"filename": "mapillary_tools-0.14.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e724acd99ed3e7d1c533250b96e8eb28",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 154453,
"upload_time": "2025-08-06T18:53:58",
"upload_time_iso_8601": "2025-08-06T18:53:58.116485Z",
"url": "https://files.pythonhosted.org/packages/a9/df/beae11a5d2b0a83840bf8ced11ab574cd304f625858111b63a5b5ae54ab1/mapillary_tools-0.14.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "59f1988830d31a2bf182322996b38bffbee21d9dd5fa5ee60d4a6062e90ee0e6",
"md5": "1a6f5d34b2c7496bd83ab8828aac7686",
"sha256": "a527cf03bd696f5a26e15f984ad6f5421065e45ddd823dc34cb0729bd20b0f20"
},
"downloads": -1,
"filename": "mapillary_tools-0.14.0.tar.gz",
"has_sig": false,
"md5_digest": "1a6f5d34b2c7496bd83ab8828aac7686",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 130019,
"upload_time": "2025-08-06T18:53:59",
"upload_time_iso_8601": "2025-08-06T18:53:59.520251Z",
"url": "https://files.pythonhosted.org/packages/59/f1/988830d31a2bf182322996b38bffbee21d9dd5fa5ee60d4a6062e90ee0e6/mapillary_tools-0.14.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-06 18:53:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mapillary",
"github_project": "mapillary_tools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mapillary-tools"
}
JSON
