# restapi
This is a Python API for working with ArcGIS REST API, ArcGIS Online, and Portal/ArcGIS Enterprise. This package has been designed to work with [arcpy](https://pro.arcgis.com/en/pro-app/arcpy/get-started/what-is-arcpy-.htm) when available, or the included open source module [pyshp](https://pypi.org/project/pyshp/). It will try to use arcpy if available for some data conversions, otherwise will use open source options. Also included is a subpackage for administering ArcGIS Server Sites. This is updated often, so continue checking here for new functionality!
### Why would you use this package?
Esri currently provides the [ArcGIS API for Python](https://developers.arcgis.com/python/) which provides complete bindings to the ArcGIS REST API. This package has less coverage of the REST API, but has many convience functions not available in the ArcGIS API for Python and has a strong focus on downloading and querying data. This package will also support older versions of Python (i.e. 2.7.x) whereas Esri's package only supports 3.x.
## Release History
[Release History](ReleaseNotes.md)
## Installation
`restapi` is supported on Python 2.7 and 3.x. It can be found on [Github](https://github.com/Bolton-and-Menk-GIS/restapi) and [PyPi](https://pypi.org/project/bmi-arcgis-restapi/). To install using pip:
````sh
pip install bmi-arcgis-restapi
````
After installation, it should be available to use in Python:
````py
import restapi
````
## A note about `arcpy`
By default, `restapi` will import Esri's `arcpy` module if available. However, this module is not required to use this package. `arcpy` is only used when available to write data to disk in esri specific formats (file geodatabase, etc) and working with `arcpy` Geometries. When `arcpy` is not availalbe, the [pyshp](https://pypi.org/project/pyshp/) module is used to write data (shapefile format only) and work with `shapefile.Shape` objects (geometry). Also worth noting is that open source version is much faster than using `arcpy`.
That being said, there may be times when you want to force `restapi` to use the open source version, even when you have access to `arcpy`. Some example scenarios being when you don't need to write any data in an Esri specific format, you want the script to execute very fast, or you are working in an environment where `arcpy` may not play very nicely ([Flask](https://palletsprojects.com/p/flask/), [Django](https://www.djangoproject.com/), etc.). To force `restapi` to use the open source version, you can simply create an environment variable called `RESTAPI_USE_ARCPY` and set it to `FALSE` or `0`. This variable will be checked before attempting to import `arcpy`.
Here is an example on how to force open source at runtime:
```py
import os
os.environ['RESTAPI_USE_ARCPY'] = 'FALSE'
# now import restapi
import restapi
```
## requests.exceptions.SSLError
If you are seeing `requests.exceptions.SSLError` exceptions in `restapi` >= 2.0, this is probaly due to a change in handling servers without valid SSL certificates. Because many ArcGIS Server instances are accessed using SSL with a self-signed certificate, or through a MITM proxy like Fiddler, `restapi` < 2.0 defaulted to ignoring SSL errors by setting the `request` client's `verify` option to `False`. The new default behavior is to enable certificate verification. If you are receiving this error, you are probably accessing your server with a self-signed certificate, or through a MITM proxy. If that is not the case, you should investigate why you are seeing SSL errors, as there would likely be an issue with the server configuration, or some other security issues.
To mimic the previous behavior in the newer versions of `restapi`, there are 2 options - disable certificate verification (less secure), or [build a custom CA bundle ](https://requests.readthedocs.io/en/stable/user/advanced/#ssl-cert-verification) which includes any self-signed certificates needed to access your server (more secure). Both of these can be done using the new [restapi.RequestClient()](#requestclient) feature.
````py
import restapi
import requests
session = requests.Session()
client = restapi.RequestClient(session)
restapi.set_request_client(client)
# Disable verification
client.session.verify = False
# -or-
# Use custom CA bundle
client.session.verify = '/path/to/certfile'
````
Since `verify = False` is a commonly used setting when dealing with ArcGIS Server instances, it's also possible to use an environment variable. The variable must be set before `restapi` is imported.
````py
os.environ['RESTAPI_VERIFY_CERT'] = 'FALSE'
import restapi
````
## Connecting to an ArcGIS Server
> samples for this section can be found in the [connecting_to_arcgis_server.py](./restapi/samples/connecting_to_arcgis_server.py) file.
One of the first things you might do is to connect to a services directory (or catalog):
````py
import restapi
# connect to esri's sample server 6
rest_url = 'https://sampleserver6.arcgisonline.com/arcgis/rest/services'
# connect to restapi.ArcServer instance
ags = restapi.ArcServer(rest_url)
````
```py
>>> # get folder and service properties
>>> print('Number of folders: {}'.format(len(ags.folders)))
>>> print('Number of services: {}\n'.format(len(ags.services)))
>>>
>>> # walk thru directories
>>> for root, services in ags.walk():
>>> print('Folder: {}'.format(root))
>>> print('Services: {}\n'.format(services))
Number of folders: 13
Number of services: 60
Folder: None
Services: ['911CallsHotspot/GPServer', '911CallsHotspot/MapServer', 'Census/MapServer', 'CharlotteLAS/ImageServer', 'CommercialDamageAssessment/FeatureServer', 'CommercialDamageAssessment/MapServer', 'CommunityAddressing/FeatureServer', 'CommunityAddressing/MapServer', '<...>', 'Water_Network/MapServer', 'Wildfire/FeatureServer', 'Wildfire/MapServer', 'WindTurbines/MapServer', 'World_Street_Map/MapServer', 'WorldTimeZones/MapServer']
Folder: AGP
Services: ['AGP/Census/MapServer', 'AGP/Hurricanes/MapServer', 'AGP/USA/MapServer', 'AGP/WindTurbines/MapServer']
Folder: Elevation
Services: ['Elevation/earthquakedemoelevation/ImageServer', 'Elevation/ESRI_Elevation_World/GPServer', 'Elevation/GlacierBay/MapServer', 'Elevation/MtBaldy_Elevation/ImageServer', 'Elevation/WorldElevations/MapServer']
# ..etc
```
#### Connecting to a child service
from an `ArcServer` instance, you can connect to any service within that instance by providing a child path or wildcard search match.
````py
# connect to a specific service
# using just the service name (at the root)
usa = ags.getService('USA') #/USA/MapServer -> restapi.common_types.MapService
# using the relative path to a service in a folder
census = ags.getService('AGP/Census') #/AGP/Census/MapServer -> restapi.common_types.MapService
# can also just use the service name, but be weary of possible duplicates
infastructure = ags.getService('Infrastructure') #/Energy/Infrastructure/FeatureServer -> restapi.common_types.FeatureService
# using a wildcard search
covid_cases = ags.getService('*Covid19Cases*') #/NYTimes_Covid19Cases_USCounties/MapServer -> restapi.common_types.MapService
````
check outputs:
```py
>>> for service in [usa, census, infastructure, covid_cases]:
>>> print('name: "{}"'.format(service.name))
>>> print('repr: "{}"'.format(repr(service)))
>>> print('url: {}\n'.format(service.url))
name: "USA"
repr: "<MapService: USA/MapServer>"
url: https://sampleserver6.arcgisonline.com/arcgis/rest/services/USA/MapServer
name: "Census"
repr: "<MapService: AGP/Census/MapServer>"
url: https://sampleserver6.arcgisonline.com/arcgis/rest/services/AGP/Census/MapServer
name: "Infrastructure"
repr: "<FeatureService: Energy/Infrastructure/FeatureServer>"
url: https://sampleserver6.arcgisonline.com/arcgis/rest/services/Energy/Infrastructure/FeatureServer
name: "NYTimes_Covid19Cases_USCounties"
repr: "<MapService: NYTimes_Covid19Cases_USCounties/MapServer>"
url: https://sampleserver6.arcgisonline.com/arcgis/rest/services/NYTimes_Covid19Cases_USCounties/MapServer
```
### Working with layers
> samples for this section can be found in the [working_with_layers.py](./restapi/samples/working_with_layers.py) file.
You can connect to `MapServiceLayer`s or `FeatureLayer`s directly by passing the url, or by accessing from the parent `FeatureService` or `MapService`.
also query the layer and get back arcpy.da Cursor like access
```py
# get access to the "Cities" layer from USA Map Service
cities = usa.layer('Cities') # or can use layer id: usa.layer(0)
# query the map layer for all cities in California with population > 100000
where = "st = 'CA' and pop2000 > 100000"
# the query operation returns a restapi.FeatureSet or restapi.FeatureCollection depending on the return format
featureSet = cities.query(where=where)
# get result count, can also use len(featureSet)
print('Found {} cities in California with Population > 100K'.format(featureSet.count))
# print first feature (restapi.Feature). The __str__ method is pretty printed JSON
# can use an index to fetch via its __getitem__ method
print(featureSet[0])
```
#### exceeding the `maxRecordCount`
In many cases, you may run into issues due to the `maxRecordCount` set for a service, which by default is `1000` features. This limit can be exceeded by using the `exceed_limit` parameter. If `exceed_limit` is set to `True`, the `query_in_chunks` method will be called internally to keep fetching until all queried features are returned.
```py
# fetch first 1000
first1000 = cities.query()
print('count without exceed_limit: {}'.format(first1000.count)) # can also use len()
# fetch all by exceeding limit
allCities = cities.query(exceed_limit=True)
print('count without exceed_limit: {}'.format(len(allCities)))
```
check outputs:
```py
Found 57 cities in California with Population > 100K
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [
-117.88976896399998,
33.836165033999976
]
},
"properties": {
"areaname": "Anaheim"
}
}
count without exceed_limit: 1000
total records: 3557
count with exceed_limit: 3557
```
#### using a search cursor
A query can also be done by using a search cursor, which behaves very similarly to the [arcpy.da.SearchCursor](https://pro.arcgis.com/en/pro-app/latest/arcpy/data-access/searchcursor-class.htm). While the `restapi.Cursor` does support usage of a `with` statement, it is not necessary as there is no file on disk that is opened. When using the `cursor()` method on a `MapServiceLayer` or `FeatureLayer` it will actually make a call to the ArcGIS Server and will return a tuple of values based on the `fields` requested. The default `fields` value is `*`, which will return all fields. Like the arcpy cursors, the `SHAPE@` token can be used to fetch geometry, while the `OID@` token will fetch the object id.
```py
# if you don't want the json/FeatureSet representation, you can use restapi cursors
# for the query which are similar to the arcpy.da cursors. The `SearchCursor` will return a tuple
cursor = restapi.SearchCursor(cities, fields=['areaname', 'pop2000', 'SHAPE@'], where=where)
for row in cursor:
print(row)
````
check outputs:
```py
('Anaheim', 328014, <restapi.shapefile.Shape object at 0x000002B1DE232400>)
('Bakersfield', 247057, <restapi.shapefile.Shape object at 0x000002B1DE232470>)
('Berkeley', 102743, <restapi.shapefile.Shape object at 0x000002B1DE232400>)
('Burbank', 100316, <restapi.shapefile.Shape object at 0x000002B1DE232470>)
('Chula Vista', 173556, <restapi.shapefile.Shape object at 0x000002B1DE232400>)
('Concord', 121780, <restapi.shapefile.Shape object at 0x000002B1DE232470>)
('Corona', 124966, <restapi.shapefile.Shape object at 0x000002B1DE232400>)
# ... etc
```
also supports `with` statements:
```py
with restapi.SearchCursor(cities, fields=['areaname', 'pop2000', 'SHAPE@'], where=where) as cursor:
for row in cursor:
print(row)
```
In the above examples, when the `SearchCursor` is instantiated, it actually kicked off a new query for the cursor results. If you already have a `FeatureSet` or `FeatureCollection` like we did above, you can also save a network call by constructing the `restapi.Cursor` from the feature set by doing:
```py
featureSet = cities.query(where=where)
# can pass in fields as second argument to limit results
cursor = restapi.Cursor(featureSet, ['areaname', 'pop2000', 'SHAPE@'])
```
> note: Both the `MapServiceLayer` and `FeatureLayer` also have a `searchCursor()` convenience method, which will allow you to use the same functionality as shown above but without passing in the layer itself:
> `rows = cities.searchCursor(['areaname', 'pop2000', 'SHAPE@'])`
#### exporting features from a layer
Data can also be easily exported from a `FeatureLayer` or `MapServicelayer`. This can be done by exporting a feature set directly or from the layer itself:
```py
# set output folder
out_folder = os.path.join(os.path.expanduser('~'), 'Documents', 'restapi_samples')
# create output folder if it doesn't exist
if not os.path.exists(out_folder):
os.makedirs(out_folder)
# output shapefile
shp = os.path.join(out_folder, 'CA_Cities_100K.shp')
# export layer from map service
cities.export_layer(shp, where=where)
```
You can also export a feature set directly if you already have one loaded:
```py
restapi.exportFeatureSet(featureSet, shp)
```
exporting layers or feature sets also supports a `fields` filter, where you can limit which fields are exported. Other options are supported as well such as an output spatial reference. One important thing to note, if you do not have access to `arcpy`, you can only export to shapefile format. If you do have `arcpy`, any output format supported by esri will work. If the output is a [geodatabase feature class](https://desktop.arcgis.com/en/arcmap/latest/manage-data/feature-classes/a-quick-tour-of-feature-classes.htm), you can also choose to include things like [domains](https://desktop.arcgis.com/en/arcmap/latest/manage-data/geodatabases/an-overview-of-attribute-domains.htm) if applicable.
```py
# exporting features
out_folder = os.path.join(os.path.expanduser('~'), 'Documents', 'restapi_samples')
if not os.path.exists(out_folder):
os.makedirs(out_folder)
shp = os.path.join(out_folder, 'CA_Cities_100K.shp')
# export layer to a shapefile
cities.export_layer(shp, where=where)
# if there is an existing feature set, you can also export that directly
# restapi.exportFeatureSet(featureSet, shp)
```
### selecting features by geometry
Both the `MapServiceLayer` and `FeatureLayer` support arcgis spatial selections using [arcgis geometry](https://developers.arcgis.com/documentation/common-data-types/geometry-objects.htm) objects. The `select_layer_by_location` method will return a `FeatureSet` or `FeatureCollection`, while the `clip` method will actually export the spatial selection to a shapffile or Feature Class (latter only available with `arcpy`).
#### the `geometry-helper` application
This package now includes a [geometry-helper](geometry-helper/README.md) application that can be used to quickly create geometries and copy the geometry in either `esri-json` or `geojson` format to form geometry necessary to make queries:
to open this application in a web browser, simply call:
```py
restapi.open_geometry_helper()
```
#### select by location
to select a layer by location, first create a geometry object as json (can be esri or geojson format) and make the selection:
```py
# select layer by location
universities_url = 'https://services1.arcgis.com/Hp6G80Pky0om7QvQ/arcgis/rest/services/Colleges_and_Universities/FeatureServer/0'
# get feature layer
universities = restapi.FeatureLayer(universities_url)
print('universities: ', repr(universities))
# form geometry (do not have to cast to restapi.Geometry, this will happen under the hood automatically)
geometry = restapi.Geometry({
"spatialReference": {
"latestWkid": 3857,
"wkid": 102100
},
"rings": [
[
[
-10423340.4579098,
5654465.8453829475
],
[
-10324889.565478457,
5654465.8453829475
],
[
-10324889.565478457,
5584449.527473665
],
[
-10423340.4579098,
5584449.527473665
],
[
-10423340.4579098,
5654465.8453829475
]
]
]
})
# make selection
featureCollection = universities.select_by_location(geometry)
print('Number of Universities in Twin Cities area: {}'.format(featureCollection.count))
# can also export the feature collection directly or call the clip() method (makes a new call to server)
universities_shp = os.path.join(out_folder, 'TwinCities_Univiersities.shp')
universities.clip(geometry, universities_shp)
```
check outputs:
```py
universities: <FeatureLayer: "CollegesUniversities" (id: 0)>
total records: 50
Number of Universities in Twin Cities area: 50
total records: 50
Created: "L:\Users\calebma\Documents\restapi_samples\TwinCities_Univiersities.shp"
Fetched all records
```
## FeatureLayer Editing
> samples for this section can be found in the [feature_layer_editing.py](./restapi/samples/feature_layer_editing.py) file.
This package also includes comprehensive support for working with [FeatureLayers](https://enterprise.arcgis.com/en/portal/latest/use/feature-layers.htm). These are editable layers hosted by an ArcGIS Server, Portal, or ArcGIS Online. A `FeatureLayer` is an extension of the `MapServiceLayer` that supports editing options.
#### add features using `FeatureLayer.addFeatures()`
```py
# connect to esri's Charlotte Hazards Sample Feature Layer for incidents
url = 'https://services.arcgis.com/V6ZHFr6zdgNZuVG0/ArcGIS/rest/services/Hazards_Uptown_Charlotte/FeatureServer/0'
# instantiate a FeatureLayer
hazards = restapi.FeatureLayer(url)
# create a new feature as json
feature = {
"attributes" : {
"HazardType" : "Road Not Passable",
"Description" : "restapi test",
"SpecialInstructions" : "Contact Dispatch",
"Priority" : "High",
"Status": "Active"
},
"geometry": create_random_coordinates()
}
# add json feature
adds = hazards.addFeatures([ feature ])
print(adds)
```
check outputs:
```py
Added 1 feature(s)
{
"addResults": [
{
"globalId": "8B643546-7253-4F0B-80DE-2DE8F1754C03",
"objectId": 11858,
"success": true,
"uniqueId": 11858
}
]
}
```
#### add an attachment
```py
# add attachment to new feature using the OBJECTID of the new feature
oid = adds.addResults[0].objectId
image = os.path.join(os.path.abspath('...'), 'docs', 'images', 'geometry-helper.png')
attRes = hazards.addAttachment(oid, image)
print(attRes)
# now query attachments for new feature
attachments = hazards.attachments(oid)
print(attachments)
```
check outputs:
```py
Added attachment '211' for feature objectId
{
"addAttachmentResult": {
"globalId": "f370a4e0-3976-4809-b0d5-f5d1b5990b4b",
"objectId": 211,
"success": true
}
}
[<Attachment ID: 211 (geometry-helper.png)>]
```
#### update features
```py
# update the feature we just added, payload only has to include OBJECTID and any fields to update
updatePayload = [
{
"attributes": {
"OBJECTID": r.objectId,
"Description": "restapi update"
}
} for r in adds.addResults
]
updates = hazards.updateFeatures(updatePayload)
print(updates)
```
check outputs:
```py
Updated 1 feature(s)
{
"updateResults": [
{
"globalId": null,
"objectId": 11858,
"success": true,
"uniqueId": 11858
}
]
}
```
#### delete features
```py
# delete features by passing in a list of objectIds
deletePayload = [r.objectId for r in updates.updateResults]
deletes = hazards.deleteFeatures(deletePayload)
print(deletes)
```
check outputs:
```py
Deleted 1 feature(s)
{
"deleteResults": [
{
"globalId": null,
"objectId": 11858,
"success": true,
"uniqueId": 11858
}
]
}
```
### feature editing with `restapi` cursors
`restapi` also supports cursors similar to what you get when using `arcpy`. However, these work directly with the REST API and JSON features while also supporting `arcpy` and `shapefile` geometry types. See the below example on how to use an `insertCursor` to add new records:
#### adding features with an `InsertCursor`
New records can be added using an `InsertCursor`. This can be instantiated using the `InsertCursor` class itself, or by calling the `insertCursor` method of a `FeatureLayer`. When called as `FeatureLayer.insertCursor`, the layer itself will not need to be passed in.
```py
>>> # add 5 new features using an insert cursor
>>> # using this in a "with" statement will call applyEdits on __exit__
>>> fields = ["SHAPE@", 'HazardType', "Description", "Priority"]
>>> with restapi.InsertCursor(hazards, fields) as irows:
for i in list(range(1,6)):
desc = "restapi insert cursor feature {}".format(i)
irows.insertRow([create_random_coordinates(), "Wire Down", desc, "High"])
Added 5 feature(s)
```
Any time an insert or update cursor will save changes, it will print a short message showing how many features were affected. You can always get at the raw edit information from the `FeatureLayer` by calling the `editResults` property. This will be an array that stores the results of every `applyEdits` operation, so the length will reflect how many times edits have been saved.
```py
>>> # we can always view the results by calling FeatureLayer.editResults which stores
>>> # an array of edit results for each time applyEdits() is called.
>>> print(hazards.editResults)
[{
"addResults": [
{
"globalId": "2774C9ED-D8F8-4B71-9F37-B26B40790710",
"objectId": 12093,
"success": true,
"uniqueId": 12093
},
{
"globalId": "18411060-F6FF-49FB-AD91-54DB65C13D06",
"objectId": 12094,
"success": true,
"uniqueId": 12094
},
{
"globalId": "8045C840-F1B9-4BD2-AEDC-72F4D65EB7A6",
"objectId": 12095,
"success": true,
"uniqueId": 12095
},
{
"globalId": "EC9DB6FC-0D34-4B83-8C98-398C7B48666D",
"objectId": 12096,
"success": true,
"uniqueId": 12096
},
{
"globalId": "C709033F-DF3B-43B3-8148-2299E7CEE986",
"objectId": 12097,
"success": true,
"uniqueId": 12097
}
],
"deleteResults": [],
"updateResults": []
}]
```
> note: when using a `with` statement for the `InsertCursor` and `UpdateCursor` it will automatically call the `applyEdits()` method on `__exit__`, which is critical to submitting the new, deleted, or updated features to the server. If not using a `with` statement, you will need to call `applyEdits()` manually after changes have been made.
#### updating features with an `UpdateCursor`
records can be updated or deleted with an `updateCursor` and a where clause. Note that the `OBJECTID` field must be included in the query to indicate which records will be updated. The `OID@` field token can be used to retreive the `objectIdFieldName`:
```py
>>> # now update records with updateCursor for the records we just added. Can use the
>>> # editResults property of the feature layer to get the oids of our added features
>>> addedOids = ','.join(map(str, [r.objectId for r in hazards.editResults[0].addResults]))
>>> whereClause = "{} in ({})".format(hazards.OIDFieldName, addedOids)
>>> with restapi.UpdateCursor(hazards, ["Priority", "Description", "OID@"], where=whereClause) as rows:
for row in rows:
if not row[2] % 2:
# update rows with even OBJECTID's
row[0] = "Low"
rows.updateRow(row)
else:
# delete odd OBJECTID rows
print('deleting row with odd objectid: ', row[2])
rows.deleteRow(row)
Updated 2 feature(s)
Deleted 3 feature(s)
>>> # now delete the rest of the records we added
>>> whereClause = "Description like 'restapi%'"
>>> with restapi.UpdateCursor(hazards, ["Description", "Priority", "OID@"], where=whereClause) as rows:
for row in rows:
rows.deleteRow(row)
irows.insertRow([create_random_coordinates(), "Wire Down", desc, "High"])
Deleted 2 feature(s)
```
#### attachment editing
Offline capabilities (Sync)
````py
# if sync were enabled, we could create a replica like this:
# can pass in layer ID (0) or name ('incidents', not case sensative)
replica = fs.createReplica(0, 'test_replica', geometry=adds[0]['geometry'], geometryType='esriGeometryPoint', inSR=4326)
# now export the replica object to file geodatabase (if arcpy access) or shapefile with hyperlinks (if open source)
restapi.exportReplica(replica, folder)
````
Working with Image Services
---------------------------
````py
url = 'http://pca-gis02.pca.state.mn.us/arcgis/rest/services/Elevation/DEM_1m/ImageServer'
im = restapi.ImageService(url)
# clip DEM
geometry = {"rings":[[
[240006.00808044084, 4954874.19629429],
[240157.31010183255, 4954868.8053006204],
[240154.85966611796, 4954800.0316874133],
[240003.55764305394, 4954805.4226145679],
[240006.00808044084, 4954874.19629429]]],
"spatialReference":{"wkid":26915,"latestWkid":26915}}
tif = os.path.join(folder, 'dem.tif')
im.clip(geometry, tif)
# test point identify
x, y = 400994.780878, 157878.398217
elevation = im.pointIdentify(x=x, y=y, sr=103793)
print(elevation)
````
Geocoding
---------
````py
# hennepin county, MN geocoder
henn = 'http://gis.hennepin.us/arcgis/rest/services/Locators/HC_COMPOSITE/GeocodeServer'
geocoder = restapi.Geocoder(henn)
# find target field, use the SingleLine address field by default
geoResult = geocoder.findAddressCandidates('353 N 5th St, Minneapolis, MN 55403')
# export results to shapefile
print('found {} candidates'.format(len(geoResult))
geocoder.exportResults(geoResult, os.path.join(folder, 'target_field.shp'))
# Esri geocoder
esri_url = 'http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Locators/ESRI_Geocode_USA/GeocodeServer'
esri_geocoder = restapi.Geocoder(esri_url)
# find candidates using key word arguments (**kwargs) to fill in locator fields, no single line option
candidates = esri_geocoder.findAddressCandidates(Address='380 New York Street', City='Redlands', State='CA', Zip='92373')
print('Number of address candidates: {}'.format(len(candidates)))
for candidate in candidates:
print(candidate.location)
# export results to shapefile
out_shp = os.path.join(folder, 'Esri_headquarters.shp')
geocoder.exportResults(candidates, out_shp)
````
Geoprocessing Services
----------------------
````py
# test esri's drive time analysis GP Task
gp_url = 'http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Network/ESRI_DriveTime_US/GPServer/CreateDriveTimePolygons'
gp = restapi.GPTask(gp_url)
# get a list of gp parameters (so we know what to pass in as kwargs)
print('\nGP Task "{}" parameters:\n'.format(gp.name)
for p in gp.parameters:
print('\t', p.name, p.dataType)
point = {"geometryType":"esriGeometryPoint",
"features":[
{"geometry":{"x":-10603050.16225853,"y":4715351.1473399615,
"spatialReference":{"wkid":102100,"latestWkid":3857}}}],
"sr":{"wkid":102100,"latestWkid":3857}}
# run task, passing in gp parameters as keyword arguments (**kwargs)
gp_res = gp.run(Input_Location=str(point), Drive_Times = '1 2 3', inSR = 102100)
# returns a GPResult() object, can get at the first result by indexing (usually only one result)
# can test if there are results by __nonzero__()
if gp_res:
result = gp_res.results[0]
# this returned a GPFeatureRecordSetLayer as an outputParameter, so we can export this to polygons
print('\nOutput Result: "{}", data type: {}\n'.format(result.paramName, result.dataType))
# now export the result value to fc (use the value property of the GPResult object from run())
drive_times = os.path.join(folder, 'drive_times.shp')
restapi.exportFeatureSet(drive_times, gp_res.value)
````
A note about input Geometries
-----------------------------
restapi will try to use arcpy first if you have it, otherwise will defer to open source. Both
support the reading of shapefiles to return the first feature back as a restapi.Geometry object
It also supports arcpy Geometries and shapefile.Shape() objects
````py
>>> shp = r'C:\TEMP\Polygons.shp' # a shapefile on disk somewhere
>>> geom = restapi.Geometry(shp)
>>> print(geom.envelope())
-121.5,38.3000000007,-121.199999999,38.6000000015
````
Token Based Security
--------------------
restapi also supports secured services. This is also session based, so if you sign in once to an
ArcGIS Server Resource (on the same ArcGIS Site), the token will automatically persist via the
IdentityManager().
There are 3 ways to authticate:
````py
# **kwargs for all accessing all ArcGIS resources are
# usr -- username
# pw -- password
# token -- token (as string or restapi.Token object)
# proxy -- url to proxy
# secured url
secured_url = 'http://some-domain.com/arcgis/rest/services'
# 1. username and password
ags = restapi.ArcServer(url, 'username', 'password') # token is generated and persists
# 2. a token that has already been requested
ags = restapi.ArcServer(url, token=token) # uses a token that is already active
# 3. via a proxy (assuming using the standard esri proxy)
# this will forward all subsequent requests through the proxy
ags = restapi.ArcServer(url, proxy='http://some-domain.com/proxy.ashx')
````
You can even just generate a token and let the IdentityManager handle the rest. It is even smart enough to handle multiple tokens for different sites:
```py
# login to instance 1
usr = 'username'
pw = 'password'
# urls to two different ArcGIS Server sites
url_1 = 'http://some-domain.com/arcserver1/rest/services'
url_2 = 'http://domain2.com/arcgis/rest/services'
# generate tokens
tok1 = restapi.generate_token(url_1, usr, pw)
tok2 = restapi.generate_token(url_2, usr, pw)
# now we should be able to access both ArcGIS Server sites via the IdentityManager
arcserver1 = restapi.ArcServer(url_1) # tok1 is automatically passed in and handled
arcserver2 = restapi.ArcServer(url_2) # tok2 is used here
```
The admin Subpackage
--------------------
restapi also contains an administrative subpackage (warning: most functionality has not been tested!). You can import this module like this:
```py
from restapi import admin
```
### Connecting to a Portal
```py
url = 'https://domain.gis.com/portal/home'
portal = admin.Portal(url, 'username', 'password')
# get servers
servers = portal.servers
# stop sample cities service
server = servers[0]
service = server.service('SampleWorldCities.MapServer')
service.stop()
```
To connect to an ArcGIS Server instance that you would like to administer you can do the following:
```py
# test with your own servers
url = 'localhost:6080/arcgis/admin/services' #server url
usr = 'username'
pw = 'password'
# connect to ArcGIS Server instance
arcserver = admin.ArcServerAdmin(url, usr, pw)
```
To list services within a folder, you can do this:
```py
folder = arcserver.folder('SomeFolder') # supply name of folder as argument
for service in folder.iter_services():
print(service.serviceName, service.configuredState
# can stop a service like this
# service.stop()
# or start like this
# service.start()
print('\n' * 3)
# show all services and configured state (use iter_services to return restapi.admin.Service() object!)
for service in arcserver.iter_services():
print(service.serviceName, service.configuredState)
```
Security
--------
You can set security at the folder or service level. By default, the addPermssion() method used by Folder and Service objects will make the service unavailable to the general public and only those in the administrator role can view the services. This is done by setting the 'esriEveryone' principal "isAllowed" value to false. You can also assign permissions based on roles.
```py
arcserver.addPermission('SomeFolder') # by default it will make private True
# now make it publically avaiable (unsecure)
arcserver.addPermission('SomeFolder', private=False)
# secure based on role, in this case will not allow assessor group to see utility data
# assessor is name of assessor group role, Watermain is folder to secure
arcserver.addPermission('Watermain', 'assessor', False)
# note, this can also be done at the folder level:
folder = arcserver.folder('Watermain')
folder.addPermission('assessor', False)
```
Stopping and Starting Services
------------------------------
Services can easily be started and stopped with this module. This can be done from the ArcServerAdmin() or Folder() object:
```py
# stop all services in a folder
arcserver.stopServices(folderName='SomeFolder') # this can take a few minutes
# look thru the folder to check the configured states, should be stopped
for service in arcserver.folder('SomeFolder').iter_services():
print(service.serviceName, service.configuredState)
# now restart
arcserver.startServices(folderName='SomeFolder') # this can take a few minutes
# look thru folder, services should be started
for service in arcserver.folder('SomeFolder').iter_services():
print(service.serviceName, service.configuredState)
# to do this from a folder, simply get a folder object back
folder = arcserver.folder('SomeFolder')
folder.stopServices()
for service in folder.iter_services():
print(service.serviceName, service.configuredState)
```
Updating Service Properties
---------------------------
The admin package can be used to update the service definitions via JSON. By default, the Service.edit() method will pass in the original service definition as JSON so no changes are made if no arguments are supplied. The first argument is the service config as JSON, but this method also supports keyword arguments to update single properties (**kwargs). These represent keys of a the dictionary in Python.
```py
# connect to an individual service (by wildcard) - do not need to include full name, just
# enough of the name to make it a unique name query
service = arcserver.service('SampleWorldCities') #provide name of service here
# get original service description
description = service.description
# now edit the description only by using description kwarg (must match key exactly to update)
service.edit(description='This is an updated service description')
# edit description again to set it back to the original description
service.edit(description=description)
```
There are also some helper methods that aren't available out of the box from the ArcGIS REST API such as enabling or disabling extensions:
```py
# disable Feature Access and kml downloads
service.disableExtensions(['FeatureServer', 'KmlServer'])
# you can also list enabled/disabled services
print(service.enabledExtensions)
# [u'KmlServer', u'WFSServer', u'FeatureServer']
service.disabledExtensions
# [u'NAServer', u'MobileServer', u'SchematicsServer', u'WCSServer', u'WMSServer']
# Edit service extension properites
# get an extension and view its properties
fs_extension = service.getExtension('FeatureServer')
print(fs_extension) # will print as pretty json
```
For Service objects, all properties are represented as pretty json. Below is what the FeatureService Extension looks like:
```py
{
"allowedUploadFileTypes": "",
"capabilities": "Query,Create,Update,Delete,Uploads,Editing",
"enabled": "true",
"maxUploadFileSize": 0,
"properties": {
"allowGeometryUpdates": "true",
"allowOthersToDelete": "false",
"allowOthersToQuery": "true",
"allowOthersToUpdate": "false",
"allowTrueCurvesUpdates": "false",
"creatorPresent": "false",
"dataInGdb": "true",
"datasetInspected": "true",
"editorTrackingRespectsDayLightSavingTime": "false",
"editorTrackingTimeInUTC": "true",
"editorTrackingTimeZoneID": "UTC",
"enableOwnershipBasedAccessControl": "false",
"enableZDefaults": "false",
"maxRecordCount": "1000",
"realm": "",
"syncEnabled": "false",
"syncVersionCreationRule": "versionPerDownloadedMap",
"versionedData": "false",
"xssPreventionEnabled": "true",
"zDefaultValue": "0"
},
"typeName": "FeatureServer"
}
```
Setting properties for extensions is also easy:
```py
# set properties for an extension using helper method, use **kwargs for setting capabilities
service.setExtensionProperties('FeatureServer', capabilities='Query,Update,Delete,Editing')
# verify changes were made
print(fs_extension.capabilities
# 'Query,Update,Delete,Editing'
# alternatively, you can edit the service json directly and call the edit method
# change it back to original settings
fs_extension.capabilities = 'Query,Create,Update,Delete,Uploads,Editing'
service.edit()
# verify one more time...
print(fs_extension.capabilities)
# 'Query,Create,Update,Delete,Uploads,Editing'
```
Access the Data Store
---------------------
You can iterate through the data store items easily to read/update/add items:
```py
# connect to the server's data store
ds = arcserver.dataStore
# iterate through all items of data store
for item in ds:
print(item.type, item.path
# if it is an enterprise database connection, you can get the connection string like this
if item.type == 'egdb':
print(item.info.connectionString)
# else if a folder, print(server path
elif item.type == 'folder':
print(item.info.path)
print('\n')
```
User and Role Stores
--------------------
When viewing usernames/roles you can limit the number of names returned using the "maxCount" keyword argument. To view and make changes to Role Store:
```py
# connect to role store
rs = arcserver.roleStore
# print roles
for role in rs:
print(role)
# find users within roles
for role in rs:
print(role, 'Users: ', rs.getUsersWithinRole(role))
# add a user to role
rs.addUsersToRole('Administrators', 'your-domain\\someuser')
# remove user from role
rs.removeUsersFromRole('Administrators', 'your-domain\\someuser')
# remove an entire role
rs.removeRole('transportation')
```
To view and make changes to the User Store:
```py
# connect to user store
us = arcserver.userStore
# get number of users
print(len(us)
# iterate through first 10 users
for user in us.searchUsers(maxCount=10):
print(user)
# add new user
us.addUser('your-domain\\someuser', 'password')
# assign roles by using comma separated list of role names
us.assignRoles('your-domain\\someuser', 'Administrators,Publishers')
# get privileges from user
us.getPrivilegeForUser('your-domain\\someuser')
# remove roles from user
us.removeRoles('your-domain\\someuser', 'Administrators,Publishers')
```
Log Files
---------
You can easily query server log files like this:
```py
import restapi
import datetime
# query log files (within last 3 days), need to convert to milliseconds
threeDaysAgo = restapi.date_to_mil(datetime.datetime.now() - relativedelta(days=3))
for log in arcserver.queryLogs(endTime=threeDaysAgo, pageSize=25):
print(log.time
for message in log:
print(message)
print('\n')
```
A note about verbosity
----------------------
When using the admin subpackage you will likely be making changes to services/permissions etc. On operations that change a configuration, the @passthrough decorator will report back if the operation is successful and return results like this:
```py
{u'status': u'SUCCESS'}
```
The printing of these messages can be shut off by changing the global "VERBOSE" variable so these messages are not reported. This can be disabled like this:
```py
admin.VERBOSE = False
```
## Advanced Usage
### RequestClient
From version 2.0, it is possible to use a custom `requests.Session()` instance. This instance can be defined globally for all requests made by `restapi`, or it can be passed on each function call as a `restapi.RequestClient()` object. This can be useful if different parameters are needed to access different servers.
Use this functionality to access servers behind HTTP or SOCKS proxies, to disable certificate validation or use custom CA certificates, or if additional authentication is needed. Refer to the [requests.Session()](https://requests.readthedocs.io/en/master/user/advanced/#session-objects) documentation for details
```py
# Create a restapi.RequestClient() object.
custom_session = requests.Session()
custom_client = restapi.RequestClient(custom_session)
# Customize the client
proxies = {
“http”: “http://10.10.10.10:8000”,
“https”: “http://10.10.10.10:8000”,
}
custom_client.session.proxies = proxies
custom_client.session.headers['Source-Client'] = 'Custom'
# Use the client for an individual call
rest_url = 'https://gis.ngdc.noaa.gov/arcgis/rest/services'
arcserver = restapi.ArcServer(rest_url, client=custom_client)
# Set a different client as client as restapi's default
global_session = requests.Session()
global_client = restapi.RequestClient(global_session)
global_client.session.headers['Source-Client'] = 'Global'
restapi.set_request_client(global_client)
# Now any call made by restapi will use the custom client
arcserver = restapi.ArcServer(rest_url)
# The global client can also be accessed directly
restapi.requestClient.headers['Another-Header'] = 'Header is here'
```
Any session objects which extend `requests.Session()` should be supported, for example, [pypac.PACSession()](https://pypi.org/project/pypac/).
## Exceptions
The module `restapi.exceptions` contains the following custom exceptions, which correspond to the ESRI REST error returns codes specified in the [ESRI documentation](https://developers.arcgis.com/net/reference/platform-error-codes/#http-network-and-rest-errors). Other, previously unknown returns codes will raise the generic `RestAPIException`:
| code | Exception Class |
| ------ | ------ |
| 400 | RestAPIUnableToCompleteOperationException |
| 401 | RestAPIAuthorizationRequiredException |
| 403 | RestAPITokenValidAccessDeniedException |
| 404 | RestAPINotFoundException |
| 413 | RestAPITooLargeException |
| 498 | RestAPIInvalidTokenException |
| 499 | RestAPITokenRequiredException |
| 500 | RestAPIErrorPerforningOperationException |
| 501 | RestAPINotImplementedException |
| 504 | RestAPIGatewayTimeoutException |
Raw data
{
"_id": null,
"home_page": "https://github.com/Bolton-and-Menk-GIS/restapi",
"name": "bmi-arcgis-restapi",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Caleb Mackey",
"author_email": "calebma@bolton-menk.com",
"download_url": "https://files.pythonhosted.org/packages/e3/97/c9e55f99ac1289ca68c324957ae85b865de736b6e4894bccd33787040b21/bmi-arcgis-restapi-2.4.7.tar.gz",
"platform": null,
"description": "# restapi\n\nThis is a Python API for working with ArcGIS REST API, ArcGIS Online, and Portal/ArcGIS Enterprise. This package has been designed to work with [arcpy](https://pro.arcgis.com/en/pro-app/arcpy/get-started/what-is-arcpy-.htm) when available, or the included open source module [pyshp](https://pypi.org/project/pyshp/). It will try to use arcpy if available for some data conversions, otherwise will use open source options. Also included is a subpackage for administering ArcGIS Server Sites. This is updated often, so continue checking here for new functionality!\n\n### Why would you use this package?\n\nEsri currently provides the [ArcGIS API for Python](https://developers.arcgis.com/python/) which provides complete bindings to the ArcGIS REST API. This package has less coverage of the REST API, but has many convience functions not available in the ArcGIS API for Python and has a strong focus on downloading and querying data. This package will also support older versions of Python (i.e. 2.7.x) whereas Esri's package only supports 3.x.\n\n## Release History\n\n[Release History](ReleaseNotes.md)\n\n## Installation\n\n`restapi` is supported on Python 2.7 and 3.x. It can be found on [Github](https://github.com/Bolton-and-Menk-GIS/restapi) and [PyPi](https://pypi.org/project/bmi-arcgis-restapi/). To install using pip:\n\n````sh\npip install bmi-arcgis-restapi\n````\n\nAfter installation, it should be available to use in Python:\n\n````py\nimport restapi\n````\n\n## A note about `arcpy`\n\nBy default, `restapi` will import Esri's `arcpy` module if available. However, this module is not required to use this package. `arcpy` is only used when available to write data to disk in esri specific formats (file geodatabase, etc) and working with `arcpy` Geometries. When `arcpy` is not availalbe, the [pyshp](https://pypi.org/project/pyshp/) module is used to write data (shapefile format only) and work with `shapefile.Shape` objects (geometry). Also worth noting is that open source version is much faster than using `arcpy`.\n\nThat being said, there may be times when you want to force `restapi` to use the open source version, even when you have access to `arcpy`. Some example scenarios being when you don't need to write any data in an Esri specific format, you want the script to execute very fast, or you are working in an environment where `arcpy` may not play very nicely ([Flask](https://palletsprojects.com/p/flask/), [Django](https://www.djangoproject.com/), etc.). To force `restapi` to use the open source version, you can simply create an environment variable called `RESTAPI_USE_ARCPY` and set it to `FALSE` or `0`. This variable will be checked before attempting to import `arcpy`.\n\nHere is an example on how to force open source at runtime:\n\n```py\nimport os\nos.environ['RESTAPI_USE_ARCPY'] = 'FALSE'\n\n# now import restapi\nimport restapi\n```\n\n## requests.exceptions.SSLError\n\nIf you are seeing `requests.exceptions.SSLError` exceptions in `restapi` >= 2.0, this is probaly due to a change in handling servers without valid SSL certificates. Because many ArcGIS Server instances are accessed using SSL with a self-signed certificate, or through a MITM proxy like Fiddler, `restapi` < 2.0 defaulted to ignoring SSL errors by setting the `request` client's `verify` option to `False`. The new default behavior is to enable certificate verification. If you are receiving this error, you are probably accessing your server with a self-signed certificate, or through a MITM proxy. If that is not the case, you should investigate why you are seeing SSL errors, as there would likely be an issue with the server configuration, or some other security issues.\n\nTo mimic the previous behavior in the newer versions of `restapi`, there are 2 options - disable certificate verification (less secure), or [build a custom CA bundle ](https://requests.readthedocs.io/en/stable/user/advanced/#ssl-cert-verification) which includes any self-signed certificates needed to access your server (more secure). Both of these can be done using the new [restapi.RequestClient()](#requestclient) feature.\n\n````py\nimport restapi\nimport requests\nsession = requests.Session()\nclient = restapi.RequestClient(session)\nrestapi.set_request_client(client)\n\n# Disable verification\nclient.session.verify = False\n\n# -or-\n\n# Use custom CA bundle\nclient.session.verify = '/path/to/certfile'\n````\n\nSince `verify = False` is a commonly used setting when dealing with ArcGIS Server instances, it's also possible to use an environment variable. The variable must be set before `restapi` is imported.\n\n````py\nos.environ['RESTAPI_VERIFY_CERT'] = 'FALSE'\nimport restapi\n````\n\n## Connecting to an ArcGIS Server\n\n> samples for this section can be found in the [connecting_to_arcgis_server.py](./restapi/samples/connecting_to_arcgis_server.py) file.\n\nOne of the first things you might do is to connect to a services directory (or catalog):\n\n````py\nimport restapi\n\n# connect to esri's sample server 6\nrest_url = 'https://sampleserver6.arcgisonline.com/arcgis/rest/services'\n\n# connect to restapi.ArcServer instance \nags = restapi.ArcServer(rest_url)\n````\n\n```py\n>>> # get folder and service properties\n>>> print('Number of folders: {}'.format(len(ags.folders)))\n>>> print('Number of services: {}\\n'.format(len(ags.services)))\n>>> \n>>> # walk thru directories\n>>> for root, services in ags.walk():\n>>> print('Folder: {}'.format(root))\n>>> print('Services: {}\\n'.format(services))\n\nNumber of folders: 13\nNumber of services: 60\n\nFolder: None\nServices: ['911CallsHotspot/GPServer', '911CallsHotspot/MapServer', 'Census/MapServer', 'CharlotteLAS/ImageServer', 'CommercialDamageAssessment/FeatureServer', 'CommercialDamageAssessment/MapServer', 'CommunityAddressing/FeatureServer', 'CommunityAddressing/MapServer', '<...>', 'Water_Network/MapServer', 'Wildfire/FeatureServer', 'Wildfire/MapServer', 'WindTurbines/MapServer', 'World_Street_Map/MapServer', 'WorldTimeZones/MapServer']\n\nFolder: AGP\nServices: ['AGP/Census/MapServer', 'AGP/Hurricanes/MapServer', 'AGP/USA/MapServer', 'AGP/WindTurbines/MapServer']\n\nFolder: Elevation\nServices: ['Elevation/earthquakedemoelevation/ImageServer', 'Elevation/ESRI_Elevation_World/GPServer', 'Elevation/GlacierBay/MapServer', 'Elevation/MtBaldy_Elevation/ImageServer', 'Elevation/WorldElevations/MapServer']\n\n# ..etc\n```\n\n#### Connecting to a child service\n\nfrom an `ArcServer` instance, you can connect to any service within that instance by providing a child path or wildcard search match.\n\n````py\n# connect to a specific service\n# using just the service name (at the root)\nusa = ags.getService('USA') #/USA/MapServer -> restapi.common_types.MapService\n\n# using the relative path to a service in a folder\ncensus = ags.getService('AGP/Census') #/AGP/Census/MapServer -> restapi.common_types.MapService\n\n# can also just use the service name, but be weary of possible duplicates\ninfastructure = ags.getService('Infrastructure') #/Energy/Infrastructure/FeatureServer -> restapi.common_types.FeatureService\n\n# using a wildcard search\ncovid_cases = ags.getService('*Covid19Cases*') #/NYTimes_Covid19Cases_USCounties/MapServer -> restapi.common_types.MapService\n````\n\ncheck outputs:\n\n```py\n>>> for service in [usa, census, infastructure, covid_cases]:\n>>> print('name: \"{}\"'.format(service.name))\n>>> print('repr: \"{}\"'.format(repr(service)))\n>>> print('url: {}\\n'.format(service.url))\n\nname: \"USA\"\nrepr: \"<MapService: USA/MapServer>\"\nurl: https://sampleserver6.arcgisonline.com/arcgis/rest/services/USA/MapServer\n\nname: \"Census\"\nrepr: \"<MapService: AGP/Census/MapServer>\"\nurl: https://sampleserver6.arcgisonline.com/arcgis/rest/services/AGP/Census/MapServer\n\nname: \"Infrastructure\"\nrepr: \"<FeatureService: Energy/Infrastructure/FeatureServer>\"\nurl: https://sampleserver6.arcgisonline.com/arcgis/rest/services/Energy/Infrastructure/FeatureServer\n\nname: \"NYTimes_Covid19Cases_USCounties\"\nrepr: \"<MapService: NYTimes_Covid19Cases_USCounties/MapServer>\"\nurl: https://sampleserver6.arcgisonline.com/arcgis/rest/services/NYTimes_Covid19Cases_USCounties/MapServer\n```\n\n### Working with layers\n\n> samples for this section can be found in the [working_with_layers.py](./restapi/samples/working_with_layers.py) file.\n\nYou can connect to `MapServiceLayer`s or `FeatureLayer`s directly by passing the url, or by accessing from the parent `FeatureService` or `MapService`.\n\nalso query the layer and get back arcpy.da Cursor like access\n\n```py\n# get access to the \"Cities\" layer from USA Map Service\ncities = usa.layer('Cities') # or can use layer id: usa.layer(0)\n\n# query the map layer for all cities in California with population > 100000\nwhere = \"st = 'CA' and pop2000 > 100000\"\n\n# the query operation returns a restapi.FeatureSet or restapi.FeatureCollection depending on the return format\nfeatureSet = cities.query(where=where)\n\n# get result count, can also use len(featureSet)\nprint('Found {} cities in California with Population > 100K'.format(featureSet.count))\n\n# print first feature (restapi.Feature). The __str__ method is pretty printed JSON\n# can use an index to fetch via its __getitem__ method\nprint(featureSet[0])\n```\n\n#### exceeding the `maxRecordCount`\n\nIn many cases, you may run into issues due to the `maxRecordCount` set for a service, which by default is `1000` features. This limit can be exceeded by using the `exceed_limit` parameter. If `exceed_limit` is set to `True`, the `query_in_chunks` method will be called internally to keep fetching until all queried features are returned.\n\n```py\n# fetch first 1000\nfirst1000 = cities.query()\nprint('count without exceed_limit: {}'.format(first1000.count)) # can also use len()\n\n# fetch all by exceeding limit\nallCities = cities.query(exceed_limit=True)\nprint('count without exceed_limit: {}'.format(len(allCities)))\n```\n\ncheck outputs:\n\n```py\nFound 57 cities in California with Population > 100K\n{\n \"type\": \"Feature\",\n \"geometry\": {\n \"type\": \"Point\",\n \"coordinates\": [\n -117.88976896399998,\n 33.836165033999976\n ]\n },\n \"properties\": {\n \"areaname\": \"Anaheim\"\n }\n}\ncount without exceed_limit: 1000\ntotal records: 3557\ncount with exceed_limit: 3557\n```\n\n#### using a search cursor\n\nA query can also be done by using a search cursor, which behaves very similarly to the [arcpy.da.SearchCursor](https://pro.arcgis.com/en/pro-app/latest/arcpy/data-access/searchcursor-class.htm). While the `restapi.Cursor` does support usage of a `with` statement, it is not necessary as there is no file on disk that is opened. When using the `cursor()` method on a `MapServiceLayer` or `FeatureLayer` it will actually make a call to the ArcGIS Server and will return a tuple of values based on the `fields` requested. The default `fields` value is `*`, which will return all fields. Like the arcpy cursors, the `SHAPE@` token can be used to fetch geometry, while the `OID@` token will fetch the object id.\n\n```py\n# if you don't want the json/FeatureSet representation, you can use restapi cursors\n# for the query which are similar to the arcpy.da cursors. The `SearchCursor` will return a tuple\ncursor = restapi.SearchCursor(cities, fields=['areaname', 'pop2000', 'SHAPE@'], where=where)\nfor row in cursor:\n print(row)\n````\n\ncheck outputs:\n\n```py\n('Anaheim', 328014, <restapi.shapefile.Shape object at 0x000002B1DE232400>)\n('Bakersfield', 247057, <restapi.shapefile.Shape object at 0x000002B1DE232470>)\n('Berkeley', 102743, <restapi.shapefile.Shape object at 0x000002B1DE232400>)\n('Burbank', 100316, <restapi.shapefile.Shape object at 0x000002B1DE232470>)\n('Chula Vista', 173556, <restapi.shapefile.Shape object at 0x000002B1DE232400>)\n('Concord', 121780, <restapi.shapefile.Shape object at 0x000002B1DE232470>)\n('Corona', 124966, <restapi.shapefile.Shape object at 0x000002B1DE232400>)\n# ... etc\n```\n\nalso supports `with` statements:\n\n```py\nwith restapi.SearchCursor(cities, fields=['areaname', 'pop2000', 'SHAPE@'], where=where) as cursor:\n for row in cursor:\n print(row)\n```\n\nIn the above examples, when the `SearchCursor` is instantiated, it actually kicked off a new query for the cursor results. If you already have a `FeatureSet` or `FeatureCollection` like we did above, you can also save a network call by constructing the `restapi.Cursor` from the feature set by doing:\n\n```py\nfeatureSet = cities.query(where=where)\n\n# can pass in fields as second argument to limit results\ncursor = restapi.Cursor(featureSet, ['areaname', 'pop2000', 'SHAPE@'])\n```\n\n> note: Both the `MapServiceLayer` and `FeatureLayer` also have a `searchCursor()` convenience method, which will allow you to use the same functionality as shown above but without passing in the layer itself:\n> `rows = cities.searchCursor(['areaname', 'pop2000', 'SHAPE@'])`\n\n#### exporting features from a layer\n\nData can also be easily exported from a `FeatureLayer` or `MapServicelayer`. This can be done by exporting a feature set directly or from the layer itself:\n\n```py\n# set output folder\nout_folder = os.path.join(os.path.expanduser('~'), 'Documents', 'restapi_samples')\n\n# create output folder if it doesn't exist\nif not os.path.exists(out_folder):\n os.makedirs(out_folder)\n\n# output shapefile\nshp = os.path.join(out_folder, 'CA_Cities_100K.shp')\n\n# export layer from map service\ncities.export_layer(shp, where=where)\n```\n\nYou can also export a feature set directly if you already have one loaded:\n\n```py\nrestapi.exportFeatureSet(featureSet, shp)\n```\n\nexporting layers or feature sets also supports a `fields` filter, where you can limit which fields are exported. Other options are supported as well such as an output spatial reference. One important thing to note, if you do not have access to `arcpy`, you can only export to shapefile format. If you do have `arcpy`, any output format supported by esri will work. If the output is a [geodatabase feature class](https://desktop.arcgis.com/en/arcmap/latest/manage-data/feature-classes/a-quick-tour-of-feature-classes.htm), you can also choose to include things like [domains](https://desktop.arcgis.com/en/arcmap/latest/manage-data/geodatabases/an-overview-of-attribute-domains.htm) if applicable.\n\n```py\n# exporting features \nout_folder = os.path.join(os.path.expanduser('~'), 'Documents', 'restapi_samples')\nif not os.path.exists(out_folder):\n os.makedirs(out_folder)\nshp = os.path.join(out_folder, 'CA_Cities_100K.shp')\n\n# export layer to a shapefile\ncities.export_layer(shp, where=where)\n\n# if there is an existing feature set, you can also export that directly\n# restapi.exportFeatureSet(featureSet, shp)\n```\n\n### selecting features by geometry\n\nBoth the `MapServiceLayer` and `FeatureLayer` support arcgis spatial selections using [arcgis geometry](https://developers.arcgis.com/documentation/common-data-types/geometry-objects.htm) objects. The `select_layer_by_location` method will return a `FeatureSet` or `FeatureCollection`, while the `clip` method will actually export the spatial selection to a shapffile or Feature Class (latter only available with `arcpy`).\n\n#### the `geometry-helper` application\n\nThis package now includes a [geometry-helper](geometry-helper/README.md) application that can be used to quickly create geometries and copy the geometry in either `esri-json` or `geojson` format to form geometry necessary to make queries:\n\n\n\nto open this application in a web browser, simply call:\n\n```py\nrestapi.open_geometry_helper()\n```\n\n#### select by location\n\nto select a layer by location, first create a geometry object as json (can be esri or geojson format) and make the selection:\n\n```py\n# select layer by location\nuniversities_url = 'https://services1.arcgis.com/Hp6G80Pky0om7QvQ/arcgis/rest/services/Colleges_and_Universities/FeatureServer/0'\n\n# get feature layer\nuniversities = restapi.FeatureLayer(universities_url)\nprint('universities: ', repr(universities))\n\n# form geometry (do not have to cast to restapi.Geometry, this will happen under the hood automatically)\ngeometry = restapi.Geometry({\n \"spatialReference\": {\n \"latestWkid\": 3857,\n \"wkid\": 102100\n },\n \"rings\": [\n [\n [\n -10423340.4579098,\n 5654465.8453829475\n ],\n [\n -10324889.565478457,\n 5654465.8453829475\n ],\n [\n -10324889.565478457,\n 5584449.527473665\n ],\n [\n -10423340.4579098,\n 5584449.527473665\n ],\n [\n -10423340.4579098,\n 5654465.8453829475\n ]\n ]\n ]\n})\n\n# make selection\nfeatureCollection = universities.select_by_location(geometry)\nprint('Number of Universities in Twin Cities area: {}'.format(featureCollection.count))\n\n# can also export the feature collection directly or call the clip() method (makes a new call to server)\nuniversities_shp = os.path.join(out_folder, 'TwinCities_Univiersities.shp')\nuniversities.clip(geometry, universities_shp)\n```\n\ncheck outputs:\n\n```py\nuniversities: <FeatureLayer: \"CollegesUniversities\" (id: 0)>\ntotal records: 50\nNumber of Universities in Twin Cities area: 50\ntotal records: 50\nCreated: \"L:\\Users\\calebma\\Documents\\restapi_samples\\TwinCities_Univiersities.shp\"\nFetched all records\n```\n\n## FeatureLayer Editing\n\n> samples for this section can be found in the [feature_layer_editing.py](./restapi/samples/feature_layer_editing.py) file.\n\nThis package also includes comprehensive support for working with [FeatureLayers](https://enterprise.arcgis.com/en/portal/latest/use/feature-layers.htm). These are editable layers hosted by an ArcGIS Server, Portal, or ArcGIS Online. A `FeatureLayer` is an extension of the `MapServiceLayer` that supports editing options.\n\n#### add features using `FeatureLayer.addFeatures()`\n\n```py\n# connect to esri's Charlotte Hazards Sample Feature Layer for incidents\nurl = 'https://services.arcgis.com/V6ZHFr6zdgNZuVG0/ArcGIS/rest/services/Hazards_Uptown_Charlotte/FeatureServer/0'\n\n# instantiate a FeatureLayer\nhazards = restapi.FeatureLayer(url)\n\n# create a new feature as json\nfeature = {\n \"attributes\" : { \n \"HazardType\" : \"Road Not Passable\", \n \"Description\" : \"restapi test\", \n \"SpecialInstructions\" : \"Contact Dispatch\", \n \"Priority\" : \"High\",\n \"Status\": \"Active\"\n }, \n \"geometry\": create_random_coordinates()\n}\n\n# add json feature\nadds = hazards.addFeatures([ feature ])\nprint(adds)\n```\n\ncheck outputs:\n\n```py\nAdded 1 feature(s)\n{\n \"addResults\": [\n {\n \"globalId\": \"8B643546-7253-4F0B-80DE-2DE8F1754C03\",\n \"objectId\": 11858,\n \"success\": true,\n \"uniqueId\": 11858\n }\n ]\n}\n```\n\n#### add an attachment\n\n```py\n# add attachment to new feature using the OBJECTID of the new feature\noid = adds.addResults[0].objectId\nimage = os.path.join(os.path.abspath('...'), 'docs', 'images', 'geometry-helper.png')\nattRes = hazards.addAttachment(oid, image)\nprint(attRes)\n\n# now query attachments for new feature\nattachments = hazards.attachments(oid)\nprint(attachments)\n```\n\ncheck outputs:\n\n```py\nAdded attachment '211' for feature objectId\n{\n \"addAttachmentResult\": {\n \"globalId\": \"f370a4e0-3976-4809-b0d5-f5d1b5990b4b\",\n \"objectId\": 211,\n \"success\": true\n }\n}\n[<Attachment ID: 211 (geometry-helper.png)>]\n```\n\n#### update features\n\n```py\n# update the feature we just added, payload only has to include OBJECTID and any fields to update\nupdatePayload = [\n { \n \"attributes\": { \n \"OBJECTID\": r.objectId, \n \"Description\": \"restapi update\" \n } \n } for r in adds.addResults\n]\nupdates = hazards.updateFeatures(updatePayload)\nprint(updates)\n```\n\ncheck outputs:\n\n```py\nUpdated 1 feature(s)\n{\n \"updateResults\": [\n {\n \"globalId\": null,\n \"objectId\": 11858,\n \"success\": true,\n \"uniqueId\": 11858\n }\n ]\n}\n```\n\n#### delete features\n\n```py\n# delete features by passing in a list of objectIds\ndeletePayload = [r.objectId for r in updates.updateResults]\ndeletes = hazards.deleteFeatures(deletePayload)\nprint(deletes)\n```\n\ncheck outputs:\n\n```py\nDeleted 1 feature(s)\n{\n \"deleteResults\": [\n {\n \"globalId\": null,\n \"objectId\": 11858,\n \"success\": true,\n \"uniqueId\": 11858\n }\n ]\n}\n```\n\n### feature editing with `restapi` cursors\n\n`restapi` also supports cursors similar to what you get when using `arcpy`. However, these work directly with the REST API and JSON features while also supporting `arcpy` and `shapefile` geometry types. See the below example on how to use an `insertCursor` to add new records:\n\n#### adding features with an `InsertCursor`\n\nNew records can be added using an `InsertCursor`. This can be instantiated using the `InsertCursor` class itself, or by calling the `insertCursor` method of a `FeatureLayer`. When called as `FeatureLayer.insertCursor`, the layer itself will not need to be passed in.\n\n```py\n>>> # add 5 new features using an insert cursor \n>>> # using this in a \"with\" statement will call applyEdits on __exit__\n>>> fields = [\"SHAPE@\", 'HazardType', \"Description\", \"Priority\"]\n>>> with restapi.InsertCursor(hazards, fields) as irows:\n for i in list(range(1,6)):\n desc = \"restapi insert cursor feature {}\".format(i)\n irows.insertRow([create_random_coordinates(), \"Wire Down\", desc, \"High\"])\n\nAdded 5 feature(s)\n```\n\nAny time an insert or update cursor will save changes, it will print a short message showing how many features were affected. You can always get at the raw edit information from the `FeatureLayer` by calling the `editResults` property. This will be an array that stores the results of every `applyEdits` operation, so the length will reflect how many times edits have been saved.\n\n```py\n>>> # we can always view the results by calling FeatureLayer.editResults which stores\n>>> # an array of edit results for each time applyEdits() is called.\n>>> print(hazards.editResults)\n[{\n \"addResults\": [\n {\n \"globalId\": \"2774C9ED-D8F8-4B71-9F37-B26B40790710\",\n \"objectId\": 12093,\n \"success\": true,\n \"uniqueId\": 12093\n },\n {\n \"globalId\": \"18411060-F6FF-49FB-AD91-54DB65C13D06\",\n \"objectId\": 12094,\n \"success\": true,\n \"uniqueId\": 12094\n },\n {\n \"globalId\": \"8045C840-F1B9-4BD2-AEDC-72F4D65EB7A6\",\n \"objectId\": 12095,\n \"success\": true,\n \"uniqueId\": 12095\n },\n {\n \"globalId\": \"EC9DB6FC-0D34-4B83-8C98-398C7B48666D\",\n \"objectId\": 12096,\n \"success\": true,\n \"uniqueId\": 12096\n },\n {\n \"globalId\": \"C709033F-DF3B-43B3-8148-2299E7CEE986\",\n \"objectId\": 12097,\n \"success\": true,\n \"uniqueId\": 12097\n }\n ],\n \"deleteResults\": [],\n \"updateResults\": []\n}]\n```\n\n> note: when using a `with` statement for the `InsertCursor` and `UpdateCursor` it will automatically call the `applyEdits()` method on `__exit__`, which is critical to submitting the new, deleted, or updated features to the server. If not using a `with` statement, you will need to call `applyEdits()` manually after changes have been made.\n\n#### updating features with an `UpdateCursor`\n\nrecords can be updated or deleted with an `updateCursor` and a where clause. Note that the `OBJECTID` field must be included in the query to indicate which records will be updated. The `OID@` field token can be used to retreive the `objectIdFieldName`:\n\n```py\n>>> # now update records with updateCursor for the records we just added. Can use the \n>>> # editResults property of the feature layer to get the oids of our added features\n>>> addedOids = ','.join(map(str, [r.objectId for r in hazards.editResults[0].addResults]))\n>>> whereClause = \"{} in ({})\".format(hazards.OIDFieldName, addedOids)\n>>> with restapi.UpdateCursor(hazards, [\"Priority\", \"Description\", \"OID@\"], where=whereClause) as rows:\n for row in rows:\n if not row[2] % 2:\n # update rows with even OBJECTID's\n row[0] = \"Low\"\n rows.updateRow(row)\n else:\n # delete odd OBJECTID rows\n print('deleting row with odd objectid: ', row[2])\n rows.deleteRow(row)\n \nUpdated 2 feature(s)\nDeleted 3 feature(s) \n\n>>> # now delete the rest of the records we added\n>>> whereClause = \"Description like 'restapi%'\"\n>>> with restapi.UpdateCursor(hazards, [\"Description\", \"Priority\", \"OID@\"], where=whereClause) as rows:\n for row in rows:\n rows.deleteRow(row)\n irows.insertRow([create_random_coordinates(), \"Wire Down\", desc, \"High\"])\n\nDeleted 2 feature(s)\n```\n\n#### attachment editing\n\nOffline capabilities (Sync)\n\n````py\n# if sync were enabled, we could create a replica like this:\n# can pass in layer ID (0) or name ('incidents', not case sensative)\nreplica = fs.createReplica(0, 'test_replica', geometry=adds[0]['geometry'], geometryType='esriGeometryPoint', inSR=4326)\n\n# now export the replica object to file geodatabase (if arcpy access) or shapefile with hyperlinks (if open source)\nrestapi.exportReplica(replica, folder)\n````\n\nWorking with Image Services\n---------------------------\n\n````py\nurl = 'http://pca-gis02.pca.state.mn.us/arcgis/rest/services/Elevation/DEM_1m/ImageServer'\nim = restapi.ImageService(url)\n\n# clip DEM\ngeometry = {\"rings\":[[\n [240006.00808044084, 4954874.19629429],\n [240157.31010183255, 4954868.8053006204],\n [240154.85966611796, 4954800.0316874133],\n [240003.55764305394, 4954805.4226145679],\n [240006.00808044084, 4954874.19629429]]],\n \"spatialReference\":{\"wkid\":26915,\"latestWkid\":26915}}\n\ntif = os.path.join(folder, 'dem.tif')\nim.clip(geometry, tif)\n\n# test point identify\nx, y = 400994.780878, 157878.398217\nelevation = im.pointIdentify(x=x, y=y, sr=103793)\nprint(elevation)\n````\n\nGeocoding\n---------\n\n````py\n# hennepin county, MN geocoder\nhenn = 'http://gis.hennepin.us/arcgis/rest/services/Locators/HC_COMPOSITE/GeocodeServer'\ngeocoder = restapi.Geocoder(henn)\n# find target field, use the SingleLine address field by default\ngeoResult = geocoder.findAddressCandidates('353 N 5th St, Minneapolis, MN 55403')\n\n# export results to shapefile\nprint('found {} candidates'.format(len(geoResult))\ngeocoder.exportResults(geoResult, os.path.join(folder, 'target_field.shp'))\n\n# Esri geocoder\nesri_url = 'http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Locators/ESRI_Geocode_USA/GeocodeServer'\nesri_geocoder = restapi.Geocoder(esri_url)\n\n# find candidates using key word arguments (**kwargs) to fill in locator fields, no single line option\ncandidates = esri_geocoder.findAddressCandidates(Address='380 New York Street', City='Redlands', State='CA', Zip='92373')\nprint('Number of address candidates: {}'.format(len(candidates)))\nfor candidate in candidates:\n print(candidate.location)\n\n# export results to shapefile\nout_shp = os.path.join(folder, 'Esri_headquarters.shp')\ngeocoder.exportResults(candidates, out_shp)\n````\n\nGeoprocessing Services\n----------------------\n\n````py\n# test esri's drive time analysis GP Task\ngp_url = 'http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Network/ESRI_DriveTime_US/GPServer/CreateDriveTimePolygons'\ngp = restapi.GPTask(gp_url)\n\n# get a list of gp parameters (so we know what to pass in as kwargs)\nprint('\\nGP Task \"{}\" parameters:\\n'.format(gp.name)\nfor p in gp.parameters:\n print('\\t', p.name, p.dataType)\n\npoint = {\"geometryType\":\"esriGeometryPoint\",\n \"features\":[\n {\"geometry\":{\"x\":-10603050.16225853,\"y\":4715351.1473399615,\n \"spatialReference\":{\"wkid\":102100,\"latestWkid\":3857}}}],\n \"sr\":{\"wkid\":102100,\"latestWkid\":3857}}\n\n# run task, passing in gp parameters as keyword arguments (**kwargs)\ngp_res = gp.run(Input_Location=str(point), Drive_Times = '1 2 3', inSR = 102100)\n\n# returns a GPResult() object, can get at the first result by indexing (usually only one result)\n# can test if there are results by __nonzero__()\nif gp_res:\n result = gp_res.results[0]\n \n # this returned a GPFeatureRecordSetLayer as an outputParameter, so we can export this to polygons\n print('\\nOutput Result: \"{}\", data type: {}\\n'.format(result.paramName, result.dataType))\n\n # now export the result value to fc (use the value property of the GPResult object from run())\n drive_times = os.path.join(folder, 'drive_times.shp')\n restapi.exportFeatureSet(drive_times, gp_res.value)\n````\n\nA note about input Geometries\n-----------------------------\n\nrestapi will try to use arcpy first if you have it, otherwise will defer to open source. Both\nsupport the reading of shapefiles to return the first feature back as a restapi.Geometry object\n\nIt also supports arcpy Geometries and shapefile.Shape() objects\n\n````py\n>>> shp = r'C:\\TEMP\\Polygons.shp' # a shapefile on disk somewhere\n>>> geom = restapi.Geometry(shp)\n>>> print(geom.envelope())\n-121.5,38.3000000007,-121.199999999,38.6000000015\n````\n\nToken Based Security\n--------------------\n\nrestapi also supports secured services. This is also session based, so if you sign in once to an\nArcGIS Server Resource (on the same ArcGIS Site), the token will automatically persist via the\nIdentityManager().\n\nThere are 3 ways to authticate:\n\n````py\n# **kwargs for all accessing all ArcGIS resources are\n# usr -- username\n# pw -- password\n# token -- token (as string or restapi.Token object)\n# proxy -- url to proxy\n\n# secured url\nsecured_url = 'http://some-domain.com/arcgis/rest/services'\n\n# 1. username and password\nags = restapi.ArcServer(url, 'username', 'password') # token is generated and persists\n\n# 2. a token that has already been requested\nags = restapi.ArcServer(url, token=token) # uses a token that is already active\n\n# 3. via a proxy (assuming using the standard esri proxy)\n# this will forward all subsequent requests through the proxy\nags = restapi.ArcServer(url, proxy='http://some-domain.com/proxy.ashx')\n````\n\nYou can even just generate a token and let the IdentityManager handle the rest. It is even smart enough to handle multiple tokens for different sites:\n\n```py\n# login to instance 1\nusr = 'username'\npw = 'password'\n\n# urls to two different ArcGIS Server sites\nurl_1 = 'http://some-domain.com/arcserver1/rest/services'\nurl_2 = 'http://domain2.com/arcgis/rest/services'\n\n# generate tokens\ntok1 = restapi.generate_token(url_1, usr, pw)\ntok2 = restapi.generate_token(url_2, usr, pw)\n\n# now we should be able to access both ArcGIS Server sites via the IdentityManager\narcserver1 = restapi.ArcServer(url_1) # tok1 is automatically passed in and handled\narcserver2 = restapi.ArcServer(url_2) # tok2 is used here\n```\n\nThe admin Subpackage\n--------------------\n\nrestapi also contains an administrative subpackage (warning: most functionality has not been tested!). You can import this module like this:\n\n```py\nfrom restapi import admin\n```\n\n### Connecting to a Portal\n\n```py\nurl = 'https://domain.gis.com/portal/home'\nportal = admin.Portal(url, 'username', 'password')\n\n# get servers\nservers = portal.servers\n\n# stop sample cities service\nserver = servers[0]\n\nservice = server.service('SampleWorldCities.MapServer')\nservice.stop()\n\n```\n\nTo connect to an ArcGIS Server instance that you would like to administer you can do the following:\n\n```py\n# test with your own servers\nurl = 'localhost:6080/arcgis/admin/services' #server url\nusr = 'username'\npw = 'password'\n\n# connect to ArcGIS Server instance\narcserver = admin.ArcServerAdmin(url, usr, pw)\n```\n\nTo list services within a folder, you can do this:\n\n```py\nfolder = arcserver.folder('SomeFolder') # supply name of folder as argument\nfor service in folder.iter_services():\n print(service.serviceName, service.configuredState\n\n # can stop a service like this\n # service.stop()\n\n # or start like this\n # service.start()\n\nprint('\\n' * 3)\n\n# show all services and configured state (use iter_services to return restapi.admin.Service() object!)\nfor service in arcserver.iter_services():\n print(service.serviceName, service.configuredState)\n```\n\nSecurity\n--------\n\nYou can set security at the folder or service level. By default, the addPermssion() method used by Folder and Service objects will make the service unavailable to the general public and only those in the administrator role can view the services. This is done by setting the 'esriEveryone' principal \"isAllowed\" value to false. You can also assign permissions based on roles.\n\n```py\narcserver.addPermission('SomeFolder') # by default it will make private True \n\n# now make it publically avaiable (unsecure)\narcserver.addPermission('SomeFolder', private=False)\n\n# secure based on role, in this case will not allow assessor group to see utility data\n# assessor is name of assessor group role, Watermain is folder to secure\narcserver.addPermission('Watermain', 'assessor', False) \n\n# note, this can also be done at the folder level:\nfolder = arcserver.folder('Watermain')\nfolder.addPermission('assessor', False)\n```\n\nStopping and Starting Services\n------------------------------\n\nServices can easily be started and stopped with this module. This can be done from the ArcServerAdmin() or Folder() object:\n\n```py\n# stop all services in a folder\narcserver.stopServices(folderName='SomeFolder') # this can take a few minutes\n\n# look thru the folder to check the configured states, should be stopped\nfor service in arcserver.folder('SomeFolder').iter_services():\n print(service.serviceName, service.configuredState)\n\n# now restart\narcserver.startServices(folderName='SomeFolder') # this can take a few minutes\n\n# look thru folder, services should be started\nfor service in arcserver.folder('SomeFolder').iter_services():\n print(service.serviceName, service.configuredState)\n \n# to do this from a folder, simply get a folder object back\nfolder = arcserver.folder('SomeFolder')\nfolder.stopServices()\nfor service in folder.iter_services():\n print(service.serviceName, service.configuredState)\n```\n\nUpdating Service Properties\n---------------------------\n\nThe admin package can be used to update the service definitions via JSON. By default, the Service.edit() method will pass in the original service definition as JSON so no changes are made if no arguments are supplied. The first argument is the service config as JSON, but this method also supports keyword arguments to update single properties (**kwargs). These represent keys of a the dictionary in Python.\n\n```py\n# connect to an individual service (by wildcard) - do not need to include full name, just\n# enough of the name to make it a unique name query\nservice = arcserver.service('SampleWorldCities') #provide name of service here\n\n# get original service description\ndescription = service.description\n\n# now edit the description only by using description kwarg (must match key exactly to update)\nservice.edit(description='This is an updated service description')\n\n# edit description again to set it back to the original description\nservice.edit(description=description)\n```\n\nThere are also some helper methods that aren't available out of the box from the ArcGIS REST API such as enabling or disabling extensions:\n\n```py\n# disable Feature Access and kml downloads\nservice.disableExtensions(['FeatureServer', 'KmlServer'])\n\n# you can also list enabled/disabled services\nprint(service.enabledExtensions)\n# [u'KmlServer', u'WFSServer', u'FeatureServer']\n\nservice.disabledExtensions\n# [u'NAServer', u'MobileServer', u'SchematicsServer', u'WCSServer', u'WMSServer']\n\n# Edit service extension properites\n# get an extension and view its properties\nfs_extension = service.getExtension('FeatureServer')\n\nprint(fs_extension) # will print as pretty json\n```\n\nFor Service objects, all properties are represented as pretty json. Below is what the FeatureService Extension looks like:\n\n```py\n{\n \"allowedUploadFileTypes\": \"\", \n \"capabilities\": \"Query,Create,Update,Delete,Uploads,Editing\", \n \"enabled\": \"true\", \n \"maxUploadFileSize\": 0, \n \"properties\": {\n \"allowGeometryUpdates\": \"true\", \n \"allowOthersToDelete\": \"false\", \n \"allowOthersToQuery\": \"true\", \n \"allowOthersToUpdate\": \"false\", \n \"allowTrueCurvesUpdates\": \"false\", \n \"creatorPresent\": \"false\", \n \"dataInGdb\": \"true\", \n \"datasetInspected\": \"true\", \n \"editorTrackingRespectsDayLightSavingTime\": \"false\", \n \"editorTrackingTimeInUTC\": \"true\", \n \"editorTrackingTimeZoneID\": \"UTC\", \n \"enableOwnershipBasedAccessControl\": \"false\", \n \"enableZDefaults\": \"false\", \n \"maxRecordCount\": \"1000\", \n \"realm\": \"\", \n \"syncEnabled\": \"false\", \n \"syncVersionCreationRule\": \"versionPerDownloadedMap\", \n \"versionedData\": \"false\", \n \"xssPreventionEnabled\": \"true\", \n \"zDefaultValue\": \"0\"\n }, \n \"typeName\": \"FeatureServer\"\n}\n```\n\nSetting properties for extensions is also easy:\n\n```py\n# set properties for an extension using helper method, use **kwargs for setting capabilities\nservice.setExtensionProperties('FeatureServer', capabilities='Query,Update,Delete,Editing')\n\n# verify changes were made\nprint(fs_extension.capabilities\n# 'Query,Update,Delete,Editing'\n\n# alternatively, you can edit the service json directly and call the edit method\n# change it back to original settings\nfs_extension.capabilities = 'Query,Create,Update,Delete,Uploads,Editing'\nservice.edit()\n\n# verify one more time...\nprint(fs_extension.capabilities)\n# 'Query,Create,Update,Delete,Uploads,Editing'\n```\n\nAccess the Data Store\n---------------------\n\nYou can iterate through the data store items easily to read/update/add items:\n\n```py\n# connect to the server's data store\nds = arcserver.dataStore\n\n# iterate through all items of data store\nfor item in ds:\n print(item.type, item.path\n # if it is an enterprise database connection, you can get the connection string like this\n if item.type == 'egdb':\n print(item.info.connectionString)\n # else if a folder, print(server path\n elif item.type == 'folder':\n print(item.info.path)\n print('\\n')\n```\n\nUser and Role Stores\n--------------------\n\nWhen viewing usernames/roles you can limit the number of names returned using the \"maxCount\" keyword argument. To view and make changes to Role Store:\n\n```py\n# connect to role store\nrs = arcserver.roleStore\n\n# print roles\nfor role in rs:\n print(role)\n\n# find users within roles\nfor role in rs:\n print(role, 'Users: ', rs.getUsersWithinRole(role))\n\n# add a user to role\nrs.addUsersToRole('Administrators', 'your-domain\\\\someuser')\n\n# remove user from role\nrs.removeUsersFromRole('Administrators', 'your-domain\\\\someuser')\n\n# remove an entire role\nrs.removeRole('transportation')\n```\n\nTo view and make changes to the User Store:\n\n```py\n# connect to user store\nus = arcserver.userStore\n\n# get number of users\nprint(len(us)\n\n# iterate through first 10 users\nfor user in us.searchUsers(maxCount=10):\n print(user)\n \n# add new user\nus.addUser('your-domain\\\\someuser', 'password')\n\n# assign roles by using comma separated list of role names\nus.assignRoles('your-domain\\\\someuser', 'Administrators,Publishers')\n\n# get privileges from user\nus.getPrivilegeForUser('your-domain\\\\someuser')\n\n# remove roles from user \nus.removeRoles('your-domain\\\\someuser', 'Administrators,Publishers')\n```\n\nLog Files\n---------\n\nYou can easily query server log files like this:\n\n```py\nimport restapi\nimport datetime\n\n# query log files (within last 3 days), need to convert to milliseconds\nthreeDaysAgo = restapi.date_to_mil(datetime.datetime.now() - relativedelta(days=3))\nfor log in arcserver.queryLogs(endTime=threeDaysAgo, pageSize=25):\n print(log.time\n for message in log:\n print(message)\n print('\\n')\n```\n\nA note about verbosity\n----------------------\n\nWhen using the admin subpackage you will likely be making changes to services/permissions etc. On operations that change a configuration, the @passthrough decorator will report back if the operation is successful and return results like this:\n\n```py\n{u'status': u'SUCCESS'}\n```\n\nThe printing of these messages can be shut off by changing the global \"VERBOSE\" variable so these messages are not reported. This can be disabled like this:\n\n```py\nadmin.VERBOSE = False \n```\n\n## Advanced Usage\n\n### RequestClient\n\nFrom version 2.0, it is possible to use a custom `requests.Session()` instance. This instance can be defined globally for all requests made by `restapi`, or it can be passed on each function call as a `restapi.RequestClient()` object. This can be useful if different parameters are needed to access different servers.\n\nUse this functionality to access servers behind HTTP or SOCKS proxies, to disable certificate validation or use custom CA certificates, or if additional authentication is needed. Refer to the [requests.Session()](https://requests.readthedocs.io/en/master/user/advanced/#session-objects) documentation for details\n\n```py\n# Create a restapi.RequestClient() object.\ncustom_session = requests.Session()\ncustom_client = restapi.RequestClient(custom_session)\n\n# Customize the client\nproxies = {\n \u201chttp\u201d: \u201chttp://10.10.10.10:8000\u201d,\n \u201chttps\u201d: \u201chttp://10.10.10.10:8000\u201d,\n}\ncustom_client.session.proxies = proxies\ncustom_client.session.headers['Source-Client'] = 'Custom'\n\n# Use the client for an individual call\nrest_url = 'https://gis.ngdc.noaa.gov/arcgis/rest/services'\narcserver = restapi.ArcServer(rest_url, client=custom_client)\n\n# Set a different client as client as restapi's default\nglobal_session = requests.Session()\nglobal_client = restapi.RequestClient(global_session)\nglobal_client.session.headers['Source-Client'] = 'Global'\nrestapi.set_request_client(global_client)\n\n# Now any call made by restapi will use the custom client\narcserver = restapi.ArcServer(rest_url)\n\n# The global client can also be accessed directly\nrestapi.requestClient.headers['Another-Header'] = 'Header is here'\n\n```\n\nAny session objects which extend `requests.Session()` should be supported, for example, [pypac.PACSession()](https://pypi.org/project/pypac/).\n\n## Exceptions\n\nThe module `restapi.exceptions` contains the following custom exceptions, which correspond to the ESRI REST error returns codes specified in the [ESRI documentation](https://developers.arcgis.com/net/reference/platform-error-codes/#http-network-and-rest-errors). Other, previously unknown returns codes will raise the generic `RestAPIException`:\n\n\n\n| code | Exception Class | \n| ------ | ------ |\n| 400 | RestAPIUnableToCompleteOperationException |\n| 401 | RestAPIAuthorizationRequiredException |\n| 403 | RestAPITokenValidAccessDeniedException |\n| 404 | RestAPINotFoundException |\n| 413 | RestAPITooLargeException |\n| 498 | RestAPIInvalidTokenException |\n| 499 | RestAPITokenRequiredException |\n| 500 | RestAPIErrorPerforningOperationException |\n| 501 | RestAPINotImplementedException |\n| 504 | RestAPIGatewayTimeoutException |\n",
"bugtrack_url": null,
"license": "GPL",
"summary": "Package for working with ArcGIS REST API",
"version": "2.4.7",
"project_urls": {
"Homepage": "https://github.com/Bolton-and-Menk-GIS/restapi"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e397c9e55f99ac1289ca68c324957ae85b865de736b6e4894bccd33787040b21",
"md5": "b286c6c84af36da2ac11ba0f8f62605c",
"sha256": "d2920889c2b00b08be543d099a559b70a9aed7ed84941c7445c3a537f5571cb8"
},
"downloads": -1,
"filename": "bmi-arcgis-restapi-2.4.7.tar.gz",
"has_sig": false,
"md5_digest": "b286c6c84af36da2ac11ba0f8f62605c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 747826,
"upload_time": "2024-01-30T14:23:34",
"upload_time_iso_8601": "2024-01-30T14:23:34.788351Z",
"url": "https://files.pythonhosted.org/packages/e3/97/c9e55f99ac1289ca68c324957ae85b865de736b6e4894bccd33787040b21/bmi-arcgis-restapi-2.4.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-30 14:23:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Bolton-and-Menk-GIS",
"github_project": "restapi",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "bmi-arcgis-restapi"
}
JSON
