IpBan: HTTP spam security for Flask
=========================================
|PyPI Version|
IpBan is a Flask extension that can help protect against ip sources spamming url requests
against unknown pages or attempts to exploit URLs. Often this is to search for security issues.
The default configuration:
- 20 attempts before ban
- 1 day blocking period
Once an ip address is banned any attempt to access a web address on your site from that ip will
result in a 403 forbidden status response. After the default 1 day blocking period of no access
attempts the ban will be lifted. Any access attempt during the ban period will extend the ban period
by the `ban_seconds` amount.
Ip addresses can be entered for banning by the api.
Url patterns can be entered to be excluded from ban calculations by the api.
Url patterns can be entered for banning by the api.
Installation & Basic Usage
--------------------------
Install via `pip <https://pypi.python.org/pypi/pip>`_:
::
pip install flask-ipban
After installing, wrap your Flask app with an `IpBan`, or call `ip_ban.init_app(app)`:
.. code:: python
from flask import Flask
from flask_ipban import IpBan
app = Flask(__name__)
ip_ban = IpBan(ban_seconds=200)
ip_ban.init_app(app)
The repository includes a small example application.
Options
-------
- ``app``, Flask application to monitor. Use ip_ban.init_app(app) to intialise later on.
- ``ban_count``, default ``20``, Number of observations before banning.
- ``ban_seconds``, default ``3600*24 (one day)``, Number of seconds ip address is banned.
- ``persist``, default ``False``, Persist ban list between restarts, using records in the ``record_dir`` folder.
- ``record_dir``, default ``None``, (Default TEMP folder). A record directory that stores ban records for ipc sync and persistence.
- ``ipc``, default ``False``, Allow multiple instances of ip_ban to cross communicate using the ``record_dir``.
- ``secret_key``, default ``flask secret key``, Key to sign reports in the ``record_dir``.
- ``ip_header``, default ``None``, Optional name of request header that contains the ip for use behind proxies when in a docker/kube hosted env.
- ``abuse_IPDB_config``, default ``None``, config {key=, report=False, load=False} to a AbuseIPDB.com account. Blocked ip addresses via url nuisance matching will be reported.
Config by env variable overrides options
########################################
These environment variables will override options from the initialisation.
- IP_BAN_LIST_COUNT - number of observations before 403 exception
- IP_BAN_LIST_SECONDS - number of seconds to retain memory of IP
Methods
=======
init_app(app)
-------------
Initialise and start ip_ban with the given Flask application.
block(ip_address, permanent=False)
----------------------------------
Block the specific address, optionally forever
add(ip=None, url=None)
----------------------
increase the observations for the current request ip or given ip address
Example for add:
.. code:: python
from flask import Flask
from flask_ipban import IpBan
app = Flask(__name__)
ip_ban = IpBan(app)
@route('/login', methods=['GET','POST']
def login:
# ....
# increment block if wrong passwords to prevent password stuffing
# ....
if request.method == 'POST':
if request.arg.get('password') != 'secret':
ip_ban.add()
remove(ip_address)
------------------
Remove the given ip address from the ban list. Returns true if ban removed.
url_pattern_add('reg-ex-pattern', match_type='regex')
-----------------------------------------------------
Exclude any url matching the pattern from checking
Example of url_pattern_add:
.. code:: python
from flask import Flask
from flask_ipban import IpBan
app = Flask(__name__)
ip_ban = IpBan(app)
ip_ban.url_pattern_add('^/whitelist$', match_type='regex')
ip_ban.url_pattern_add('/flash/dance', match_type='string')
url_pattern_remove('reg-ex-pattern')
------------------------------------
Remove pattern from the url whitelist
url_block_pattern_add('reg-ex-pattern', match_type='regex')
-----------------------------------------------------------
Add any url matching the pattern to the block list. match_type can be 'string' or 'regex'. String is direct match. Regex is a regex pattern.
url_block_pattern_remove('reg-ex-pattern')
------------------------------------------
Remove pattern from the url block list
ip_whitelist_add('ip-address')
------------------------------
Exclude the given ip from checking
ip_whitelist_remove('ip-address')
---------------------------------
Remove the given ip from the ip whitelist
Example of ip_whitelist_add
.. code:: python
from flask import Flask
from flask_ipban import IpBan
app = Flask(__name__)
ip_ban = IpBan(app)
ip_ban.ip_whitelist_add('127.0.0.1')
load_nuisances(file_name=None)
------------------------------
Add a list of nuisances to url pattern block list from a file. See below for more information.
Example:
.. code:: python
ip_ban = IpBan()
app = Flask(__name__)
ip_ban.init_app(app)
ip_ban.load_nuisances()
load_allowed(file_name=None)
----------------------------
Add a list of allowed patterns from a file. See nuisance for format details.
By default `allowed.yaml` in the ip_ban folder is used. To add to the default patterns supply your own file.
Must be a yaml file with the following example format (which are also the default patterns):
.. code:: yaml
regex:
- ^/\.well-known/
- ^/robots\.txt$
- ^/ads\.txt$
- ^/favicon\.ico$
Example:
.. code:: python
ip_ban = IpBan()
app = Flask(__name__)
ip_ban.init_app(app)
ip_ban.load_allowed()
get_block_list()
----------------
return a copy of the internal block list. Usually will be a dict with the key of `ip` and have
dict values of `count`, `permanent`, `url` and `timestamp`.
- timestamp: datetime object
- count: number of offences
- url: offending url requested
- permanent: bool if ban is permanent
Example:
.. code:: python
s = ''
s += '<table class="table"><thead>\n'
s += '<tr><th>ip</th><th>count</th><th>permanent</th><th>url</th><th>timestamp</th></tr>\n'
s += '</thead><tbody>\n'
for k, r in ip_ban.get_block_list().items():
s += '<tr><td>{}</td><td>{}</td><td>{}</td><td>{}</td><td>{}</td></tr>\n'.format(k, r['count'],
r.get('permanent', ''),
r.get('url', ''),
r['timestamp'])
Url patterns
============
Url matching match_type can be 'string' or 'regex'. String is direct match. Regex is a regex pattern.
Block networks / cidr
=====================
Use the `block_cidr(network)` method to block a range of addresses or whole regions.
Example:
.. code:: python
ip_ban = IpBan()
app = Flask(__name__)
ip_ban.init_app(app)
# block a network in Aruba
ip_ban.block_cidr('190.220.142.104/29')
Nuisance file
=============
ip_ban includes a file of common web nuisances that should not be allowed on a flask site. It includes:
- Blocking any non flask extension such as .jsp, .asp etc.
- Known hacking urls.
Nuisance urls are only checked as a result of a 404. If you have legitimate routes
that use nuisance url patterns they won't result in a block.
Load them by calling ip_ban.load_nuisances()
You can add your own nuisance yaml file by calling with the parameter `file_name`.
See the nuisance.yaml file in the source for formatting and details.
IPC and persistence
===================
When you have multiple applications or processes serving a web application it can be handy to share
any abuse ip between processes. The ipc option allows this.
Set ipc to True to allow writing out each 404/ban event to a file in the ``record_dir`` folder, which has a default in linux of
``/tmp/flask-ip-ban``. This folder has to be writable by the process running your app. Obviously if you use multiple
different apps they can share ip_ban reporting. Each record is signed with the ``secret_key``, so this must be shared
amongst all applications that use the ``record_dir`` folder. The ``secret_key`` is by default the flask secret key.
This folder and secret key is also used by the persistence feature.
Only ip records using the `block`, `add` and `remove` methods or by 404; are persisted or shared. Any whitelisting or
pattern bans are not persisted/shared and must be done for each instance of your application.
The bit that shares ipc records between processes only updates during the `before_request` handler
of the Flask app. It only updates every 5 seconds at the most. If the app does no
request handling between bans then that ban record won't be shared between processes.
IP Header
=========
When running a flask app in a docker hosted environment (or similar) the ip address will be the virtual
adapter ip and won't change for differing requests. Use your proxy server to set the real IP address in a header
so that ip-ban can find what it really is. For apache:
``RequestHeader set X_TRUE_IP "%{REMOTE_ADDR}s"``
``ProxyPass / http://localhost:8080/``
``ProxyPassReverse / http://localhost:8080/``
Then when initializing ip_ban set the header name using the parameter ``ip_header``, in this example: ip_header='X_TRUE_IP'.
Abuse IPDB
==========
see: https://docs.abuseipdb.com/#introduction
You can setup flask-ipban so it will auto report url hacking attempts to the Abuse IPDB. Or you can
load the Abuse IPDB list of blocked ip address on start. Warning! Loading takes a while for the default 10000 records.
*Config*
abuse_IPDB_config = {key=, report=False, load=False, debug=False}
* key - your abuse IPDB api v2 key
* report - True/False (default is False) - report hack attempts to the DB.
* load - True/False (default is False) - load and block already blocked ip addresses from the DB on startup
* debug - True/False (default is False) - debug mode, uses ip 127.0.0.1.
Release History
===============
* 1.0.13 - Remove reason= which did nothing. Add url to report table. Add more nuisances. Add release history.
* 1.1.0 - Add more nuisances. Add ability to block regions by using `block_cidr()`. Remove support for obsolete Python releases (2.7,3.4,3.5).
* 1.1.1 - Fix doco typo.
* 1.1.2 - allow ip as list for ip_whitelist_add()/ip_whitelist_remove().
* 1.1.3 - Fix documentation errors. Add wellknown.yaml and default web URLs commonly used by bots. Remove raise exception for ip abuse db.
* 1.1.4 - Fix missing allowed.yaml in MANIFEST.in
* 1.1.5 - Add new nuisances. Add more allowed. Do not repeat report ips to abuse ip. Use utcnow for timestamps.
Licensing
=========
- Apache 2.0
.. |PyPI Version| image:: https://img.shields.io/pypi/v/flask-ipban.svg
:target: https://pypi.python.org/pypi/flask-ipban
Raw data
{
"_id": null,
"home_page": "https://github.com/Martlark/flask-ipban",
"name": "flask-ipban",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "flask security spam url ban",
"author": "Andrew Rowe",
"author_email": "rowe.andrew.d@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/fc/b4/2cb869ef530ad26828f9f5c1f36380b09406c15456c03449fa6fe2459c49/flask-ipban-1.1.5.tar.gz",
"platform": "",
"description": "IpBan: HTTP spam security for Flask\n=========================================\n\n|PyPI Version|\n\nIpBan is a Flask extension that can help protect against ip sources spamming url requests\nagainst unknown pages or attempts to exploit URLs. Often this is to search for security issues.\n\nThe default configuration:\n\n- 20 attempts before ban\n- 1 day blocking period\n\nOnce an ip address is banned any attempt to access a web address on your site from that ip will\nresult in a 403 forbidden status response. After the default 1 day blocking period of no access\nattempts the ban will be lifted. Any access attempt during the ban period will extend the ban period\nby the `ban_seconds` amount.\n\nIp addresses can be entered for banning by the api.\n\nUrl patterns can be entered to be excluded from ban calculations by the api.\n\nUrl patterns can be entered for banning by the api.\n\nInstallation & Basic Usage\n--------------------------\n\nInstall via `pip <https://pypi.python.org/pypi/pip>`_:\n\n::\n\n pip install flask-ipban\n\nAfter installing, wrap your Flask app with an `IpBan`, or call `ip_ban.init_app(app)`:\n\n.. code:: python\n\n from flask import Flask\n from flask_ipban import IpBan\n\n app = Flask(__name__)\n ip_ban = IpBan(ban_seconds=200)\n ip_ban.init_app(app)\n\n\nThe repository includes a small example application.\n\nOptions\n-------\n\n- ``app``, Flask application to monitor. Use ip_ban.init_app(app) to intialise later on.\n- ``ban_count``, default ``20``, Number of observations before banning.\n- ``ban_seconds``, default ``3600*24 (one day)``, Number of seconds ip address is banned.\n- ``persist``, default ``False``, Persist ban list between restarts, using records in the ``record_dir`` folder.\n- ``record_dir``, default ``None``, (Default TEMP folder). A record directory that stores ban records for ipc sync and persistence.\n- ``ipc``, default ``False``, Allow multiple instances of ip_ban to cross communicate using the ``record_dir``.\n- ``secret_key``, default ``flask secret key``, Key to sign reports in the ``record_dir``.\n- ``ip_header``, default ``None``, Optional name of request header that contains the ip for use behind proxies when in a docker/kube hosted env.\n- ``abuse_IPDB_config``, default ``None``, config {key=, report=False, load=False} to a AbuseIPDB.com account. Blocked ip addresses via url nuisance matching will be reported.\n\nConfig by env variable overrides options\n########################################\n\nThese environment variables will override options from the initialisation.\n\n- IP_BAN_LIST_COUNT - number of observations before 403 exception\n- IP_BAN_LIST_SECONDS - number of seconds to retain memory of IP\n\n\nMethods\n=======\n\ninit_app(app)\n-------------\n\nInitialise and start ip_ban with the given Flask application.\n\nblock(ip_address, permanent=False)\n----------------------------------\n\nBlock the specific address, optionally forever\n\n\nadd(ip=None, url=None)\n----------------------\n\nincrease the observations for the current request ip or given ip address\n\nExample for add:\n\n.. code:: python\n\n from flask import Flask\n from flask_ipban import IpBan\n\n app = Flask(__name__)\n ip_ban = IpBan(app)\n\n @route('/login', methods=['GET','POST']\n def login:\n # ....\n # increment block if wrong passwords to prevent password stuffing\n # ....\n if request.method == 'POST':\n if request.arg.get('password') != 'secret':\n ip_ban.add()\n\nremove(ip_address)\n------------------\nRemove the given ip address from the ban list. Returns true if ban removed.\n\nurl_pattern_add('reg-ex-pattern', match_type='regex')\n-----------------------------------------------------\n\nExclude any url matching the pattern from checking\n\n\nExample of url_pattern_add:\n\n.. code:: python\n\n from flask import Flask\n from flask_ipban import IpBan\n\n app = Flask(__name__)\n ip_ban = IpBan(app)\n ip_ban.url_pattern_add('^/whitelist$', match_type='regex')\n ip_ban.url_pattern_add('/flash/dance', match_type='string')\n\n\nurl_pattern_remove('reg-ex-pattern')\n------------------------------------\n\nRemove pattern from the url whitelist\n\n\nurl_block_pattern_add('reg-ex-pattern', match_type='regex')\n-----------------------------------------------------------\n\nAdd any url matching the pattern to the block list. match_type can be 'string' or 'regex'. String is direct match. Regex is a regex pattern.\n\nurl_block_pattern_remove('reg-ex-pattern')\n------------------------------------------\n\nRemove pattern from the url block list\n\nip_whitelist_add('ip-address')\n------------------------------\n\nExclude the given ip from checking\n\nip_whitelist_remove('ip-address')\n---------------------------------\n\nRemove the given ip from the ip whitelist\n\nExample of ip_whitelist_add\n\n.. code:: python\n\n from flask import Flask\n from flask_ipban import IpBan\n\n app = Flask(__name__)\n ip_ban = IpBan(app)\n ip_ban.ip_whitelist_add('127.0.0.1')\n\n\nload_nuisances(file_name=None)\n------------------------------\nAdd a list of nuisances to url pattern block list from a file. See below for more information.\n\nExample:\n\n.. code:: python\n\n ip_ban = IpBan()\n app = Flask(__name__)\n ip_ban.init_app(app)\n ip_ban.load_nuisances()\n\nload_allowed(file_name=None)\n----------------------------\n\nAdd a list of allowed patterns from a file. See nuisance for format details.\nBy default `allowed.yaml` in the ip_ban folder is used. To add to the default patterns supply your own file.\nMust be a yaml file with the following example format (which are also the default patterns):\n\n\n.. code:: yaml\n\n regex:\n - ^/\\.well-known/\n - ^/robots\\.txt$\n - ^/ads\\.txt$\n - ^/favicon\\.ico$\n\n\n\nExample:\n\n.. code:: python\n\n ip_ban = IpBan()\n app = Flask(__name__)\n ip_ban.init_app(app)\n ip_ban.load_allowed()\n\n\nget_block_list()\n----------------\n\nreturn a copy of the internal block list. Usually will be a dict with the key of `ip` and have\ndict values of `count`, `permanent`, `url` and `timestamp`.\n\n - timestamp: datetime object\n - count: number of offences\n - url: offending url requested\n - permanent: bool if ban is permanent\n\nExample:\n\n.. code:: python\n\n s = ''\n s += '<table class=\"table\"><thead>\\n'\n s += '<tr><th>ip</th><th>count</th><th>permanent</th><th>url</th><th>timestamp</th></tr>\\n'\n s += '</thead><tbody>\\n'\n for k, r in ip_ban.get_block_list().items():\n s += '<tr><td>{}</td><td>{}</td><td>{}</td><td>{}</td><td>{}</td></tr>\\n'.format(k, r['count'],\n r.get('permanent', ''),\n r.get('url', ''),\n r['timestamp'])\n\nUrl patterns\n============\n\nUrl matching match_type can be 'string' or 'regex'. String is direct match. Regex is a regex pattern.\n\nBlock networks / cidr\n=====================\n\nUse the `block_cidr(network)` method to block a range of addresses or whole regions.\n\nExample:\n\n.. code:: python\n\n ip_ban = IpBan()\n app = Flask(__name__)\n ip_ban.init_app(app)\n # block a network in Aruba\n ip_ban.block_cidr('190.220.142.104/29')\n\n\nNuisance file\n=============\n\nip_ban includes a file of common web nuisances that should not be allowed on a flask site. It includes:\n\n- Blocking any non flask extension such as .jsp, .asp etc.\n- Known hacking urls.\n\nNuisance urls are only checked as a result of a 404. If you have legitimate routes\nthat use nuisance url patterns they won't result in a block.\n\nLoad them by calling ip_ban.load_nuisances()\n\nYou can add your own nuisance yaml file by calling with the parameter `file_name`.\n\nSee the nuisance.yaml file in the source for formatting and details.\n\nIPC and persistence\n===================\n\nWhen you have multiple applications or processes serving a web application it can be handy to share\nany abuse ip between processes. The ipc option allows this.\n\nSet ipc to True to allow writing out each 404/ban event to a file in the ``record_dir`` folder, which has a default in linux of\n``/tmp/flask-ip-ban``. This folder has to be writable by the process running your app. Obviously if you use multiple\ndifferent apps they can share ip_ban reporting. Each record is signed with the ``secret_key``, so this must be shared\namongst all applications that use the ``record_dir`` folder. The ``secret_key`` is by default the flask secret key.\n\nThis folder and secret key is also used by the persistence feature.\n\nOnly ip records using the `block`, `add` and `remove` methods or by 404; are persisted or shared. Any whitelisting or\npattern bans are not persisted/shared and must be done for each instance of your application.\n\nThe bit that shares ipc records between processes only updates during the `before_request` handler\nof the Flask app. It only updates every 5 seconds at the most. If the app does no\nrequest handling between bans then that ban record won't be shared between processes.\n\nIP Header\n=========\n\nWhen running a flask app in a docker hosted environment (or similar) the ip address will be the virtual\nadapter ip and won't change for differing requests. Use your proxy server to set the real IP address in a header\nso that ip-ban can find what it really is. For apache:\n\n ``RequestHeader set X_TRUE_IP \"%{REMOTE_ADDR}s\"``\n\n ``ProxyPass / http://localhost:8080/``\n\n ``ProxyPassReverse / http://localhost:8080/``\n\nThen when initializing ip_ban set the header name using the parameter ``ip_header``, in this example: ip_header='X_TRUE_IP'.\n\nAbuse IPDB\n==========\n\nsee: https://docs.abuseipdb.com/#introduction\n\nYou can setup flask-ipban so it will auto report url hacking attempts to the Abuse IPDB. Or you can\nload the Abuse IPDB list of blocked ip address on start. Warning! Loading takes a while for the default 10000 records.\n\n*Config*\n\nabuse_IPDB_config = {key=, report=False, load=False, debug=False}\n\n* key - your abuse IPDB api v2 key\n* report - True/False (default is False) - report hack attempts to the DB.\n* load - True/False (default is False) - load and block already blocked ip addresses from the DB on startup\n* debug - True/False (default is False) - debug mode, uses ip 127.0.0.1.\n\n\nRelease History\n===============\n\n* 1.0.13 - Remove reason= which did nothing. Add url to report table. Add more nuisances. Add release history.\n* 1.1.0 - Add more nuisances. Add ability to block regions by using `block_cidr()`. Remove support for obsolete Python releases (2.7,3.4,3.5).\n* 1.1.1 - Fix doco typo.\n* 1.1.2 - allow ip as list for ip_whitelist_add()/ip_whitelist_remove().\n* 1.1.3 - Fix documentation errors. Add wellknown.yaml and default web URLs commonly used by bots. Remove raise exception for ip abuse db.\n* 1.1.4 - Fix missing allowed.yaml in MANIFEST.in\n* 1.1.5 - Add new nuisances. Add more allowed. Do not repeat report ips to abuse ip. Use utcnow for timestamps.\n\nLicensing\n=========\n\n- Apache 2.0\n\n.. |PyPI Version| image:: https://img.shields.io/pypi/v/flask-ipban.svg\n :target: https://pypi.python.org/pypi/flask-ipban\n\n\n\n",
"bugtrack_url": null,
"license": "Apache Software License",
"summary": "URL spam security for Flask.",
"version": "1.1.5",
"project_urls": {
"Download": "https://github.com/Martlark/flask-ipban/archive/1.1.5.tar.gz",
"Homepage": "https://github.com/Martlark/flask-ipban"
},
"split_keywords": [
"flask",
"security",
"spam",
"url",
"ban"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "65a6828d191a4df151957b59f8470e90ceda6f53854ab1347f69801d13a0f882",
"md5": "55ee524b8db082c0314fc1a0e01861ab",
"sha256": "95ff755d33021e8580629f238820cee5f979c315763f6623d67de3dc10676abb"
},
"downloads": -1,
"filename": "flask_ipban-1.1.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "55ee524b8db082c0314fc1a0e01861ab",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 26717,
"upload_time": "2021-04-14T21:39:50",
"upload_time_iso_8601": "2021-04-14T21:39:50.902782Z",
"url": "https://files.pythonhosted.org/packages/65/a6/828d191a4df151957b59f8470e90ceda6f53854ab1347f69801d13a0f882/flask_ipban-1.1.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fcb42cb869ef530ad26828f9f5c1f36380b09406c15456c03449fa6fe2459c49",
"md5": "1ad54097cfa6fb9f75584890d43ccf87",
"sha256": "5486fef4f0d8d4354a8e9fbc1461bdbbc4c955cc4e42d52b98491a61f30cbedc"
},
"downloads": -1,
"filename": "flask-ipban-1.1.5.tar.gz",
"has_sig": false,
"md5_digest": "1ad54097cfa6fb9f75584890d43ccf87",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 23006,
"upload_time": "2021-04-14T21:39:52",
"upload_time_iso_8601": "2021-04-14T21:39:52.846794Z",
"url": "https://files.pythonhosted.org/packages/fc/b4/2cb869ef530ad26828f9f5c1f36380b09406c15456c03449fa6fe2459c49/flask-ipban-1.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2021-04-14 21:39:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Martlark",
"github_project": "flask-ipban",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "flask-ipban"
}
JSON
