# ICS - Interactive Classification System
The Interactive Classification System (ICS), is a web-based application that supports the activity
of manual text classification, i.e., labeling documents according to their content.
The system is designed to give total freedom of action to its users: they can at any time modify any classification
schema and any label assignment, possibly reusing any relevant information from previous activities.
The application uses machine learning to actively support its users with classification suggestions The machine learning
component of the system is an unobtrusive observer of the users' activities, never interrupting them, constantly
adapting and updating its models in response to their actions, and always available to perform automatic
classifications.
* [Publication](#publication)
* [Installation](#installation)
* [Starting the main app](#startmain)
* [Login](#login)
* [Configuration](#configuration)
* [Additional apps](#apps)
* [Video tutorials](#videos)
* [License](#license)
## <a name="publication"></a> Publication
ICS is described in the paper:
[A. Esuli, "ICS: Total Freedom in Manual Text Classification Supported by Unobtrusive Machine Learning," in IEEE Access, vol. 10, pp. 64741-64760, 2022, doi: 10.1109/ACCESS.2022.3184009](https://doi.org/10.1109/ACCESS.2022.3184009)
## <a name="installation"></a> Installation
You can have a working installation of ICS in many ways:
- [Single file executable](#single-file-executable) (to start using ICS)
- [Docker](#docker) (for a single user)
- [Docker compose](#docker-compose) (for larger installation)
- [Pip install](#pip)
- [From source](#from-source)
### Single file executable
Executable files of ICS are downloadable from the [releases page](https://github.com/aesuli/ics/releases).
Once downloaded it can be run and have a working instance of ICS, provided a [database is configured](#db-configuration).
```shell
ics-webapp
```
The executable are from source using [pyinstaller](https://pyinstaller.org/):
```shell
pyinstaller -F ics\scripts\webapp.py --add-data="ics\apps\media;ics\apps\media" --collect-all sklearn --name ics-webapp
```
### Docker
A quick way have a running instance of ICS is to use [Docker](https://docker.com).
```shell
docker run -p 8080:8080 ghcr.io/aesuli/ics
```
This command pulls the ICS image from Docker hub and runs it, publishing the application on port 8080 of the host machine, accessible from any interface.
Once started ICS is accessible from the host machine using a browser at the address [http://127.0.0.1:8080](http://127.0.0.1:8080)
To have ICS accessible only from the local host machine add local ip address:
```shell
docker run -p 127.0.0.1:8080:8080 ghcr.io/aesuli/ics
```
__NOTE:__ by default the ICS image uses the SQLite database engine, which may result in reduced efficiency and functionalities.
A configuration using PostgreSQL is strongly recommended. It can be easily set up using docker compose.
#### Data persistence
ICS image use volumes to keep information persistent:
- ics-db stores the sqlite file, this is the only volume that should be saved to keep the state of the application.
- ics-data stores the files that are uploaded or downloaded from the system. It is defined for inspection in case of failures, it is not necessary to save it.
- ics-log stores the log files. It is defined for inspection in case of failures, it is not necessary to save it.
### Docker compose
An instance of ICS using PostgreSQL can be obtained downloading the [docker-compose.yml](https://github.com/aesuli/ics/raw/main/docker-compose.yml) file to a local directory and running
```shell
docker compose up
```
from that directory.
#### Host and port
The environment variables ``ICS_HOST`` and ``ICS_PORT`` define the interface and port on which ICS is accessible on the host machine.
Default is 127.0.0.1 and 8080.
#### Data persistence
The compose-based version of ICS use volumes to keep information persistent:
- db-data stores the PostgreSQL, this is the only volume that should be saved to keep the state of the application.
- ics-data stores the files that are uploaded or downloaded from the system. It is defined for inspection in case of failures, it is not necessary to save it.
- ics-log stores the log files. It is defined for inspection in case of failures, it is not necessary to save it.
A volume can be linked to a path on the host machine by defining an environment variable (or by editing the docker-compose.yml file):
- DB_DATA for the db-data volume (recommended)
- ICS_DATA for the ics-data volume (not necessary)
- ICS_LOG for the ics-log volume (not necessary)
For example, on Windows:
```shell
set DB_DATA=D:\ics_db_data
docker compose up
```
On Linux/Mac:
```shell
DB_DATA=/var/lib/ics/data docker compose up
```
### Pip
The suggested way to quickly set up the python environment is to use
the [Anaconda/Miniconda distribution](https://www.anaconda.com/products/distribution) and the `conda` package manager to
create the virtual enviroment.
```shell
conda create -n ics python
conda activate ics
````
ICS is published as a [`pip` package](https://pypi.org/project/ics-pkg).
```shell
pip install ics-pkg
```
The last required step is to [configure a database](#db-configuration).
### From source
Download source code from [GitHub repo](https://github.com/aesuli/ics).
Create a virtual environment and install the required packages.
```shell
cd [directory with ICS code]
conda create -n ics python
conda activate ics
pip install -r requirements.txt
```
The last required step is to [configure a database](#db-configuration).
### DB configuration
The Docker compose installation already includes the setup of the PostgreSQL database, so you can skip this section.
Any another requires to have a database available to connect to.
The use of [PostgreSQL](https://www.postgresql.org/) is strongly recommended.
#### PostgreSQL
To connect to PostgreSQL, a dedicated DB must be created.
These are the SQL commands to create the required user and database on PostgreSQL.
```
CREATE USER ics WITH PASSWORD 'ics';
CREATE DATABASE ics;
GRANT ALL PRIVILEGES ON DATABASE ics to ics;
```
These commands can be issued using the `psql` SQL shell (or using pgAdmin, or similar db frontends).
The tables required by ICS are created automatically at the first run.
Then ICS can be launched passing the DB connection string:
```shell
ics-webapp --db_connection_string postgresql://ics:ics@localhost:5432/ics
```
The above connection string is the correct one for a locally running database, change it according to your configuration.
#### SQLite
By default ICS uses SQLite as the DB, yet please note that **the use of SQLite is intended only for a first exploration of ICS and that using PostgreSQL is strongly recommended**.
Using SQLite can result in reduced efficiency and some functionalities may be missing or not properly working.
To use SQLite use the following ``--db_connection_string`` argument to the launch script:
```shell
ics-webapp --db_connection_string sqlite:///ics.sqlite
```
This is the default connection string, it creates the DB file in the current working directory.
Change it to point to the path where you want to store your file.
Again, PostgreSQL is the recommended database.
## <a name="startmain"></a> The main app
Running the docker image automatically starts the main application, which can be accessed with a browser at the ip and
port defined with the docker launch command or docker compose file.
Installations that do not use docker can run ics by using the ics-webapp script.
Activate the virtual environment:
```shell
conda activate ics
```
When installed using `pip`, the main application can be started with the command:
```shell
ics-webapp
```
When working on source code, it can be launched from the `ics-webapp.py` script:
Linux/Mac:
```shell
PYTHONPATH=. python ics/scripts/ics-webapp.py
```
Windows:
```shell
set PYTHONPATH=.
python ics/scripts/ics-webapp.py
```
When launched, the app will print the URL at which it is accessible.
```
[30/Mar/2022:15:31:59] ENGINE Bus STARTING
[30/Mar/2022:15:31:59] ENGINE Started monitor thread 'Autoreloader'.
[30/Mar/2022:15:31:59] ENGINE Serving on http://127.0.0.1:8080
[30/Mar/2022:15:31:59] ENGINE Bus STARTED
[30/Mar/2022:15:31:59] ENGINE Started monitor thread 'Session cleanup'.
```
## <a name="login"></a> Login
After the installation, only the `admin` user is defined, with password `adminadmin`.
Change the default password on the first run.
## <a name="configuration"></a> Configuration
A configuration for `ics-webapp` can be saved to a file using the `-s` argument with the filename to use. For example,
this command creates a `default.conf` file that lists all the default values (if any other argument is used in the
command, the value of the argument is saved in the configuration file).
```shell
ics-webapp -s default.conf
```
A configuration file can be used to set the launch arguments, using the `-c` argument:
```shell
ics-webapp -c myinstance.conf
```
Any additional argument passed on the command line overrides the one specified in the configuration file.
## <a name="apps"></a> Additional apps
These apps are clients that connect to the ICS web applications.
If you run ICS from Docker you must install them in a local python environment (``pip install ics-pkg``, note that you don't need to set up the DB for them)
If ICS is not running on the local machine with default port, you must use the ``--host [ip address or name]`` and/or the ``--port [number]`` arguments.
### Command line interface
When the ics-webapp is running, ICS can be also accessed from command line
```
> ics-cli
Welcome, type help to have a list of commands
> login admin
Password:
'Ok'
>
```
### Twitter stream collector
A command line app, based on [TwiGet](https://github.com/aesuli/twiget), automatically upload to ICS the tweets
collected from filtered stream queries.
```
> ics-twitter-uploader
Logging into http://127.0.0.1:8080/service/userauth/
Username: admin
Password:
TwiGet 0.1.5
Available commands (type help <command> for details):
create, delete, exit, help, list, refresh, start, stop
Reminder: add -is:retweet to a rule to exclude retweets from results, and to get only original content.
Registered queries:
no registered queries
[not collecting (0 since last start)]>
```
## <a name="videos"></a> Video tutorials
[This YouTube playlist](https://www.youtube.com/playlist?list=PLde6PofTv7SzplW73XNjiS6zyNyDBfsN9) collects videos showing what you can do with ICS.
## <a name="license"></a> License
This software is licensed under the [3-Clause BSD license](https://opensource.org/licenses/BSD-3-Clause) unless
otherwise noted.
Raw data
{
"_id": null,
"home_page": "https://github.com/aesuli/ics",
"name": "ics-pkg",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "text,classification,labeling,machine learning,active learning",
"author": "Andrea Esuli",
"author_email": "andrea@esuli.it",
"download_url": "https://files.pythonhosted.org/packages/11/c4/8c647cf7b326d3a2e765707f1dd0bc83a40e11747384b4fc2886f4b70026/ics-pkg-0.2.0.tar.gz",
"platform": null,
"description": "# ICS - Interactive Classification System\n\nThe Interactive Classification System (ICS), is a web-based application that supports the activity\nof manual text classification, i.e., labeling documents according to their content.\n\nThe system is designed to give total freedom of action to its users: they can at any time modify any classification\nschema and any label assignment, possibly reusing any relevant information from previous activities.\n\nThe application uses machine learning to actively support its users with classification suggestions The machine learning\ncomponent of the system is an unobtrusive observer of the users' activities, never interrupting them, constantly\nadapting and updating its models in response to their actions, and always available to perform automatic\nclassifications.\n\n* [Publication](#publication)\n* [Installation](#installation)\n* [Starting the main app](#startmain)\n* [Login](#login)\n* [Configuration](#configuration)\n* [Additional apps](#apps)\n* [Video tutorials](#videos)\n* [License](#license)\n\n## <a name=\"publication\"></a> Publication\n\nICS is described in the paper:\n\n[A. Esuli, \"ICS: Total Freedom in Manual Text Classification Supported by Unobtrusive Machine Learning,\" in IEEE Access, vol. 10, pp. 64741-64760, 2022, doi: 10.1109/ACCESS.2022.3184009](https://doi.org/10.1109/ACCESS.2022.3184009)\n\n## <a name=\"installation\"></a> Installation\n\nYou can have a working installation of ICS in many ways:\n\n- [Single file executable](#single-file-executable) (to start using ICS)\n- [Docker](#docker) (for a single user)\n- [Docker compose](#docker-compose) (for larger installation)\n- [Pip install](#pip)\n- [From source](#from-source)\n\n### Single file executable\n\nExecutable files of ICS are downloadable from the [releases page](https://github.com/aesuli/ics/releases).\nOnce downloaded it can be run and have a working instance of ICS, provided a [database is configured](#db-configuration).\n\n```shell\nics-webapp\n```\n\n\nThe executable are from source using [pyinstaller](https://pyinstaller.org/):\n\n```shell\npyinstaller -F ics\\scripts\\webapp.py --add-data=\"ics\\apps\\media;ics\\apps\\media\" --collect-all sklearn --name ics-webapp\n```\n\n### Docker\n\nA quick way have a running instance of ICS is to use [Docker](https://docker.com).\n\n```shell\ndocker run -p 8080:8080 ghcr.io/aesuli/ics\n```\n\nThis command pulls the ICS image from Docker hub and runs it, publishing the application on port 8080 of the host machine, accessible from any interface.\nOnce started ICS is accessible from the host machine using a browser at the address [http://127.0.0.1:8080](http://127.0.0.1:8080)\n\nTo have ICS accessible only from the local host machine add local ip address:\n\n```shell\ndocker run -p 127.0.0.1:8080:8080 ghcr.io/aesuli/ics\n```\n\n__NOTE:__ by default the ICS image uses the SQLite database engine, which may result in reduced efficiency and functionalities.\nA configuration using PostgreSQL is strongly recommended. It can be easily set up using docker compose.\n\n#### Data persistence\n\nICS image use volumes to keep information persistent:\n- ics-db stores the sqlite file, this is the only volume that should be saved to keep the state of the application.\n- ics-data stores the files that are uploaded or downloaded from the system. It is defined for inspection in case of failures, it is not necessary to save it.\n- ics-log stores the log files. It is defined for inspection in case of failures, it is not necessary to save it.\n\n### Docker compose\n\nAn instance of ICS using PostgreSQL can be obtained downloading the [docker-compose.yml](https://github.com/aesuli/ics/raw/main/docker-compose.yml) file to a local directory and running\n\n```shell\ndocker compose up\n```\nfrom that directory.\n\n#### Host and port\n\nThe environment variables ``ICS_HOST`` and ``ICS_PORT`` define the interface and port on which ICS is accessible on the host machine.\nDefault is 127.0.0.1 and 8080.\n\n#### Data persistence\n\nThe compose-based version of ICS use volumes to keep information persistent:\n- db-data stores the PostgreSQL, this is the only volume that should be saved to keep the state of the application.\n- ics-data stores the files that are uploaded or downloaded from the system. It is defined for inspection in case of failures, it is not necessary to save it.\n- ics-log stores the log files. It is defined for inspection in case of failures, it is not necessary to save it.\n\nA volume can be linked to a path on the host machine by defining an environment variable (or by editing the docker-compose.yml file):\n- DB_DATA for the db-data volume (recommended)\n- ICS_DATA for the ics-data volume (not necessary)\n- ICS_LOG for the ics-log volume (not necessary)\n\nFor example, on Windows:\n```shell\nset DB_DATA=D:\\ics_db_data\ndocker compose up\n```\n\nOn Linux/Mac:\n```shell\nDB_DATA=/var/lib/ics/data docker compose up\n```\n\n### Pip\n\nThe suggested way to quickly set up the python environment is to use\nthe [Anaconda/Miniconda distribution](https://www.anaconda.com/products/distribution) and the `conda` package manager to\ncreate the virtual enviroment.\n\n```shell\nconda create -n ics python\nconda activate ics\n````\n\nICS is published as a [`pip` package](https://pypi.org/project/ics-pkg).\n\n```shell\npip install ics-pkg\n```\n\nThe last required step is to [configure a database](#db-configuration).\n\n### From source\n\nDownload source code from [GitHub repo](https://github.com/aesuli/ics).\nCreate a virtual environment and install the required packages.\n\n```shell\ncd [directory with ICS code]\nconda create -n ics python\nconda activate ics\npip install -r requirements.txt\n```\n\nThe last required step is to [configure a database](#db-configuration).\n\n### DB configuration\n\nThe Docker compose installation already includes the setup of the PostgreSQL database, so you can skip this section.\nAny another requires to have a database available to connect to.\nThe use of [PostgreSQL](https://www.postgresql.org/) is strongly recommended.\n\n#### PostgreSQL\n\nTo connect to PostgreSQL, a dedicated DB must be created.\nThese are the SQL commands to create the required user and database on PostgreSQL.\n\n```\nCREATE USER ics WITH PASSWORD 'ics';\nCREATE DATABASE ics;\nGRANT ALL PRIVILEGES ON DATABASE ics to ics;\n```\n\nThese commands can be issued using the `psql` SQL shell (or using pgAdmin, or similar db frontends).\nThe tables required by ICS are created automatically at the first run.\n\nThen ICS can be launched passing the DB connection string:\n\n```shell\nics-webapp --db_connection_string postgresql://ics:ics@localhost:5432/ics\n```\n\nThe above connection string is the correct one for a locally running database, change it according to your configuration.\n\n#### SQLite\n\nBy default ICS uses SQLite as the DB, yet please note that **the use of SQLite is intended only for a first exploration of ICS and that using PostgreSQL is strongly recommended**.\nUsing SQLite can result in reduced efficiency and some functionalities may be missing or not properly working.\n\nTo use SQLite use the following ``--db_connection_string`` argument to the launch script:\n\n```shell\nics-webapp --db_connection_string sqlite:///ics.sqlite\n```\n\nThis is the default connection string, it creates the DB file in the current working directory.\nChange it to point to the path where you want to store your file.\n\nAgain, PostgreSQL is the recommended database.\n\n## <a name=\"startmain\"></a> The main app\n\nRunning the docker image automatically starts the main application, which can be accessed with a browser at the ip and \nport defined with the docker launch command or docker compose file.\nInstallations that do not use docker can run ics by using the ics-webapp script.\n\nActivate the virtual environment:\n```shell\nconda activate ics\n```\n\nWhen installed using `pip`, the main application can be started with the command:\n\n```shell\nics-webapp\n```\n\nWhen working on source code, it can be launched from the `ics-webapp.py` script:\n\nLinux/Mac:\n```shell\nPYTHONPATH=. python ics/scripts/ics-webapp.py\n```\nWindows:\n```shell\nset PYTHONPATH=. \npython ics/scripts/ics-webapp.py\n```\n\nWhen launched, the app will print the URL at which it is accessible.\n\n```\n[30/Mar/2022:15:31:59] ENGINE Bus STARTING\n[30/Mar/2022:15:31:59] ENGINE Started monitor thread 'Autoreloader'.\n[30/Mar/2022:15:31:59] ENGINE Serving on http://127.0.0.1:8080\n[30/Mar/2022:15:31:59] ENGINE Bus STARTED\n[30/Mar/2022:15:31:59] ENGINE Started monitor thread 'Session cleanup'.\n```\n\n## <a name=\"login\"></a> Login\n\nAfter the installation, only the `admin` user is defined, with password `adminadmin`.\nChange the default password on the first run.\n\n## <a name=\"configuration\"></a> Configuration\n\nA configuration for `ics-webapp` can be saved to a file using the `-s` argument with the filename to use. For example,\nthis command creates a `default.conf` file that lists all the default values (if any other argument is used in the\ncommand, the value of the argument is saved in the configuration file).\n\n```shell\nics-webapp -s default.conf\n```\n\nA configuration file can be used to set the launch arguments, using the `-c` argument:\n\n```shell\nics-webapp -c myinstance.conf\n```\n\nAny additional argument passed on the command line overrides the one specified in the configuration file.\n\n## <a name=\"apps\"></a> Additional apps\n\nThese apps are clients that connect to the ICS web applications.\n\nIf you run ICS from Docker you must install them in a local python environment (``pip install ics-pkg``, note that you don't need to set up the DB for them)\n\nIf ICS is not running on the local machine with default port, you must use the ``--host [ip address or name]`` and/or the ``--port [number]`` arguments.\n### Command line interface\n\nWhen the ics-webapp is running, ICS can be also accessed from command line\n\n```\n> ics-cli\nWelcome, type help to have a list of commands\n> login admin\nPassword: \n'Ok'\n>\n```\n\n### Twitter stream collector\n\nA command line app, based on [TwiGet](https://github.com/aesuli/twiget), automatically upload to ICS the tweets\ncollected from filtered stream queries.\n\n```\n> ics-twitter-uploader\nLogging into http://127.0.0.1:8080/service/userauth/\nUsername: admin\nPassword: \nTwiGet 0.1.5\n\nAvailable commands (type help <command> for details):\ncreate, delete, exit, help, list, refresh, start, stop\n\nReminder: add -is:retweet to a rule to exclude retweets from results, and to get only original content.\nRegistered queries:\n no registered queries\n\n[not collecting (0 since last start)]>\n```\n\n## <a name=\"videos\"></a> Video tutorials\n\n[This YouTube playlist](https://www.youtube.com/playlist?list=PLde6PofTv7SzplW73XNjiS6zyNyDBfsN9) collects videos showing what you can do with ICS.\n\n## <a name=\"license\"></a> License\n\nThis software is licensed under the [3-Clause BSD license](https://opensource.org/licenses/BSD-3-Clause) unless\notherwise noted.\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "Interactive Classification System (ICS): a tool for machine learning-supported labeling of text",
"version": "0.2.0",
"project_urls": {
"Bug Reports": "https://github.com/aesuli/ics/issues",
"Homepage": "https://github.com/aesuli/ics",
"Source": "https://github.com/aesuli/ics/"
},
"split_keywords": [
"text",
"classification",
"labeling",
"machine learning",
"active learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "801e3b42d83e93e1c6427d2b9b377f463809c39e5ea99e7bff699face45b5bcf",
"md5": "ca07a363b70e295eb49772f0569845c1",
"sha256": "b4b7cf59d631791499f2b801bab36f5c1a067d55f3c48d1e81da31047d28daf2"
},
"downloads": -1,
"filename": "ics_pkg-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ca07a363b70e295eb49772f0569845c1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 117542,
"upload_time": "2023-07-04T07:01:04",
"upload_time_iso_8601": "2023-07-04T07:01:04.761133Z",
"url": "https://files.pythonhosted.org/packages/80/1e/3b42d83e93e1c6427d2b9b377f463809c39e5ea99e7bff699face45b5bcf/ics_pkg-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "11c48c647cf7b326d3a2e765707f1dd0bc83a40e11747384b4fc2886f4b70026",
"md5": "8835e31ca02a4ad889d12ef651877f7a",
"sha256": "aa68d720234bd068a9ffbb3cc2008b8d1ce9d5438ac1ce50500a656297117aea"
},
"downloads": -1,
"filename": "ics-pkg-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "8835e31ca02a4ad889d12ef651877f7a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 90974,
"upload_time": "2023-07-04T07:01:06",
"upload_time_iso_8601": "2023-07-04T07:01:06.074026Z",
"url": "https://files.pythonhosted.org/packages/11/c4/8c647cf7b326d3a2e765707f1dd0bc83a40e11747384b4fc2886f4b70026/ics-pkg-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-04 07:01:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aesuli",
"github_project": "ics",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "ics-pkg"
}
JSON
