[](https://github.com/jung-geun/PSO/actions/workflows/pypi.yml)
[](https://pypi.org/project/pso2keras/)
[](https://sonar.pieroot.xyz/dashboard?id=pieroot_pso_6a2f36a9-2688-4900-a4a5-5be85f36f75a)
[](https://sonar.pieroot.xyz/dashboard?id=pieroot_pso_6a2f36a9-2688-4900-a4a5-5be85f36f75a)
[](https://sonar.pieroot.xyz/dashboard?id=pieroot_pso_6a2f36a9-2688-4900-a4a5-5be85f36f75a)
# PSO
keras model on particle swarm optimization
현재 모델을 python 3.9 버전, tensorflow 2.11 버전에서 테스트 되었습니다
### 목차
> [PSO 알고리즘 구현 및 새로운 시도](#pso-알고리즘-구현-및-새로운-시도)</br>
>
> [초기 세팅 및 사용 방법](#초기-세팅-및-사용-방법)</br>
>
> [구조 및 작동 방식](#구조-및-작동-방식)</br>
>
> [PSO 알고리즘을 이용하여 풀이한 문제들의 정확도](#pso-알고리즘을-이용하여-풀이한-문제들의-정확도)</br>
>
> [참고 자료](#참고-자료)</br>
# PSO 알고리즘 구현 및 새로운 시도
Particle Swarm Optimization on tensorflow package
pso 알고리즘을 사용하여 새로운 학습 방법을 찾는중 입니다</br>
병렬처리로 사용하는 논문을 찾아보았지만 이보다 더 좋은 방법이 있을 것 같아서 찾아보고 있습니다 - [[1]](#참고-자료)</br>
기본 pso 알고리즘의 속도를 구하는 수식은 다음과 같습니다
> $$V_{t+1} = W_t + c_1 * r_1 * (Pbest_t - x_t) + c_2 * r_2 * (Gbest_t - x_t)$$
다음 위치를 업데이트하는 수식입니다
> $$x_{t+1} = x_{t} + V_{t+1}$$
다음과 같은 변수를 사용합니다
> $Pbest_t : 각 파티클의 지역 최적해$</br> $Gbest_t : 전역 최적해$</br> $W_t : 가중치$</br> $c_1, c_2 : 파라미터$</br> $r_1, r_2 : 랜덤 값$</br> $x_t : 현재 위치$</br> $V_{(t+1)} : 다음 속도$</br>
pso 알고리즘을 이용하여 keras 모델을 학습하는 방법을 탐구하고 있습니다</br>
현재는 xor, iris, mnist 문제를 풀어보았으며, xor 문제와 iris 문제는 100%의 정확도를 보이고 있습니다</br>
mnist 문제는 63%의 정확도를 보이고 있습니다</br>
[xor](#1-xor-문제) </br> [iris](#2-iris-문제) </br> [mnist](#3-mnist-문제)
# 초기 세팅 및 사용 방법
자동으로 conda 환경을 설정하기 위해서는 다음 명령어를 사용합니다
```shell
conda env create -f conda_env/environment.yaml
```
현재 python 3.9 버전, tensorflow 2.11 버전에서 테스트 되었습니다
</br>
직접 설치하여 사용할 경우 pso2keras 패키지를 pypi 에서 다운로드 받아서 사용하시기 바랍니다
```shell
pip install pso2keras
```
위의 패키지를 사용하기 위해서는 tensorflow 와 tensorboard 가 설치되어 있어야 합니다
python 패키지를 사용하기 위한 라이브러리는 아래 코드를 사용합니다
```python
from pso import Optimizer
pso_model = Optimizer(...)
pso_model.fit(...)
```
<a href="https://colab.research.google.com/github/jung-geun/PSO/blob/master/example/pso2mnist.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
# 구조 및 작동 방식
## 파일 구조
```plain
|-- /conda_env # conda 환경 설정 파일
| |-- environment.yaml # conda 환경 설정 파일
|-- /metacode # pso 기본 코드
| |-- pso_bp.py # 오차역전파 함수를 최적화하는 PSO 알고리즘 구현 - 성능이 99% 이상으로 나오나 목적과 다름
| |-- pso_meta.py # PSO 기본 알고리즘 구현
| |-- pso_tf.py # tensorflow 모델을 이용가능한 PSO 알고리즘 구현
|-- /pso # tensorflow 모델을 학습하기 위해 기본 pso 코드에서 수정 - (psokeras 코드 의 구조를 사용하여 만듬)
| |-- __init__.py # pso 모듈을 사용하기 위한 초기화 파일
| |-- optimizer.py # pso 알고리즘 이용을 위한 기본 코드
| |-- particle.py # 각 파티클의 정보 및 위치를 저장하는 코드
|-- xor.py # pso 를 이용한 xor 문제 풀이
|-- iris.py # pso 를 이용한 iris 문제 풀이
|-- iris_tf.py # tensorflow 를 이용한 iris 문제 풀이
|-- mnist.py # pso 를 이용한 mnist 문제 풀이
|-- mnist_tf.py # tensorflow 를 이용한 mnist 문제 풀이
|-- plt.ipynb # pyplot 으로 학습 결과를 그래프로 표현
|-- README.md # 현재 파일
|-- requirements.txt # pypi 에서 다운로드 받을 패키지 목록
```
pso 라이브러리는 tensorflow 모델을 학습하기 위해 기본 ./metacode/pso_meta.py 코드에서 수정하였습니다 [[2]](#참고-자료)
pso 알고리즘을 이용하여 오차역전파 함수를 최적화 하는 방법을 찾는 중입니다
## 알고리즘 작동 방식
> 1. 파티클의 위치와 속도를 초기화 한다.
> 2. 각 파티클의 점수를 계산한다.
> 3. 각 파티클의 지역 최적해와 전역 최적해를 구한다.
> 4. 각 파티클의 속도를 업데이트 한다.
# PSO 알고리즘을 이용하여 풀이한 문제들의 정확도
## 1. xor 문제
```python
loss = 'mean_squared_error'
pso_xor = Optimizer(
model,
loss=loss,
n_particles=50,
c0=0.35,
c1=0.8,
w_min=0.6,
w_max=1.2,
negative_swarm=0.1,
mutation_swarm=0.2,
particle_min=-3,
particle_max=3,
)
best_score = pso_xor.fit(
x_test,
y_test,
epochs=200,
save=True,
save_path="./result/xor",
renewal="acc",
empirical_balance=False,
Dispersion=False,
check_point=25,
)
```
위의 파라미터 기준 10 세대 근처부터 정확도가 100%가 나오는 것을 확인하였습니다
## 2. iris 문제
```python
loss = 'mean_squared_error'
pso_iris = Optimizer(
model,
loss=loss,
n_particles=100,
c0=0.35,
c1=0.7,
w_min=0.5,
w_max=0.9,
negative_swarm=0.1,
mutation_swarm=0.2,
particle_min=-3,
particle_max=3,
)
best_score = pso_iris.fit(
x_train,
y_train,
epochs=200,
save=True,
save_path="./result/iris",
renewal="acc",
empirical_balance=False,
Dispersion=False,
check_point=25
)
```
위의 파라미터 기준 7 세대에 97%, 35 세대에 99.16%의 정확도를 보였습니다
위의 그래프를 보면 epochs 이 늘어나도 정확도와 loss 가 수렴하지 않는것을 보면 파라미터의 이동 속도가 너무 빠르다고 생각합니다
## 3. mnist 문제
```python
loss = 'mean_squared_error'
pso_mnist = Optimizer(
model,
loss=loss,
n_particles=500,
c0= 0.4,
c1= 0.6,
w_min= 0.5,
w_max= 0.8,
negative_swarm=0.1,
mutation_swarm=0.2,
particle_min=-5,
particle_max=5,
)
best_score = pso_mnist.fit(
x_train,
y_train,
epochs=200,
save_info=True,
log=2,
log_name="mnist",
save_path="./result/mnist",
renewal="acc",
check_point=25,
)
```
위의 파라미터 기준 현재 정확도 63.84%를 보이고 있습니다
63%의 정확도가 나타나는 것으로 보아 최적화가 되어가고 있다고 볼 수 있을 것 같습니다.
하지만 정확도가 더 이상 올라가지 않고 정체되는 것으로 보아 조기 수렴하는 문제가 발생하고 있다고 생각합니다.
## Trouble Shooting
> 1. 딥러닝 알고리즘 특성상 weights는 처음 컴파일시 무작위하게 생성된다. weights의 각 지점의 중요도는 매번 무작위로 정해지기에 전역 최적값으로 찾아갈 때 값이 높은 loss를 향해서 상승하는 현상이 나타난다.
>
> > 따라서 weights의 이동 방법을 더 탐구하거나, weights를 초기화 할때 random 중요도를 좀더 노이즈가 적게 생성하는 방향을 모색해야할 것 같다.
-> 고르게 초기화 하기 위해 np.random.uniform 함수를 사용하였습니다
> 2. 지역최적값에 계속 머무르는 조기 수렴 현상이 나타난다. - 30% 정도의 정확도를 가진다
-> 지역최적값에 머무르는 것을 방지하기 위해 negative_swarm, mutation_swarm 파라미터를 추가하였습니다 - 현재 63% 정도의 정확도를 보이고 있습니다
> 3. 파티클의 수를 늘리면 전역 최적해에 좀더 가까워지는 현상을 발견하였다. 하지만 파티클의 수를 늘리면 메모리 사용량이 기하급수적으로 늘어난다.
-> keras 모델을 사용할때 predict, evaluate 함수를 사용하면 메모리 누수가 발생하는 문제를 찾았습니다. 해결방법을 추가로 찾아보는중 입니다. -> 메모리 누수를 획기적으로 줄여 현재는 파티클의 수를 500개에서 1000개까지 증가시켜도 문제가 없습니다.</br>
-> 추가로 파티클의 수가 적을때에도 전역 최적해를 쉽게 찾는 방법을 찾는중 입니다</br>
> 4. 현재 tensorboard 로 로그 저장시 994개 이상 저장이 안되는 문제가 발생하고 있습니다.
-> csv 파일로 저장할 경우 갯수에는 문제가 발생하지 않습니다.
-> 수가 적을때 한 파티클이 지역 최적해에서 머무를 경우 파티클을 초기화 하는 방법이 필요해 보입니다.
> 5. 모델의 크기가 커지면 수렴이 늦어지고 정확도가 떨어지는 현상이 발견되었다. 모델의 크기에 맞는 파라미터를 찾아야할 것 같다.
> 6. EBPSO 의 방식을 추가로 적용을 하였으나 수식을 잘못 적용을 한것인지 기본 pso 보다 더 떨어지는 정확도를 보이고 있다. (현재 수정중)
### 개인적인 생각
> 머신러닝 분류 방식에 존재하는 random forest 방식을 이용하여, 오차역전파 함수를 최적화 하는 방법이 있을것 같습니다
>
> > pso 와 random forest 방식이 매우 유사하다고 생각하여 학습할 때 뿐만 아니라 예측 할 때도 이러한 방식으로 사용할 수 있을 것 같습니다
# 참고 자료
[1]: [A partilce swarm optimization algorithm with empirical balance stategy](https://www.sciencedirect.com/science/article/pii/S2590054422000185#bib0005) </br>
[2]: [psokeras](https://github.com/mike-holcomb/PSOkeras) </br>
[3]: [PSO의 다양한 영역 탐색과 지역적 미니멈 인식을 위한 전략](https://koreascience.kr/article/JAKO200925836515680.pdf) </br>
[4]: [PC 클러스터 기반의 Multi-HPSO를 이용한 안전도 제약의 경제 급전](https://koreascience.kr/article/JAKO200932056732373.pdf) </br>
[5]: [Particle 2-Swarm Optimization for Robust Search](https://s-space.snu.ac.kr/bitstream/10371/29949/3/management_information_v18_01_p01.pdf) </br>
Raw data
{
"_id": null,
"home_page": "https://github.com/jung-geun/PSO",
"name": "pso2keras",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "pso,tensorflow,keras,optimization,particle swarm optimization,pso2keras",
"author": "pieroot",
"author_email": "jgbong0306@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/dc/87/7cb6d79c13c5c28a36a589fe8b0e3934836df407aa540a05d81f17cd191a/pso2keras-1.0.5.1.tar.gz",
"platform": null,
"description": "[](https://github.com/jung-geun/PSO/actions/workflows/pypi.yml)\n[](https://pypi.org/project/pso2keras/)\n[](https://sonar.pieroot.xyz/dashboard?id=pieroot_pso_6a2f36a9-2688-4900-a4a5-5be85f36f75a)\n[](https://sonar.pieroot.xyz/dashboard?id=pieroot_pso_6a2f36a9-2688-4900-a4a5-5be85f36f75a)\n[](https://sonar.pieroot.xyz/dashboard?id=pieroot_pso_6a2f36a9-2688-4900-a4a5-5be85f36f75a)\n\n# PSO\n\nkeras model on particle swarm optimization\n\n\ud604\uc7ac \ubaa8\ub378\uc744 python 3.9 \ubc84\uc804, tensorflow 2.11 \ubc84\uc804\uc5d0\uc11c \ud14c\uc2a4\ud2b8 \ub418\uc5c8\uc2b5\ub2c8\ub2e4\n\n### \ubaa9\ucc28\n\n> [PSO \uc54c\uace0\ub9ac\uc998 \uad6c\ud604 \ubc0f \uc0c8\ub85c\uc6b4 \uc2dc\ub3c4](#pso-\uc54c\uace0\ub9ac\uc998-\uad6c\ud604-\ubc0f-\uc0c8\ub85c\uc6b4-\uc2dc\ub3c4)</br>\n>\n> [\ucd08\uae30 \uc138\ud305 \ubc0f \uc0ac\uc6a9 \ubc29\ubc95](#\ucd08\uae30-\uc138\ud305-\ubc0f-\uc0ac\uc6a9-\ubc29\ubc95)</br>\n>\n> [\uad6c\uc870 \ubc0f \uc791\ub3d9 \ubc29\uc2dd](#\uad6c\uc870-\ubc0f-\uc791\ub3d9-\ubc29\uc2dd)</br>\n>\n> [PSO \uc54c\uace0\ub9ac\uc998\uc744 \uc774\uc6a9\ud558\uc5ec \ud480\uc774\ud55c \ubb38\uc81c\ub4e4\uc758 \uc815\ud655\ub3c4](#pso-\uc54c\uace0\ub9ac\uc998\uc744-\uc774\uc6a9\ud558\uc5ec-\ud480\uc774\ud55c-\ubb38\uc81c\ub4e4\uc758-\uc815\ud655\ub3c4)</br>\n>\n> [\ucc38\uace0 \uc790\ub8cc](#\ucc38\uace0-\uc790\ub8cc)</br>\n\n# PSO \uc54c\uace0\ub9ac\uc998 \uad6c\ud604 \ubc0f \uc0c8\ub85c\uc6b4 \uc2dc\ub3c4\n\nParticle Swarm Optimization on tensorflow package\n\npso \uc54c\uace0\ub9ac\uc998\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc0c8\ub85c\uc6b4 \ud559\uc2b5 \ubc29\ubc95\uc744 \ucc3e\ub294\uc911 \uc785\ub2c8\ub2e4</br>\n\ubcd1\ub82c\ucc98\ub9ac\ub85c \uc0ac\uc6a9\ud558\ub294 \ub17c\ubb38\uc744 \ucc3e\uc544\ubcf4\uc558\uc9c0\ub9cc \uc774\ubcf4\ub2e4 \ub354 \uc88b\uc740 \ubc29\ubc95\uc774 \uc788\uc744 \uac83 \uac19\uc544\uc11c \ucc3e\uc544\ubcf4\uace0 \uc788\uc2b5\ub2c8\ub2e4 - [[1]](#\ucc38\uace0-\uc790\ub8cc)</br>\n\n\uae30\ubcf8 pso \uc54c\uace0\ub9ac\uc998\uc758 \uc18d\ub3c4\ub97c \uad6c\ud558\ub294 \uc218\uc2dd\uc740 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4\n\n> $$V_{t+1} = W_t + c_1 * r_1 * (Pbest_t - x_t) + c_2 * r_2 * (Gbest_t - x_t)$$\n\n\ub2e4\uc74c \uc704\uce58\ub97c \uc5c5\ub370\uc774\ud2b8\ud558\ub294 \uc218\uc2dd\uc785\ub2c8\ub2e4\n\n> $$x_{t+1} = x_{t} + V_{t+1}$$\n\n\ub2e4\uc74c\uacfc \uac19\uc740 \ubcc0\uc218\ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4\n\n> $Pbest_t : \uac01 \ud30c\ud2f0\ud074\uc758 \uc9c0\uc5ed \ucd5c\uc801\ud574$</br> $Gbest_t : \uc804\uc5ed \ucd5c\uc801\ud574$</br> $W_t : \uac00\uc911\uce58$</br> $c_1, c_2 : \ud30c\ub77c\ubbf8\ud130$</br> $r_1, r_2 : \ub79c\ub364 \uac12$</br> $x_t : \ud604\uc7ac \uc704\uce58$</br> $V_{(t+1)} : \ub2e4\uc74c \uc18d\ub3c4$</br>\n\npso \uc54c\uace0\ub9ac\uc998\uc744 \uc774\uc6a9\ud558\uc5ec keras \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\ub294 \ubc29\ubc95\uc744 \ud0d0\uad6c\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4</br>\n\ud604\uc7ac\ub294 xor, iris, mnist \ubb38\uc81c\ub97c \ud480\uc5b4\ubcf4\uc558\uc73c\uba70, xor \ubb38\uc81c\uc640 iris \ubb38\uc81c\ub294 100%\uc758 \uc815\ud655\ub3c4\ub97c \ubcf4\uc774\uace0 \uc788\uc2b5\ub2c8\ub2e4</br>\nmnist \ubb38\uc81c\ub294 63%\uc758 \uc815\ud655\ub3c4\ub97c \ubcf4\uc774\uace0 \uc788\uc2b5\ub2c8\ub2e4</br>\n\n[xor](#1-xor-\ubb38\uc81c) </br> [iris](#2-iris-\ubb38\uc81c) </br> [mnist](#3-mnist-\ubb38\uc81c)\n\n# \ucd08\uae30 \uc138\ud305 \ubc0f \uc0ac\uc6a9 \ubc29\ubc95\n\n\uc790\ub3d9\uc73c\ub85c conda \ud658\uacbd\uc744 \uc124\uc815\ud558\uae30 \uc704\ud574\uc11c\ub294 \ub2e4\uc74c \uba85\ub839\uc5b4\ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4\n\n```shell\nconda env create -f conda_env/environment.yaml\n```\n\n\ud604\uc7ac python 3.9 \ubc84\uc804, tensorflow 2.11 \ubc84\uc804\uc5d0\uc11c \ud14c\uc2a4\ud2b8 \ub418\uc5c8\uc2b5\ub2c8\ub2e4\n</br>\n\uc9c1\uc811 \uc124\uce58\ud558\uc5ec \uc0ac\uc6a9\ud560 \uacbd\uc6b0 pso2keras \ud328\ud0a4\uc9c0\ub97c pypi \uc5d0\uc11c \ub2e4\uc6b4\ub85c\ub4dc \ubc1b\uc544\uc11c \uc0ac\uc6a9\ud558\uc2dc\uae30 \ubc14\ub78d\ub2c8\ub2e4\n\n```shell\npip install pso2keras\n```\n\n\uc704\uc758 \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uae30 \uc704\ud574\uc11c\ub294 tensorflow \uc640 tensorboard \uac00 \uc124\uce58\ub418\uc5b4 \uc788\uc5b4\uc57c \ud569\ub2c8\ub2e4\n\npython \ud328\ud0a4\uc9c0\ub97c \uc0ac\uc6a9\ud558\uae30 \uc704\ud55c \ub77c\uc774\ube0c\ub7ec\ub9ac\ub294 \uc544\ub798 \ucf54\ub4dc\ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4\n\n```python\nfrom pso import Optimizer\n\npso_model = Optimizer(...)\npso_model.fit(...)\n```\n\n<a href=\"https://colab.research.google.com/github/jung-geun/PSO/blob/master/example/pso2mnist.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n\n# \uad6c\uc870 \ubc0f \uc791\ub3d9 \ubc29\uc2dd\n\n## \ud30c\uc77c \uad6c\uc870\n\n```plain\n|-- /conda_env # conda \ud658\uacbd \uc124\uc815 \ud30c\uc77c\n| |-- environment.yaml # conda \ud658\uacbd \uc124\uc815 \ud30c\uc77c\n|-- /metacode # pso \uae30\ubcf8 \ucf54\ub4dc\n| |-- pso_bp.py # \uc624\ucc28\uc5ed\uc804\ud30c \ud568\uc218\ub97c \ucd5c\uc801\ud654\ud558\ub294 PSO \uc54c\uace0\ub9ac\uc998 \uad6c\ud604 - \uc131\ub2a5\uc774 99% \uc774\uc0c1\uc73c\ub85c \ub098\uc624\ub098 \ubaa9\uc801\uacfc \ub2e4\ub984\n| |-- pso_meta.py # PSO \uae30\ubcf8 \uc54c\uace0\ub9ac\uc998 \uad6c\ud604\n| |-- pso_tf.py # tensorflow \ubaa8\ub378\uc744 \uc774\uc6a9\uac00\ub2a5\ud55c PSO \uc54c\uace0\ub9ac\uc998 \uad6c\ud604\n|-- /pso # tensorflow \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\uae30 \uc704\ud574 \uae30\ubcf8 pso \ucf54\ub4dc\uc5d0\uc11c \uc218\uc815 - (psokeras \ucf54\ub4dc \uc758 \uad6c\uc870\ub97c \uc0ac\uc6a9\ud558\uc5ec \ub9cc\ub4ec)\n| |-- __init__.py # pso \ubaa8\ub4c8\uc744 \uc0ac\uc6a9\ud558\uae30 \uc704\ud55c \ucd08\uae30\ud654 \ud30c\uc77c\n| |-- optimizer.py # pso \uc54c\uace0\ub9ac\uc998 \uc774\uc6a9\uc744 \uc704\ud55c \uae30\ubcf8 \ucf54\ub4dc\n| |-- particle.py # \uac01 \ud30c\ud2f0\ud074\uc758 \uc815\ubcf4 \ubc0f \uc704\uce58\ub97c \uc800\uc7a5\ud558\ub294 \ucf54\ub4dc\n|-- xor.py # pso \ub97c \uc774\uc6a9\ud55c xor \ubb38\uc81c \ud480\uc774\n|-- iris.py # pso \ub97c \uc774\uc6a9\ud55c iris \ubb38\uc81c \ud480\uc774\n|-- iris_tf.py # tensorflow \ub97c \uc774\uc6a9\ud55c iris \ubb38\uc81c \ud480\uc774\n|-- mnist.py # pso \ub97c \uc774\uc6a9\ud55c mnist \ubb38\uc81c \ud480\uc774\n|-- mnist_tf.py # tensorflow \ub97c \uc774\uc6a9\ud55c mnist \ubb38\uc81c \ud480\uc774\n|-- plt.ipynb # pyplot \uc73c\ub85c \ud559\uc2b5 \uacb0\uacfc\ub97c \uadf8\ub798\ud504\ub85c \ud45c\ud604\n|-- README.md # \ud604\uc7ac \ud30c\uc77c\n|-- requirements.txt # pypi \uc5d0\uc11c \ub2e4\uc6b4\ub85c\ub4dc \ubc1b\uc744 \ud328\ud0a4\uc9c0 \ubaa9\ub85d\n```\n\npso \ub77c\uc774\ube0c\ub7ec\ub9ac\ub294 tensorflow \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\uae30 \uc704\ud574 \uae30\ubcf8 ./metacode/pso_meta.py \ucf54\ub4dc\uc5d0\uc11c \uc218\uc815\ud558\uc600\uc2b5\ub2c8\ub2e4 [[2]](#\ucc38\uace0-\uc790\ub8cc)\n\npso \uc54c\uace0\ub9ac\uc998\uc744 \uc774\uc6a9\ud558\uc5ec \uc624\ucc28\uc5ed\uc804\ud30c \ud568\uc218\ub97c \ucd5c\uc801\ud654 \ud558\ub294 \ubc29\ubc95\uc744 \ucc3e\ub294 \uc911\uc785\ub2c8\ub2e4\n\n## \uc54c\uace0\ub9ac\uc998 \uc791\ub3d9 \ubc29\uc2dd\n\n> 1. \ud30c\ud2f0\ud074\uc758 \uc704\uce58\uc640 \uc18d\ub3c4\ub97c \ucd08\uae30\ud654 \ud55c\ub2e4.\n> 2. \uac01 \ud30c\ud2f0\ud074\uc758 \uc810\uc218\ub97c \uacc4\uc0b0\ud55c\ub2e4.\n> 3. \uac01 \ud30c\ud2f0\ud074\uc758 \uc9c0\uc5ed \ucd5c\uc801\ud574\uc640 \uc804\uc5ed \ucd5c\uc801\ud574\ub97c \uad6c\ud55c\ub2e4.\n> 4. \uac01 \ud30c\ud2f0\ud074\uc758 \uc18d\ub3c4\ub97c \uc5c5\ub370\uc774\ud2b8 \ud55c\ub2e4.\n\n# PSO \uc54c\uace0\ub9ac\uc998\uc744 \uc774\uc6a9\ud558\uc5ec \ud480\uc774\ud55c \ubb38\uc81c\ub4e4\uc758 \uc815\ud655\ub3c4\n\n## 1. xor \ubb38\uc81c\n\n```python\nloss = 'mean_squared_error'\n\npso_xor = Optimizer(\n model,\n loss=loss,\n n_particles=50,\n c0=0.35,\n c1=0.8,\n w_min=0.6,\n w_max=1.2,\n negative_swarm=0.1,\n mutation_swarm=0.2,\n particle_min=-3,\n particle_max=3,\n)\n\nbest_score = pso_xor.fit(\n x_test,\n y_test,\n epochs=200,\n save=True,\n save_path=\"./result/xor\",\n renewal=\"acc\",\n empirical_balance=False,\n Dispersion=False,\n check_point=25,\n)\n```\n\n\uc704\uc758 \ud30c\ub77c\ubbf8\ud130 \uae30\uc900 10 \uc138\ub300 \uadfc\ucc98\ubd80\ud130 \uc815\ud655\ub3c4\uac00 100%\uac00 \ub098\uc624\ub294 \uac83\uc744 \ud655\uc778\ud558\uc600\uc2b5\ub2c8\ub2e4\n\n\n\n## 2. iris \ubb38\uc81c\n\n```python\nloss = 'mean_squared_error'\n\npso_iris = Optimizer(\n model,\n loss=loss,\n n_particles=100,\n c0=0.35,\n c1=0.7,\n w_min=0.5,\n w_max=0.9,\n negative_swarm=0.1,\n mutation_swarm=0.2,\n particle_min=-3,\n particle_max=3,\n)\n\nbest_score = pso_iris.fit(\n x_train,\n y_train,\n epochs=200,\n save=True,\n save_path=\"./result/iris\",\n renewal=\"acc\",\n empirical_balance=False,\n Dispersion=False,\n check_point=25\n)\n```\n\n\uc704\uc758 \ud30c\ub77c\ubbf8\ud130 \uae30\uc900 7 \uc138\ub300\uc5d0 97%, 35 \uc138\ub300\uc5d0 99.16%\uc758 \uc815\ud655\ub3c4\ub97c \ubcf4\uc600\uc2b5\ub2c8\ub2e4\n\n\n\n\uc704\uc758 \uadf8\ub798\ud504\ub97c \ubcf4\uba74 epochs \uc774 \ub298\uc5b4\ub098\ub3c4 \uc815\ud655\ub3c4\uc640 loss \uac00 \uc218\ub834\ud558\uc9c0 \uc54a\ub294\uac83\uc744 \ubcf4\uba74 \ud30c\ub77c\ubbf8\ud130\uc758 \uc774\ub3d9 \uc18d\ub3c4\uac00 \ub108\ubb34 \ube60\ub974\ub2e4\uace0 \uc0dd\uac01\ud569\ub2c8\ub2e4\n\n## 3. mnist \ubb38\uc81c\n\n```python\nloss = 'mean_squared_error'\n\npso_mnist = Optimizer(\n model,\n loss=loss,\n n_particles=500,\n c0= 0.4,\n c1= 0.6,\n w_min= 0.5,\n w_max= 0.8,\n negative_swarm=0.1,\n mutation_swarm=0.2,\n particle_min=-5,\n particle_max=5,\n)\n\nbest_score = pso_mnist.fit(\n x_train,\n y_train,\n epochs=200,\n save_info=True,\n log=2,\n log_name=\"mnist\",\n save_path=\"./result/mnist\",\n renewal=\"acc\",\n check_point=25,\n)\n```\n\n\uc704\uc758 \ud30c\ub77c\ubbf8\ud130 \uae30\uc900 \ud604\uc7ac \uc815\ud655\ub3c4 63.84%\ub97c \ubcf4\uc774\uace0 \uc788\uc2b5\ub2c8\ub2e4\n\n\n\n\n\n63%\uc758 \uc815\ud655\ub3c4\uac00 \ub098\ud0c0\ub098\ub294 \uac83\uc73c\ub85c \ubcf4\uc544 \ucd5c\uc801\ud654\uac00 \ub418\uc5b4\uac00\uace0 \uc788\ub2e4\uace0 \ubcfc \uc218 \uc788\uc744 \uac83 \uac19\uc2b5\ub2c8\ub2e4.\n\n\ud558\uc9c0\ub9cc \uc815\ud655\ub3c4\uac00 \ub354 \uc774\uc0c1 \uc62c\ub77c\uac00\uc9c0 \uc54a\uace0 \uc815\uccb4\ub418\ub294 \uac83\uc73c\ub85c \ubcf4\uc544 \uc870\uae30 \uc218\ub834\ud558\ub294 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud558\uace0 \uc788\ub2e4\uace0 \uc0dd\uac01\ud569\ub2c8\ub2e4.\n\n## Trouble Shooting\n\n> 1. \ub525\ub7ec\ub2dd \uc54c\uace0\ub9ac\uc998 \ud2b9\uc131\uc0c1 weights\ub294 \ucc98\uc74c \ucef4\ud30c\uc77c\uc2dc \ubb34\uc791\uc704\ud558\uac8c \uc0dd\uc131\ub41c\ub2e4. weights\uc758 \uac01 \uc9c0\uc810\uc758 \uc911\uc694\ub3c4\ub294 \ub9e4\ubc88 \ubb34\uc791\uc704\ub85c \uc815\ud574\uc9c0\uae30\uc5d0 \uc804\uc5ed \ucd5c\uc801\uac12\uc73c\ub85c \ucc3e\uc544\uac08 \ub54c \uac12\uc774 \ub192\uc740 loss\ub97c \ud5a5\ud574\uc11c \uc0c1\uc2b9\ud558\ub294 \ud604\uc0c1\uc774 \ub098\ud0c0\ub09c\ub2e4.\n>\n> > \ub530\ub77c\uc11c weights\uc758 \uc774\ub3d9 \ubc29\ubc95\uc744 \ub354 \ud0d0\uad6c\ud558\uac70\ub098, weights\ub97c \ucd08\uae30\ud654 \ud560\ub54c random \uc911\uc694\ub3c4\ub97c \uc880\ub354 \ub178\uc774\uc988\uac00 \uc801\uac8c \uc0dd\uc131\ud558\ub294 \ubc29\ud5a5\uc744 \ubaa8\uc0c9\ud574\uc57c\ud560 \uac83 \uac19\ub2e4.\n\n-> \uace0\ub974\uac8c \ucd08\uae30\ud654 \ud558\uae30 \uc704\ud574 np.random.uniform \ud568\uc218\ub97c \uc0ac\uc6a9\ud558\uc600\uc2b5\ub2c8\ub2e4\n\n> 2. \uc9c0\uc5ed\ucd5c\uc801\uac12\uc5d0 \uacc4\uc18d \uba38\ubb34\ub974\ub294 \uc870\uae30 \uc218\ub834 \ud604\uc0c1\uc774 \ub098\ud0c0\ub09c\ub2e4. - 30% \uc815\ub3c4\uc758 \uc815\ud655\ub3c4\ub97c \uac00\uc9c4\ub2e4\n\n-> \uc9c0\uc5ed\ucd5c\uc801\uac12\uc5d0 \uba38\ubb34\ub974\ub294 \uac83\uc744 \ubc29\uc9c0\ud558\uae30 \uc704\ud574 negative_swarm, mutation_swarm \ud30c\ub77c\ubbf8\ud130\ub97c \ucd94\uac00\ud558\uc600\uc2b5\ub2c8\ub2e4 - \ud604\uc7ac 63% \uc815\ub3c4\uc758 \uc815\ud655\ub3c4\ub97c \ubcf4\uc774\uace0 \uc788\uc2b5\ub2c8\ub2e4\n\n> 3. \ud30c\ud2f0\ud074\uc758 \uc218\ub97c \ub298\ub9ac\uba74 \uc804\uc5ed \ucd5c\uc801\ud574\uc5d0 \uc880\ub354 \uac00\uae4c\uc6cc\uc9c0\ub294 \ud604\uc0c1\uc744 \ubc1c\uacac\ud558\uc600\ub2e4. \ud558\uc9c0\ub9cc \ud30c\ud2f0\ud074\uc758 \uc218\ub97c \ub298\ub9ac\uba74 \uba54\ubaa8\ub9ac \uc0ac\uc6a9\ub7c9\uc774 \uae30\ud558\uae09\uc218\uc801\uc73c\ub85c \ub298\uc5b4\ub09c\ub2e4.\n\n-> keras \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud560\ub54c predict, evaluate \ud568\uc218\ub97c \uc0ac\uc6a9\ud558\uba74 \uba54\ubaa8\ub9ac \ub204\uc218\uac00 \ubc1c\uc0dd\ud558\ub294 \ubb38\uc81c\ub97c \ucc3e\uc558\uc2b5\ub2c8\ub2e4. \ud574\uacb0\ubc29\ubc95\uc744 \ucd94\uac00\ub85c \ucc3e\uc544\ubcf4\ub294\uc911 \uc785\ub2c8\ub2e4. -> \uba54\ubaa8\ub9ac \ub204\uc218\ub97c \ud68d\uae30\uc801\uc73c\ub85c \uc904\uc5ec \ud604\uc7ac\ub294 \ud30c\ud2f0\ud074\uc758 \uc218\ub97c 500\uac1c\uc5d0\uc11c 1000\uac1c\uae4c\uc9c0 \uc99d\uac00\uc2dc\ucf1c\ub3c4 \ubb38\uc81c\uac00 \uc5c6\uc2b5\ub2c8\ub2e4.</br>\n-> \ucd94\uac00\ub85c \ud30c\ud2f0\ud074\uc758 \uc218\uac00 \uc801\uc744\ub54c\uc5d0\ub3c4 \uc804\uc5ed \ucd5c\uc801\ud574\ub97c \uc27d\uac8c \ucc3e\ub294 \ubc29\ubc95\uc744 \ucc3e\ub294\uc911 \uc785\ub2c8\ub2e4</br>\n\n> 4. \ud604\uc7ac tensorboard \ub85c \ub85c\uadf8 \uc800\uc7a5\uc2dc 994\uac1c \uc774\uc0c1 \uc800\uc7a5\uc774 \uc548\ub418\ub294 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.\n\n-> csv \ud30c\uc77c\ub85c \uc800\uc7a5\ud560 \uacbd\uc6b0 \uac2f\uc218\uc5d0\ub294 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud558\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4.\n-> \uc218\uac00 \uc801\uc744\ub54c \ud55c \ud30c\ud2f0\ud074\uc774 \uc9c0\uc5ed \ucd5c\uc801\ud574\uc5d0\uc11c \uba38\ubb34\ub97c \uacbd\uc6b0 \ud30c\ud2f0\ud074\uc744 \ucd08\uae30\ud654 \ud558\ub294 \ubc29\ubc95\uc774 \ud544\uc694\ud574 \ubcf4\uc785\ub2c8\ub2e4.\n\n> 5. \ubaa8\ub378\uc758 \ud06c\uae30\uac00 \ucee4\uc9c0\uba74 \uc218\ub834\uc774 \ub2a6\uc5b4\uc9c0\uace0 \uc815\ud655\ub3c4\uac00 \ub5a8\uc5b4\uc9c0\ub294 \ud604\uc0c1\uc774 \ubc1c\uacac\ub418\uc5c8\ub2e4. \ubaa8\ub378\uc758 \ud06c\uae30\uc5d0 \ub9de\ub294 \ud30c\ub77c\ubbf8\ud130\ub97c \ucc3e\uc544\uc57c\ud560 \uac83 \uac19\ub2e4.\n\n> 6. EBPSO \uc758 \ubc29\uc2dd\uc744 \ucd94\uac00\ub85c \uc801\uc6a9\uc744 \ud558\uc600\uc73c\ub098 \uc218\uc2dd\uc744 \uc798\ubabb \uc801\uc6a9\uc744 \ud55c\uac83\uc778\uc9c0 \uae30\ubcf8 pso \ubcf4\ub2e4 \ub354 \ub5a8\uc5b4\uc9c0\ub294 \uc815\ud655\ub3c4\ub97c \ubcf4\uc774\uace0 \uc788\ub2e4. (\ud604\uc7ac \uc218\uc815\uc911)\n\n### \uac1c\uc778\uc801\uc778 \uc0dd\uac01\n\n> \uba38\uc2e0\ub7ec\ub2dd \ubd84\ub958 \ubc29\uc2dd\uc5d0 \uc874\uc7ac\ud558\ub294 random forest \ubc29\uc2dd\uc744 \uc774\uc6a9\ud558\uc5ec, \uc624\ucc28\uc5ed\uc804\ud30c \ud568\uc218\ub97c \ucd5c\uc801\ud654 \ud558\ub294 \ubc29\ubc95\uc774 \uc788\uc744\uac83 \uac19\uc2b5\ub2c8\ub2e4\n>\n> > pso \uc640 random forest \ubc29\uc2dd\uc774 \ub9e4\uc6b0 \uc720\uc0ac\ud558\ub2e4\uace0 \uc0dd\uac01\ud558\uc5ec \ud559\uc2b5\ud560 \ub54c \ubfd0\ub9cc \uc544\ub2c8\ub77c \uc608\uce21 \ud560 \ub54c\ub3c4 \uc774\ub7ec\ud55c \ubc29\uc2dd\uc73c\ub85c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc744 \uac83 \uac19\uc2b5\ub2c8\ub2e4\n\n# \ucc38\uace0 \uc790\ub8cc\n\n[1]: [A partilce swarm optimization algorithm with empirical balance stategy](https://www.sciencedirect.com/science/article/pii/S2590054422000185#bib0005) </br>\n[2]: [psokeras](https://github.com/mike-holcomb/PSOkeras) </br>\n[3]: [PSO\uc758 \ub2e4\uc591\ud55c \uc601\uc5ed \ud0d0\uc0c9\uacfc \uc9c0\uc5ed\uc801 \ubbf8\ub2c8\uba48 \uc778\uc2dd\uc744 \uc704\ud55c \uc804\ub7b5](https://koreascience.kr/article/JAKO200925836515680.pdf) </br>\n[4]: [PC \ud074\ub7ec\uc2a4\ud130 \uae30\ubc18\uc758 Multi-HPSO\ub97c \uc774\uc6a9\ud55c \uc548\uc804\ub3c4 \uc81c\uc57d\uc758 \uacbd\uc81c \uae09\uc804](https://koreascience.kr/article/JAKO200932056732373.pdf) </br>\n[5]: [Particle 2-Swarm Optimization for Robust Search](https://s-space.snu.ac.kr/bitstream/10371/29949/3/management_information_v18_01_p01.pdf) </br>\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Particle Swarm Optimization on tensorflow package",
"version": "1.0.5.1",
"project_urls": {
"Homepage": "https://github.com/jung-geun/PSO"
},
"split_keywords": [
"pso",
"tensorflow",
"keras",
"optimization",
"particle swarm optimization",
"pso2keras"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f9cf9295a3764f70e879f7ca322fec4f841fcc4e83d2b7777676d2e21257f53c",
"md5": "babb28262ac515e007f6f72b06365ac1",
"sha256": "9cb530263c90f026ed037be53b8f4a4a4a1bcdc2671a3f8783fcddda453d65a5"
},
"downloads": -1,
"filename": "pso2keras-1.0.5.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "babb28262ac515e007f6f72b06365ac1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 17013,
"upload_time": "2024-03-08T10:36:56",
"upload_time_iso_8601": "2024-03-08T10:36:56.923614Z",
"url": "https://files.pythonhosted.org/packages/f9/cf/9295a3764f70e879f7ca322fec4f841fcc4e83d2b7777676d2e21257f53c/pso2keras-1.0.5.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "dc877cb6d79c13c5c28a36a589fe8b0e3934836df407aa540a05d81f17cd191a",
"md5": "961b1796cf2b15d71cb7c01a6f1f62b9",
"sha256": "06e28ae358259ec91a2f5be0c8b1099e2578cee99340cc71a146186116904c80"
},
"downloads": -1,
"filename": "pso2keras-1.0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "961b1796cf2b15d71cb7c01a6f1f62b9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 19757,
"upload_time": "2024-03-08T10:36:58",
"upload_time_iso_8601": "2024-03-08T10:36:58.787466Z",
"url": "https://files.pythonhosted.org/packages/dc/87/7cb6d79c13c5c28a36a589fe8b0e3934836df407aa540a05d81f17cd191a/pso2keras-1.0.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-08 10:36:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jung-geun",
"github_project": "PSO",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "ipython",
"specs": []
},
{
"name": "keras",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "pandas",
"specs": []
},
{
"name": "tensorflow",
"specs": []
},
{
"name": "tqdm",
"specs": []
},
{
"name": "scikit-learn",
"specs": []
},
{
"name": "tensorboard",
"specs": []
}
],
"lcname": "pso2keras"
}
JSON
