| Name | jenn JSON |
| Version |
1.0.8
JSON |
| download |
| home_page | None |
| Summary | Jacobian-Enhanced Neural Nets (JENN) |
| upload_time | 2024-07-26 17:14:21 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Jacobian-Enhanced Neural Network (JENN)
Jacobian-Enhanced Neural Networks (JENN) are fully connected multi-layer
perceptrons, whose training process is modified to predict partial
derivatives accurately. This is accomplished by minimizing a modified version
of the Least Squares Estimator (LSE) that accounts for Jacobian prediction error (see [paper](https://doi.org/10.48550/arXiv.2406.09132)).
The main benefit of jacobian-enhancement is better accuracy with
fewer training points compared to standard fully connected neural nets, as illustrated below.
<div align="center">
| Example #1 | Example #2 |
|:----------------------------------------------:|:-------------------------------:|
|  | |
| Example #3 |
|:--------------------------------:|
|  |
</div>
---
# Citation
If you use JENN in a scientific publication, please consider citing it:
```
@misc{berguin2024jacobianenhanced,
title={Jacobian-Enhanced Neural Networks},
author={Steven H. Berguin},
year={2024},
eprint={2406.09132},
archivePrefix={arXiv},
primaryClass={id='cs.LG' full_name='Machine Learning' is_active=True alt_name=None in_archive='cs' is_general=False description='Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.'}
}
```
----
# Main Features
* Multi-Task Learning : predict more than one output with same model Y = f(X) where Y = [y1, y2, ...]
* Jacobian prediction : analytically compute the Jacobian (_i.e._ forward propagation of dY/dX)
* Gradient-Enhancement: minimize prediction error of partials (_i.e._ back-prop accounts for dY/dX)
----
# Installation
pip install jenn
----
# Example Usage
_See [demo](./docs/examples/) notebooks for more details_
Import library:
import jenn
Generate example training and test data:
x_train, y_train, dydx_train = jenn.synthetic.Sinusoid.sample(
m_lhs=0,
m_levels=4,
lb=-3.14,
ub=3.14,
)
x_test, y_test, dydx_test = jenn.synthetic.Sinusoid.sample(
m_lhs=30,
m_levels=0,
lb=-3.14,
ub=3.14,
)
Train a model:
nn = jenn.model.NeuralNet(
layer_sizes=[1, 12, 1],
).fit(
x=x_train,
y=y_train,
dydx=dydx_train,
lambd=0.1, # regularization parameter
is_normalize=True, # normalize data before fitting it
)
Make predictions:
y, dydx = nn.evaluate(x)
# OR
y = nn.predict(x)
dydx = nn.predict_partials(x)
Save model (parameters) for later use:
nn.save('parameters.json')
Reload saved parameters into new model:
reloaded = jenn.model.NeuralNet(layer_sizes=[1, 12, 1]).load('parameters.json')
Optionally, if `matplotlib` is installed, import plotting utilities:
from jenn.utils import plot
Optionally, if `matplotlib` is installed, check goodness of fit:
plot.goodness_of_fit(
y_true=dydx_test[0],
y_pred=nn.predict_partials(x_test)[0],
title="Partial Derivative: dy/dx (JENN)"
)
Optionally, if `matplotlib` is installed, show sensitivity profiles:
plot.sensitivity_profiles(
f=[jenn.synthetic.Sinusoid.evaluate, nn.predict],
x_min=x_train.min(),
x_max=x_train.max(),
x_true=x_train,
y_true=y_train,
resolution=100,
legend=['true', 'pred'],
xlabels=['x'],
ylabels=['y'],
)
----
# Use Case
JENN is intended for the field of computer aided design, where there is often
a need to replace computationally expensive, physics-based models with so-called _surrogate models_ in
order to save time down the line. Since the _surrogate model_ emulates the original model accurately
in real time, it offers a speed benefit that can be used to carry out orders of magnitude
more function calls quickly, opening the door to Monte Carlo simulation of expensive functions for example.
In general, the value proposition of a surrogate is that the computational
expense of generating training data to fit the model
is much less than the computational expense of performing the analysis with the original physics-based model itself.
However, in the special case of gradient-enhanced methods, there is the additional value proposition that partials
are accurate which is a critical property for one important use-case: **surrogate-based optimization**. The field of
aerospace engineering is rich in [applications](https://doi.org/10.1002/9780470686652.eae496) of such a use-case.
----
# Limitations
Gradient-enhanced methods require responses to be continuous and smooth,
but they are only beneficial if the cost of obtaining partials
is not excessive in the first place (e.g. adjoint methods), or if the need for accuracy outweighs the cost of
computing the partials. Users should therefore carefully weigh the benefit of
gradient-enhanced methods relative to the needs of their application.
---
# License
Distributed under the terms of the MIT License.
----
# Acknowledgement
This code used the code by Prof. Andrew Ng in the
[Coursera Deep Learning Specialization](https://www.coursera.org/specializations/deep-learning)
as a starting point. It then built upon it to include additional features such
as line search and plotting but, most of all, it fundamentally changed the formulation
to include gradient-enhancement and made sure all arrays were updated in place (data is never copied).
The author would like to thank Andrew Ng for
offering the fundamentals of deep learning on Coursera, which took a complicated
subject and explained it in simple terms that even an aerospace engineer could understand.
Raw data
{
"_id": null,
"home_page": null,
"name": "jenn",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "\"Steven H. Berguin\" <stevenberguin@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/4e/75/2a7d2fa706010a33d317587ef02933e1c74a484c3ea4d5edd97243257a5b/jenn-1.0.8.tar.gz",
"platform": null,
"description": "# Jacobian-Enhanced Neural Network (JENN)\n\nJacobian-Enhanced Neural Networks (JENN) are fully connected multi-layer\nperceptrons, whose training process is modified to predict partial \nderivatives accurately. This is accomplished by minimizing a modified version \nof the Least Squares Estimator (LSE) that accounts for Jacobian prediction error (see [paper](https://doi.org/10.48550/arXiv.2406.09132)). \nThe main benefit of jacobian-enhancement is better accuracy with\nfewer training points compared to standard fully connected neural nets, as illustrated below. \n\n<div align=\"center\">\n\n| Example #1 | Example #2 |\n|:----------------------------------------------:|:-------------------------------:|\n|  | |\n\n| Example #3 |\n|:--------------------------------:|\n|  |\n\n</div>\n \n---\n# Citation\n\nIf you use JENN in a scientific publication, please consider citing it: \n\n```\n@misc{berguin2024jacobianenhanced,\n title={Jacobian-Enhanced Neural Networks}, \n author={Steven H. Berguin},\n year={2024},\n eprint={2406.09132},\n archivePrefix={arXiv},\n primaryClass={id='cs.LG' full_name='Machine Learning' is_active=True alt_name=None in_archive='cs' is_general=False description='Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.'}\n}\n```\n\n----\n# Main Features\n\n* Multi-Task Learning : predict more than one output with same model Y = f(X) where Y = [y1, y2, ...]\n* Jacobian prediction : analytically compute the Jacobian (_i.e._ forward propagation of dY/dX)\n* Gradient-Enhancement: minimize prediction error of partials (_i.e._ back-prop accounts for dY/dX)\n\n----\n\n# Installation\n\n pip install jenn \n\n----\n\n# Example Usage\n\n_See [demo](./docs/examples/) notebooks for more details_\n\nImport library: \n\n import jenn\n\nGenerate example training and test data: \n\n x_train, y_train, dydx_train = jenn.synthetic.Sinusoid.sample(\n m_lhs=0, \n m_levels=4, \n lb=-3.14, \n ub=3.14,\n )\n x_test, y_test, dydx_test = jenn.synthetic.Sinusoid.sample(\n m_lhs=30, \n m_levels=0, \n lb=-3.14, \n ub=3.14,\n )\n\n\nTrain a model: \n\n nn = jenn.model.NeuralNet(\n layer_sizes=[1, 12, 1],\n ).fit(\n x=x_train, \n y=y_train, \n dydx=dydx_train,\n lambd=0.1, # regularization parameter \n is_normalize=True, # normalize data before fitting it\n )\n \n Make predictions: \n\n y, dydx = nn.evaluate(x)\n\n # OR \n\n y = nn.predict(x)\n dydx = nn.predict_partials(x)\n\n\nSave model (parameters) for later use: \n\n nn.save('parameters.json') \n\nReload saved parameters into new model: \n\n reloaded = jenn.model.NeuralNet(layer_sizes=[1, 12, 1]).load('parameters.json')\n\nOptionally, if `matplotlib` is installed, import plotting utilities: \n\n from jenn.utils import plot\n\nOptionally, if `matplotlib` is installed, check goodness of fit: \n\n plot.goodness_of_fit(\n y_true=dydx_test[0], \n y_pred=nn.predict_partials(x_test)[0], \n title=\"Partial Derivative: dy/dx (JENN)\"\n )\n\nOptionally, if `matplotlib` is installed, show sensitivity profiles:\n\n plot.sensitivity_profiles(\n f=[jenn.synthetic.Sinusoid.evaluate, nn.predict], \n x_min=x_train.min(), \n x_max=x_train.max(), \n x_true=x_train, \n y_true=y_train, \n resolution=100, \n legend=['true', 'pred'], \n xlabels=['x'], \n ylabels=['y'],\n )\n\n----\n\n# Use Case\n\nJENN is intended for the field of computer aided design, where there is often \na need to replace computationally expensive, physics-based models with so-called _surrogate models_ in\norder to save time down the line. Since the _surrogate model_ emulates the original model accurately \nin real time, it offers a speed benefit that can be used to carry out orders of magnitude \nmore function calls quickly, opening the door to Monte Carlo simulation of expensive functions for example. \n\nIn general, the value proposition of a surrogate is that the computational \nexpense of generating training data to fit the model \nis much less than the computational expense of performing the analysis with the original physics-based model itself. \nHowever, in the special case of gradient-enhanced methods, there is the additional value proposition that partials \nare accurate which is a critical property for one important use-case: **surrogate-based optimization**. The field of \naerospace engineering is rich in [applications](https://doi.org/10.1002/9780470686652.eae496) of such a use-case. \n\n----\n\n# Limitations\n\nGradient-enhanced methods require responses to be continuous and smooth, \nbut they are only beneficial if the cost of obtaining partials \nis not excessive in the first place (e.g. adjoint methods), or if the need for accuracy outweighs the cost of \ncomputing the partials. Users should therefore carefully weigh the benefit of \ngradient-enhanced methods relative to the needs of their application. \n\n--- \n# License\nDistributed under the terms of the MIT License.\n\n----\n\n# Acknowledgement\n\nThis code used the code by Prof. Andrew Ng in the\n[Coursera Deep Learning Specialization](https://www.coursera.org/specializations/deep-learning)\nas a starting point. It then built upon it to include additional features such\nas line search and plotting but, most of all, it fundamentally changed the formulation \nto include gradient-enhancement and made sure all arrays were updated in place (data is never copied). \nThe author would like to thank Andrew Ng for\noffering the fundamentals of deep learning on Coursera, which took a complicated\nsubject and explained it in simple terms that even an aerospace engineer could understand.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Jacobian-Enhanced Neural Nets (JENN)",
"version": "1.0.8",
"project_urls": {
"Documentation": "https://shb84.github.io/JENN/",
"Homepage": "https://github.com/shb84/JENN",
"Issues": "https://github.com/shb84/JENN/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1ecd8d705a417b2dd5bb7383abeef7cb5d2522805d70c804a7ba784be5a5bb91",
"md5": "72834d3011ecabd573850a7c2fd7a407",
"sha256": "145e6936c42bd5d88e24a22f6d1100e6fe04ad1daaa5ea4a16682fe29b2f01cd"
},
"downloads": -1,
"filename": "jenn-1.0.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "72834d3011ecabd573850a7c2fd7a407",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 35367,
"upload_time": "2024-07-26T17:13:42",
"upload_time_iso_8601": "2024-07-26T17:13:42.764608Z",
"url": "https://files.pythonhosted.org/packages/1e/cd/8d705a417b2dd5bb7383abeef7cb5d2522805d70c804a7ba784be5a5bb91/jenn-1.0.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4e752a7d2fa706010a33d317587ef02933e1c74a484c3ea4d5edd97243257a5b",
"md5": "1fc53cbcc4531e0d0432382e0b3991b9",
"sha256": "2c0e07bc3ce04897d9ffdbaccfe7e0f434e2251dba137d7cbe79f06d9a0f4705"
},
"downloads": -1,
"filename": "jenn-1.0.8.tar.gz",
"has_sig": false,
"md5_digest": "1fc53cbcc4531e0d0432382e0b3991b9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 47089,
"upload_time": "2024-07-26T17:14:21",
"upload_time_iso_8601": "2024-07-26T17:14:21.743275Z",
"url": "https://files.pythonhosted.org/packages/4e/75/2a7d2fa706010a33d317587ef02933e1c74a484c3ea4d5edd97243257a5b/jenn-1.0.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-26 17:14:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "shb84",
"github_project": "JENN",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "jenn"
}
