TUCAN (Tool to Unformat, Clean, and Analyze) is a code parser for scientific codebases. Its target languages are:
- Very old FORTRAN
- Recent FORTRAN
- Python (Under development)
- C/C++ (Early development)
## Installation
You can instal it from [PyPI](https://pypi.org/project/tucan/) with:
```bash
pip install tucan
```
You can also install from the sources from one of our [gitlab mirrors](https://codehub.hlrs.de/coes/excellerat-p2/uc-2/tucan).
## What is does?
### Remove coding archaisms
First it is a code cleaner. For example, this loop in `tranfit.f', a piece of [CHEMKIN](https://en.wikipedia.org/wiki/CHEMKIN) II package in good'old FORTRAN77. (Do not worry, recent Chemkin is not written that way, probably) :
```fortran
(547) DO 2000 K = 1, KK-1
(548) DO 2000 J = K+1, KK
(549) DO 2000 N = 1, NO
(550) COFD(N,J,K) = COFD(N,K,J)
(551) 2000 CONTINUE
```
Is translated with the command `tucan clean tranfit.f` as :
```fortran
(547-547) do 2000 k = 1,kk-1
(548-548) do 2000 j = k+1,kk
(549-549) do 2000 n = 1,no
(550-550) cofd(n,j,k) = cofd(n,k,j)
(551-551) end do ! 2000
(551-551) end do ! 2000
(551-551) end do ! 2000
```
The cleaned version is a simpler code for further analysis passes, like computing cyclomatic complexity, extracting structures, etc...
### Extracting code structure
Here we start from a file of [neko](https://github.com/ExtremeFLOW/neko/blob/develop/src/adt/htable.f90), an HPC code in recent Fortran, finalist for the Gordon Bell Prize in 2023.
`tucan struct htable.f90` provides a nested dictionary of the code structure. Here is a part of the output:
```yaml
(...)
type htable.h_tuple_t :
At path ['htable', 'h_tuple_t'], name h_tuple_t, lines 47 -> 52
Nb. Statements 6
Nb. lines of code 6
Ctls. Pts. (McCabe) 0 | 0 Int. avg.
Halstead Difficulty 8.25 | 8.25 Int. avg.
Maintainability Index 94.17 | 94.17 Int. avg.
Average indents 2.2 | 2.2 Int. avg.
Halstead time 48 sec | 48 sec Ext. avg.
Structural complexity 6 | 6 Ext. avg.
Nb. of Loops 0 | 0 Ext. avg.
procedure htable.htable_t.hash :
At path ['htable', 'htable_t', 'hash'], name htable.htable_t.hash, lines 60 -> 60
Nb. Statements 1
Nb. lines of code 1
Ctls. Pts. (McCabe) 0 | 0 Int. avg.
Halstead Difficulty 0 | 0 Int. avg.
Maintainability Index 378.29 | 378.29 Int. avg.
Average indents 0 | 0 Int. avg.
Halstead time 0 sec | 0 sec Ext. avg.
Structural complexity 4 | 4 Ext. avg.
Nb. of Loops 0 | 0 Ext. avg.
Refers to 1 callables:
- htable.hash
procedure htable.htable_t.htable_clear :
At path ['htable', 'htable_t', 'htable_clear'], name htable.htable_t.htable_clear, lines 61 -> 61
Nb. Statements 1
Nb. lines of code 1
Ctls. Pts. (McCabe) 0 | 0 Int. avg.
Halstead Difficulty 0 | 0 Int. avg.
Maintainability Index 378.29 | 378.29 Int. avg.
Average indents 0 | 0 Int. avg.
Halstead time 0 sec | 0 sec Ext. avg.
Structural complexity 4 | 4 Ext. avg.
Nb. of Loops 0 | 0 Ext. avg.
Refers to 1 callables:
- htable.htable_clear
procedure htable.htable_t.htable_free :
At path ['htable', 'htable_t', 'htable_free'], name htable.htable_t.htable_free, lines 62 -> 62
Nb. Statements 1
Nb. lines of code 1
Ctls. Pts. (McCabe) 0 | 0 Int. avg.
Halstead Difficulty 0 | 0 Int. avg.
Maintainability Index 378.29 | 378.29 Int. avg.
Average indents 0 | 0 Int. avg.
Halstead time 0 sec | 0 sec Ext. avg.
Structural complexity 4 | 4 Ext. avg.
Nb. of Loops 0 | 0 Ext. avg.
Refers to 1 callables:
- htable.htable_free
(...)
module htable :
At path ['htable'], name htable, lines 36 -> 1482
Nb. Statements 1079
Nb. lines of code 1447
Ctls. Pts. (McCabe) 0 | 3.01 Int. avg.
Halstead Difficulty 4 | 15.25 Int. avg.
Maintainability Index -28.7 | 76.61 Int. avg.
Average indents 1 | 2.36 Int. avg.
Halstead time 28 sec | 7.87 hrs Ext. avg.
Structural complexity 1 | 474 Ext. avg.
Nb. of Loops 0 | 18 Ext. avg.
Refers to 89 contains:
- htable.h_tuple_t
- htable.htable_t
- htable.interface66
- htable.htable_i4_t
- htable.htable_i8_t
- htable.htable_r8_t
(...)
```
*(This output will change as we update and improve tucan in the next versions!)*
This information allows the creation and manipulation of graphs to extract the structure of the code
### Interpreting Conditional Inclusions "IF DEFS".
An other example of tucan is the analysis of [ifdefs](https://en.cppreference.com/w/c/preprocessor/conditional) in C or FORTRAN:
```
#ifdef FRONT
WRITE(*,*) " FRONT is enabled " ! partial front subroutine
SUBROUTINE dummy_front(a,b,c)
WRITE(*,*) " FRONT 1" ! partial front subroutine
#else
SUBROUTINE dummy_front(a,d,e)
WRITE(*,*) " FRONT 2" ! partial front subroutine
#endif
END SUBROUTINE
SUBROUTINE dummy_back(a,b,c)
#ifdef BACK
WRITE(*,*) " FRONT is enabled " ! partial front subroutine
WRITE(*,*) " BACK 1" ! partial back subroutine
END SUBROUTINE
#else
WRITE(*,*) " BACK 2" ! partial back subroutine
END SUBROUTINE
#endif
```
Depending on the pre-definition of variables FRONT and BACK, this code snippet can be read in four ways possible.
Here are usages:
`tucan cpp-clean templates_ifdef.f` yields:
```fortran
SUBROUTINE dummy_front(a,d,e)
WRITE(*,*) " FRONT 2" ! partial front subroutine
END SUBROUTINE
SUBROUTINE dummy_back(a,b,c)
WRITE(*,*) " BACK 2" ! partial back subroutine
END SUBROUTINE
```
`tucan cpp-clean templates_ifdef.f -v FRONT` yields:
```fortran
WRITE(*,*) " FRONT is enabled " ! partial front subroutine
SUBROUTINE dummy_front(a,b,c)
WRITE(*,*) " FRONT 1" ! partial front subroutine
END SUBROUTINE
SUBROUTINE dummy_back(a,b,c)
WRITE(*,*) " BACK 2" ! partial back subroutine
END SUBROUTINE
```
`tucan cpp-clean templates_ifdef.f -v FRONT,BACK` yields:
```fortran
WRITE(*,*) " FRONT is enabled " ! partial front subroutine
SUBROUTINE dummy_front(a,b,c)
WRITE(*,*) " FRONT 1" ! partial front subroutine
END SUBROUTINE
SUBROUTINE dummy_back(a,b,c)
WRITE(*,*) " BACK is enabled " ! partial front subroutine
WRITE(*,*) " BACK 1" ! partial back subroutine
END SUBROUTINE
```
#### scanning ifdef variables
A simpler usage of tucan : scan the current ifdefs variables. Still on [neko](https://github.com/ExtremeFLOW/neko) in the `/src` folder (an old version though) :
```bash
/neko/src >tucan cpp-scan-pkge .
- Recursive path gathering ...
- Cleaning the paths ...
- Analysis completed.
- Global ifdef variables : HAVE_PARMETIS, __APPLE__
- Local to device/opencl/check.c : CL_ERR_STR(err)
- Local to math/bcknd/device/opencl/opr_opgrad.c : CASE(LX), STR(X)
- Local to math/bcknd/device/opencl/opr_dudxyz.c : CASE(LX), STR(X)
- Local to common/sighdl.c : SIGHDL_ALRM, SIGHDL_USR1, SIGHDL_USR2, SIGHDL_XCPU
- Local to math/bcknd/device/opencl/opr_conv1.c : CASE(LX), STR(X)
- Local to math/bcknd/device/opencl/opr_cfl.c : CASE(LX), STR(X)
- Local to krylov/bcknd/device/opencl/pc_jacobi.c : CASE(LX), STR(X)
- Local to math/bcknd/device/opencl/ax_helm.c : CASE(LX), STR(X)
- Local to bc/bcknd/device/opencl/symmetry.c : MAX(a,
- Local to gs/bcknd/device/opencl/gs.c : GS_OP_ADD, GS_OP_MAX, GS_OP_MIN, GS_OP_MUL
- Local to sem/bcknd/device/opencl/coef.c : DXYZDRST_CASE(LX), GEO_CASE(LX), STR(X)
- Local to math/bcknd/device/opencl/opr_cdtp.c : CASE(LX), STR(X)
```
This feature is useful to see all potential variables that surcharge your codebase via conditional inclusions.
## More about tucan
`Tucan` is used by `anubis`, our open-source tool to explore the git repository of a code, and `marauder's map` our open-source tool to show codes structures by in-depth vizualisation of callgraphs and code circular-packing .
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/cerfacs/tucan",
"name": "tucan",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "code, fortran, python",
"author": "Antoine Dauptain, Thibault Marzlin, CoopTeam-CERFACS",
"author_email": "coop@cerfacs.fr",
"download_url": "https://files.pythonhosted.org/packages/1b/e5/9c88c67ea76852a5b8433a356a2dc55cf94f79d8712ad166d1f2e53fdcf1/tucan-0.4.5.tar.gz",
"platform": null,
"description": "\n\nTUCAN (Tool to Unformat, Clean, and Analyze) is a code parser for scientific codebases. Its target languages are:\n\n- Very old FORTRAN\n- Recent FORTRAN\n- Python (Under development)\n- C/C++ (Early development)\n\n## Installation\n\nYou can instal it from [PyPI](https://pypi.org/project/tucan/) with:\n\n```bash\npip install tucan\n```\n\nYou can also install from the sources from one of our [gitlab mirrors](https://codehub.hlrs.de/coes/excellerat-p2/uc-2/tucan).\n\n\n## What is does?\n\n\n### Remove coding archaisms\n\nFirst it is a code cleaner. For example, this loop in `tranfit.f', a piece of [CHEMKIN](https://en.wikipedia.org/wiki/CHEMKIN) II package in good'old FORTRAN77. (Do not worry, recent Chemkin is not written that way, probably) :\n\n```fortran\n(547) DO 2000 K = 1, KK-1\n(548) DO 2000 J = K+1, KK\n(549) DO 2000 N = 1, NO\n(550) COFD(N,J,K) = COFD(N,K,J)\n(551) 2000 CONTINUE\n```\n\nIs translated with the command `tucan clean tranfit.f` as : \n```fortran\n(547-547) do 2000 k = 1,kk-1\n(548-548) do 2000 j = k+1,kk\n(549-549) do 2000 n = 1,no\n(550-550) cofd(n,j,k) = cofd(n,k,j)\n(551-551) end do ! 2000\n(551-551) end do ! 2000\n(551-551) end do ! 2000\n```\n\n\nThe cleaned version is a simpler code for further analysis passes, like computing cyclomatic complexity, extracting structures, etc...\n\n\n### Extracting code structure\n\n\nHere we start from a file of [neko](https://github.com/ExtremeFLOW/neko/blob/develop/src/adt/htable.f90), an HPC code in recent Fortran, finalist for the Gordon Bell Prize in 2023.\n\n`tucan struct htable.f90` provides a nested dictionary of the code structure. Here is a part of the output:\n\n```yaml\n(...)\ntype htable.h_tuple_t :\n At path ['htable', 'h_tuple_t'], name h_tuple_t, lines 47 -> 52\n Nb. Statements 6\n Nb. lines of code 6\n Ctls. Pts. (McCabe) 0 | 0 Int. avg.\n Halstead Difficulty 8.25 | 8.25 Int. avg.\n Maintainability Index 94.17 | 94.17 Int. avg.\n Average indents 2.2 | 2.2 Int. avg.\n Halstead time 48 sec | 48 sec Ext. avg.\n Structural complexity 6 | 6 Ext. avg.\n Nb. of Loops 0 | 0 Ext. avg.\n\nprocedure htable.htable_t.hash :\n At path ['htable', 'htable_t', 'hash'], name htable.htable_t.hash, lines 60 -> 60\n Nb. Statements 1\n Nb. lines of code 1\n Ctls. Pts. (McCabe) 0 | 0 Int. avg.\n Halstead Difficulty 0 | 0 Int. avg.\n Maintainability Index 378.29 | 378.29 Int. avg.\n Average indents 0 | 0 Int. avg.\n Halstead time 0 sec | 0 sec Ext. avg.\n Structural complexity 4 | 4 Ext. avg.\n Nb. of Loops 0 | 0 Ext. avg.\n Refers to 1 callables:\n - htable.hash\n\nprocedure htable.htable_t.htable_clear :\n At path ['htable', 'htable_t', 'htable_clear'], name htable.htable_t.htable_clear, lines 61 -> 61\n Nb. Statements 1\n Nb. lines of code 1\n Ctls. Pts. (McCabe) 0 | 0 Int. avg.\n Halstead Difficulty 0 | 0 Int. avg.\n Maintainability Index 378.29 | 378.29 Int. avg.\n Average indents 0 | 0 Int. avg.\n Halstead time 0 sec | 0 sec Ext. avg.\n Structural complexity 4 | 4 Ext. avg.\n Nb. of Loops 0 | 0 Ext. avg.\n Refers to 1 callables:\n - htable.htable_clear\n\nprocedure htable.htable_t.htable_free :\n At path ['htable', 'htable_t', 'htable_free'], name htable.htable_t.htable_free, lines 62 -> 62\n Nb. Statements 1\n Nb. lines of code 1\n Ctls. Pts. (McCabe) 0 | 0 Int. avg.\n Halstead Difficulty 0 | 0 Int. avg.\n Maintainability Index 378.29 | 378.29 Int. avg.\n Average indents 0 | 0 Int. avg.\n Halstead time 0 sec | 0 sec Ext. avg.\n Structural complexity 4 | 4 Ext. avg.\n Nb. of Loops 0 | 0 Ext. avg.\n Refers to 1 callables:\n - htable.htable_free\n(...)\nmodule htable :\n At path ['htable'], name htable, lines 36 -> 1482\n Nb. Statements 1079\n Nb. lines of code 1447\n Ctls. Pts. (McCabe) 0 | 3.01 Int. avg.\n Halstead Difficulty 4 | 15.25 Int. avg.\n Maintainability Index -28.7 | 76.61 Int. avg.\n Average indents 1 | 2.36 Int. avg.\n Halstead time 28 sec | 7.87 hrs Ext. avg.\n Structural complexity 1 | 474 Ext. avg.\n Nb. of Loops 0 | 18 Ext. avg.\n Refers to 89 contains:\n - htable.h_tuple_t\n - htable.htable_t\n - htable.interface66\n - htable.htable_i4_t\n - htable.htable_i8_t\n - htable.htable_r8_t\n(...)\n\n```\n\n*(This output will change as we update and improve tucan in the next versions!)*\n\nThis information allows the creation and manipulation of graphs to extract the structure of the code\n\n\n### Interpreting Conditional Inclusions \"IF DEFS\". \n\nAn other example of tucan is the analysis of [ifdefs](https://en.cppreference.com/w/c/preprocessor/conditional) in C or FORTRAN:\n\n```\n#ifdef FRONT\n WRITE(*,*) \" FRONT is enabled \" ! partial front subroutine\n SUBROUTINE dummy_front(a,b,c)\n WRITE(*,*) \" FRONT 1\" ! partial front subroutine\n#else \n SUBROUTINE dummy_front(a,d,e)\n WRITE(*,*) \" FRONT 2\" ! partial front subroutine\n#endif\n END SUBROUTINE\n\n SUBROUTINE dummy_back(a,b,c)\n#ifdef BACK\n WRITE(*,*) \" FRONT is enabled \" ! partial front subroutine\n WRITE(*,*) \" BACK 1\" ! partial back subroutine\n END SUBROUTINE \n#else\n WRITE(*,*) \" BACK 2\" ! partial back subroutine\n END SUBROUTINE \n#endif\n```\n\nDepending on the pre-definition of variables FRONT and BACK, this code snippet can be read in four ways possible.\nHere are usages:\n\n`tucan cpp-clean templates_ifdef.f` yields:\n\n```fortran\n SUBROUTINE dummy_front(a,d,e)\n WRITE(*,*) \" FRONT 2\" ! partial front subroutine\n END SUBROUTINE\n\n SUBROUTINE dummy_back(a,b,c)\n\n\n WRITE(*,*) \" BACK 2\" ! partial back subroutine\n END SUBROUTINE\n```\n\n\n`tucan cpp-clean templates_ifdef.f -v FRONT` yields:\n\n```fortran\n WRITE(*,*) \" FRONT is enabled \" ! partial front subroutine\n SUBROUTINE dummy_front(a,b,c)\n WRITE(*,*) \" FRONT 1\" ! partial front subroutine\n\n\n END SUBROUTINE\n\n SUBROUTINE dummy_back(a,b,c)\n\n\n WRITE(*,*) \" BACK 2\" ! partial back subroutine\n END SUBROUTINE\n```\n\n`tucan cpp-clean templates_ifdef.f -v FRONT,BACK` yields:\n\n```fortran\n WRITE(*,*) \" FRONT is enabled \" ! partial front subroutine\n SUBROUTINE dummy_front(a,b,c)\n WRITE(*,*) \" FRONT 1\" ! partial front subroutine\n\n\n END SUBROUTINE\n\n SUBROUTINE dummy_back(a,b,c)\n WRITE(*,*) \" BACK is enabled \" ! partial front subroutine\n WRITE(*,*) \" BACK 1\" ! partial back subroutine\n END SUBROUTINE\n```\n\n#### scanning ifdef variables\n\nA simpler usage of tucan : scan the current ifdefs variables. Still on [neko](https://github.com/ExtremeFLOW/neko) in the `/src` folder (an old version though) : \n\n```bash\n/neko/src >tucan cpp-scan-pkge .\n - Recursive path gathering ...\n - Cleaning the paths ...\n - Analysis completed.\n - Global ifdef variables : HAVE_PARMETIS, __APPLE__\n - Local to device/opencl/check.c : CL_ERR_STR(err)\n - Local to math/bcknd/device/opencl/opr_opgrad.c : CASE(LX), STR(X)\n - Local to math/bcknd/device/opencl/opr_dudxyz.c : CASE(LX), STR(X)\n - Local to common/sighdl.c : SIGHDL_ALRM, SIGHDL_USR1, SIGHDL_USR2, SIGHDL_XCPU\n - Local to math/bcknd/device/opencl/opr_conv1.c : CASE(LX), STR(X)\n - Local to math/bcknd/device/opencl/opr_cfl.c : CASE(LX), STR(X)\n - Local to krylov/bcknd/device/opencl/pc_jacobi.c : CASE(LX), STR(X)\n - Local to math/bcknd/device/opencl/ax_helm.c : CASE(LX), STR(X)\n - Local to bc/bcknd/device/opencl/symmetry.c : MAX(a,\n - Local to gs/bcknd/device/opencl/gs.c : GS_OP_ADD, GS_OP_MAX, GS_OP_MIN, GS_OP_MUL\n - Local to sem/bcknd/device/opencl/coef.c : DXYZDRST_CASE(LX), GEO_CASE(LX), STR(X)\n - Local to math/bcknd/device/opencl/opr_cdtp.c : CASE(LX), STR(X)\n```\nThis feature is useful to see all potential variables that surcharge your codebase via conditional inclusions.\n\n## More about tucan\n\n`Tucan` is used by `anubis`, our open-source tool to explore the git repository of a code, and `marauder's map` our open-source tool to show codes structures by in-depth vizualisation of callgraphs and code circular-packing .\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Unformatter Tool to allow parsing and analysis of code base.",
"version": "0.4.5",
"project_urls": {
"Bug Tracker": "https://gitlab.com/cerfacs/tucan/-/issues",
"Documentation": "https://tucan.readthedocs.io/en/latest/",
"Homepage": "https://gitlab.com/cerfacs/tucan"
},
"split_keywords": [
"code",
" fortran",
" python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "33e9baed3fb22a4a8e428d97ae914c72a20eae46c3f7c8a31fe9f3190d844eed",
"md5": "b759ebe34ed6f573d708c0ba615cd0b8",
"sha256": "6a9b6d8c372bb9d827e074a06a37055bb7d7b7efba3f47f0ba280a690ee20932"
},
"downloads": -1,
"filename": "tucan-0.4.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b759ebe34ed6f573d708c0ba615cd0b8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 62491,
"upload_time": "2024-12-19T16:10:23",
"upload_time_iso_8601": "2024-12-19T16:10:23.862185Z",
"url": "https://files.pythonhosted.org/packages/33/e9/baed3fb22a4a8e428d97ae914c72a20eae46c3f7c8a31fe9f3190d844eed/tucan-0.4.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1be59c88c67ea76852a5b8433a356a2dc55cf94f79d8712ad166d1f2e53fdcf1",
"md5": "827605df04fb663f0fb3475966da4148",
"sha256": "9942fe46c45683b59f3b73b94cb2c840e20bea5ea799c29b5917f1481116108a"
},
"downloads": -1,
"filename": "tucan-0.4.5.tar.gz",
"has_sig": false,
"md5_digest": "827605df04fb663f0fb3475966da4148",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 157588,
"upload_time": "2024-12-19T16:10:26",
"upload_time_iso_8601": "2024-12-19T16:10:26.898306Z",
"url": "https://files.pythonhosted.org/packages/1b/e5/9c88c67ea76852a5b8433a356a2dc55cf94f79d8712ad166d1f2e53fdcf1/tucan-0.4.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-19 16:10:26",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "cerfacs",
"gitlab_project": "tucan",
"lcname": "tucan"
}
JSON