#+title: sxpyr: a flexible Lisp reader for Python
Parse s-expressions, edn, and a variety of lisp dialects.
* Introduction
Currently supports something approximating the union of Common Lisp,
Scheme, Racket, Clojure, Hy, Fennel, and Emacs Lisp.
Since sxpyr is designed to support a variety of different dialects, a
second pass is almost always needed in order to match the output of
native readers. This second pass is a good place to raise additional
syntax errors for the dialect you are working with. Some dialects have
already been implemented.
Due to deeper differences between dialects, in some cases it is
necessary to configure the reader in advance. This happens in cases
where a particular form is valid in two different dialects and one
or both parses are sufficiently divergent that it is not possible
to recover the correct parse for the other dialect.
At the moment the reader cannot be modified at runtime. This means
that it cannot be used to parse named readtables or things like
Racket's ~#lang at-exp~. In the future it might be possible to pass
the generator through to the functions that handle dispatch macros and
make it possible to register new reader macros, but that is well out
of scope at the moment.
Originally this library was intended to be vendored, however since
then it is significantly increased in complexity. A stripped down
version that is suitable for parsing a minimal sane dialect that can
be used as a configuration language may be added in the future.
* Usage
#+begin_src bash
git clone https://github.com/tgbugs/sxpyr.git
pushd sxpyr
pip install --user -e .[cli]
python -m sxpyr.cli parse test/data/nested-splicing-unsyntax.rkt test/data/plist-test.sxpr
popd
#+end_src
* Testing
In order to run the tests you will need the following repos.
- sbcl
- ccl
- emacs
- org-mode
- clojure
- spec.alpha
- code.specs.alpha
- tawny-owl
- hy
- Fennel
- txr
For now you will also need the following installed on your system.
- xemacs
- app-xemacs/xemacs-packages-all
- guile
- gambit
* Fuzzing
You will need a copy of afl.
#+begin_src bash
pip install --user -e .[fuzz]
pushd test
mkdir initial-inputs
cp data/plist-test.sxpr initial-inputs/ # other seed files can be added as well
py-afl-fuzz -m 400 -i initial-inputs/ -o fuzzing-results/ -- python -m sxpyr.cli parse --fuzz
#+end_src
#+begin_src bash
python -m sxpyr.cli parse test/fuzzing-results/crashes/*
#+end_src
Note that initial inputs cannot be symlinks becuase afl creates hardlinks to the input files.
Also note that afl seems to die after 175k execs?
Sometimes it also seems that some errors are escaping the except block
and making it be classified as crashes?
Raw data
{
"_id": null,
"home_page": "https://github.com/tgbugs/sxpyr",
"name": "sxpyr",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "lisp reader edn clojure racket scheme elisp emacs sexp s-expression parser parsing",
"author": "Tom Gillespie",
"author_email": "tgbugs@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/63/df/014c20a5c4b1e8335a58c8e59d1d88389804ab153df9a117644dad2eb995/sxpyr-0.0.4.tar.gz",
"platform": null,
"description": "#+title: sxpyr: a flexible Lisp reader for Python\n\nParse s-expressions, edn, and a variety of lisp dialects.\n\n* Introduction\nCurrently supports something approximating the union of Common Lisp,\nScheme, Racket, Clojure, Hy, Fennel, and Emacs Lisp.\n\nSince sxpyr is designed to support a variety of different dialects, a\nsecond pass is almost always needed in order to match the output of\nnative readers. This second pass is a good place to raise additional\nsyntax errors for the dialect you are working with. Some dialects have\nalready been implemented.\n\nDue to deeper differences between dialects, in some cases it is\nnecessary to configure the reader in advance. This happens in cases\nwhere a particular form is valid in two different dialects and one\nor both parses are sufficiently divergent that it is not possible\nto recover the correct parse for the other dialect.\n\nAt the moment the reader cannot be modified at runtime. This means\nthat it cannot be used to parse named readtables or things like\nRacket's ~#lang at-exp~. In the future it might be possible to pass\nthe generator through to the functions that handle dispatch macros and\nmake it possible to register new reader macros, but that is well out\nof scope at the moment.\n\nOriginally this library was intended to be vendored, however since\nthen it is significantly increased in complexity. A stripped down\nversion that is suitable for parsing a minimal sane dialect that can\nbe used as a configuration language may be added in the future.\n\n* Usage\n#+begin_src bash\ngit clone https://github.com/tgbugs/sxpyr.git\npushd sxpyr\npip install --user -e .[cli]\npython -m sxpyr.cli parse test/data/nested-splicing-unsyntax.rkt test/data/plist-test.sxpr\npopd\n#+end_src\n\n* Testing\nIn order to run the tests you will need the following repos.\n- sbcl\n- ccl\n- emacs\n- org-mode\n- clojure\n- spec.alpha\n- code.specs.alpha\n- tawny-owl\n- hy\n- Fennel\n- txr\n\nFor now you will also need the following installed on your system.\n- xemacs\n- app-xemacs/xemacs-packages-all\n- guile\n- gambit\n* Fuzzing\nYou will need a copy of afl.\n#+begin_src bash\npip install --user -e .[fuzz]\npushd test\nmkdir initial-inputs\ncp data/plist-test.sxpr initial-inputs/ # other seed files can be added as well\npy-afl-fuzz -m 400 -i initial-inputs/ -o fuzzing-results/ -- python -m sxpyr.cli parse --fuzz\n#+end_src\n\n#+begin_src bash\npython -m sxpyr.cli parse test/fuzzing-results/crashes/*\n#+end_src\n\nNote that initial inputs cannot be symlinks becuase afl creates hardlinks to the input files.\n\nAlso note that afl seems to die after 175k execs?\n\nSometimes it also seems that some errors are escaping the except block\nand making it be classified as crashes?\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A flexible Lisp reader.",
"version": "0.0.4",
"project_urls": {
"Homepage": "https://github.com/tgbugs/sxpyr"
},
"split_keywords": [
"lisp",
"reader",
"edn",
"clojure",
"racket",
"scheme",
"elisp",
"emacs",
"sexp",
"s-expression",
"parser",
"parsing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9063134e7feb2b913b81f038d6853b6bcb37feccf29d4af084e1428a2c665aad",
"md5": "e4bd4e11fce7abe81a89bd288d893541",
"sha256": "b0e4d824055b8ae4e426ebd5e2e61976f973fdcff28af901f38c261dedb3a142"
},
"downloads": -1,
"filename": "sxpyr-0.0.4-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "e4bd4e11fce7abe81a89bd288d893541",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.6",
"size": 32760,
"upload_time": "2023-07-27T23:29:18",
"upload_time_iso_8601": "2023-07-27T23:29:18.087725Z",
"url": "https://files.pythonhosted.org/packages/90/63/134e7feb2b913b81f038d6853b6bcb37feccf29d4af084e1428a2c665aad/sxpyr-0.0.4-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "63df014c20a5c4b1e8335a58c8e59d1d88389804ab153df9a117644dad2eb995",
"md5": "7e6695c7fc81f5a02a055aec8dd2fe19",
"sha256": "68a070b4996eb514cb878a209e15fa4797389a7b10e6172cb21353f02f5140f9"
},
"downloads": -1,
"filename": "sxpyr-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "7e6695c7fc81f5a02a055aec8dd2fe19",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 84153,
"upload_time": "2023-07-27T23:29:19",
"upload_time_iso_8601": "2023-07-27T23:29:19.881663Z",
"url": "https://files.pythonhosted.org/packages/63/df/014c20a5c4b1e8335a58c8e59d1d88389804ab153df9a117644dad2eb995/sxpyr-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-27 23:29:19",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tgbugs",
"github_project": "sxpyr",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "sxpyr"
}
JSON
