# py-overlay-evm
Ethereum VM written in Python with minimal external dependencies and aimed at a mock execution of smart contracts.
## Rationale
This project grew from what was essentially an exercise to understand how smart contracts work on the Ethereum blockchain and its clones. One way to gain understanding is to program an EVM from scratch, without referencing existing code, and that is precisely what I did. The codebase includes an implementation for the Ethereum bytecodes, including the corresponding gas calculations, own Keccak-256 implementation, as well as a Solidity wrapper for conveniently calling functions in Solidity contracts.
As of now, the implementation of the EVM is incomplete, and the API is still likely to change. I reckon the code is good enough to play around with, but it is certainly not ready for production (for instance, there is zero test coverage). The examples below demonstrate what can be done now.
## Features
- *Minimal external dependencies:* numpy, requests.
- *Functional style state changes:* each call to a smart contract returns a new view of the blockchain with the respective changes implemented, the initial view from which the call started remains available and unchanged.
- *Overlay architecture:* data is read from the (public) node if these data have not been previously accessed, further reads are cached, and any changes are kept in an overlay in memory.
- *Gas calculations:* gas usage is calculated exactly (work in progress).
## Examples
The following examples all use the Ethereum blockchain.
### Example I (manual contract calls)
```python
from py_overlay_evm.rpc import Node
from py_overlay_evm.evm import Chain, execute
from py_overlay_evm.keccak import keccak
# Initialize a link to a public node
#
# During initialization, the current block number is noted, and all the
# subsequent requests to the chain are done for the same block number. This
# behaviour ensures consistency but it also means that all the subsequent
# requests must be done within a short time interval, because public nodes do
# not return data for somewhat older blocks.
#
# For testing purposes, we can use the public nodes from the Flashbots project.
url = "https://rpc.flashbots.net"
chain = Chain(Node(url, verbose=True))
# We consider a Wrapped Ether contract for this example
#
# Addresses are always encoded as integers.
weth = int("0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2", 16)
# We will be calling the contract from a mock address
caller = int("0xabababababababababababababababababababab", 16)
# We are going to call the `name()` method on the contract. The method accepts
# zero arguments. According to the Solidity ABI, we need to compute the Keccak
# signature of "name()", take its first 4 bytes and pass it as the byte input
# to the contract.
data = keccak(b'name()')[:4]
rslt = execute(
chain = chain,
caller = caller,
address = weth,
value = 0,
data = data,
trace = True,
)
# `rslt.data` contains the string "Wrapped Ether" encoded according to the
# Solidity ABI
print(rslt.data)
# prints an equivalent of:
# bytearray(b'
# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x20
# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0d
# Wrapped Ether\x00\x00\x00
# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
# ')
# `rslt.chain` contains the new state of the blockchain after a successful
# execution of a contract. In this case, given that the method `name()` is
# read-only, `rslt.chain` will be equivalent to `chain`.
# `rslt.gas` reports the exact gas used by the transaction. Please note that
# gas calculations have not been extensively tested and could well be
# erroneous.
print(rslt.gas) # prints "24174"
# Finally, `rslt.trace` contains the trace of the contract execution.
print(len(rslt.trace)) # prints "228"
for op in rslt.trace:
print(op)
# prints:
# 0x60 push1() -> 0x60, gas: 3
# 0x60 push1() -> 0x40, gas: 3
# 0x52 mstore(0x40, 0x60), gas: 12
# 0x60 push1() -> 0x4, gas: 3
# 0x36 calldatasize() -> 0x4, gas: 2
# ...
# 0x80 dup1() -> 0xa0, gas: 3
# 0x91 swap2(), gas: 3
# 0x03 sub(0x100, 0xa0) -> 0x60, gas: 3
# 0x90 swap1(), gas: 3
# 0xf3 op_return(0xa0, 0x60), gas: 0
# In this case there are no external calls to other contracts, but if there are
# such calls, their traces also get recorded.
```
### Example II (Solidity wrapper)
For conveniently calling Solidity contracts, there is a wrapper that handles the encoding of the call signature as well as the translation of the data from Python types to Solidity binary format and back. Here, we repeat the first example but using the Solidity wrapper.
```python
from py_overlay_evm.rpc import Node
from py_overlay_evm.evm import Chain, execute, mkcall
from py_overlay_evm.solidity import solidity, string
# Initialize a link to a public node
url = "https://rpc.flashbots.net"
chain = Chain(Node(url, verbose=True))
# We consider a Wrapped Ether contract for this example
weth = int("0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2", 16)
# We will be calling the contract from a mock address
caller = int("0xabababababababababababababababababababab", 16)
# To use the Solidity wrapper, we simply define a Python function with the same
# name as the Solidity function and we type annotate it using Solidity type
# names. The body of the function does not matter, so we'll just use `pass`.
@solidity
def name() -> string:
pass
# A wrapped function accepts a partial call to `execute()` as the first
# parameter, and the remaining parameters are simply the parameters for the
# Solidity function, in this case none.
rslt = name(
lambda data: execute(chain, caller, weth, 0, data, trace=True),
)
print(rslt.value) # prints "Wrapped Ether"
# To simplify making a partial call to `execute()` there is also `mkcall()`:
rslt = name(
mkcall(chain, caller, weth, 0, trace=True),
)
```
### Example III (Uniswap V3 router)
This is a longer example that walks through a non-trivial operation, namely depositing mock WETH coins to a mock account, and then changing them into USDT coins via a Uniswap V3 router. This example also demonstrates more extensive usage of the Solidity wrapper.
```python
from dataclasses import dataclass
from py_overlay_evm.evm import Chain, mkcall, save_trace
from py_overlay_evm.rpc import Node
from py_overlay_evm.solidity import solidity, address
from py_overlay_evm.solidity import uint24, uint160, uint256
## Firstly, we define all the Solidity functions that we will need
@solidity
def deposit():
pass
@solidity
def balanceOf(address: address) -> uint256:
pass
@solidity
def approve(
guy: address,
wad: uint256,
) -> None:
pass
# A dataclass can be used if a Solidity function wants a structure as a
# parameter. The dataclass can have an arbitrary name, because the name does
# not form a part of the function's signature.
@dataclass
class SwapParams:
tokenIn: address
tokenOut: address
fee: uint24
recipient: address
deadline: uint256
amountIn: uint256
amountOutMinimum: uint256
sqrtPriceLimitX96: uint160
@solidity
def exactInputSingle(params: SwapParams) -> uint256:
pass
## Secondly, we define all the required addresses
# Tokens
weth = int("0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2", 16)
usdt = int("0xdac17f958d2ee523a2206206994597c13d831ec7", 16)
# Uniswap V2 router
router = int("0xe592427a0aece92de3edee1f18e0157c05861564", 16)
# Mock holder address
holder = int("0xabababababababababababababababababababab", 16)
## Finally, we prepare mock WETH and change them into USDT
# Initialize a link to a public node
url = "https://rpc.flashbots.net"
chain = Chain(Node(url, verbose=True))
# Create an unlimited supply of ETH at the mock address :3
chain[holder].balance = (1 << 256) - 1
# Deposit 100 ETH to WETH
amount = 100*10**18
rslt1 = deposit(
mkcall(chain, holder, weth, amount),
)
# Check that the deposit operation was successful
x = balanceOf(
mkcall(rslt1.chain, holder, weth, 0),
address = holder,
).value
print(f'WETH balance: {x/10**18}')
# Approve withdrawal of 100 WETH
rslt2 = approve(
mkcall(rslt1.chain, holder, weth, 0),
guy = router,
wad = amount,
)
# We use the block's timestamp plus 60 seconds as a deadline
deadline = chain.node.block_timestamp() + 60
# Exchange 100 WETH to USDT
rslt3 = exactInputSingle(
mkcall(rslt2.chain, holder, router, 0, trace=True),
params = SwapParams(
tokenIn = weth,
tokenOut = usdt,
fee = 500, # 0.05%
recipient = holder,
deadline = deadline,
amountIn = amount,
amountOutMinimum = 0,
sqrtPriceLimitX96 = 0,
),
)
print(f'USDT amount out: {rslt3.value/10**6:.2f}')
# prints "USDT amount out: 256474.98" at the time or writing
# The trace is large (8,402 operations at the time of writing, but that will
# differ depending on how many ticks get traversed), so we save it to a file for
# viewing in an external editor
save_trace(rslt3.trace, 'swap.trace')
# Check final WETH balance
x = balanceOf(
mkcall(rslt3.chain, holder, weth, 0),
address = holder,
).value
print(f'WETH balance: {x/10**18:.2f}') # prints "0.00"
# Check final USDT balance
x = balanceOf(
mkcall(rslt3.chain, holder, usdt, 0),
address = holder,
).value
print(f'USDT balance: {x/10**6:.2f}')
# prints "USDT balance: 256474.98" at the time or writing
```
## Roadmap
In case I happen to have time to further work on this project, the overall roadmap is as follows:
- [x] Gas calculations
- [x] Python package
- [ ] Full coverage of EVM bytecodes
- [ ] Tests for EVM bytecodes
- [ ] Full coverage of Solidity datatypes
- [ ] Tests for Solidity datatypes
- [ ] Documentation
Raw data
{
"_id": null,
"home_page": "",
"name": "py-overlay-evm",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "evm,ethereum virtual machine,smart contract,simulation",
"author": "",
"author_email": "Andrei Dubovik <andrei@dubovik.eu>",
"download_url": "https://files.pythonhosted.org/packages/3a/2c/49297312167090285462313883a9c74061711ac80864159956411e56c430/py-overlay-evm-0.1.0.tar.gz",
"platform": null,
"description": "# py-overlay-evm\n\nEthereum VM written in Python with minimal external dependencies and aimed at a mock execution of smart contracts.\n\n## Rationale\n\nThis project grew from what was essentially an exercise to understand how smart contracts work on the Ethereum blockchain and its clones. One way to gain understanding is to program an EVM from scratch, without referencing existing code, and that is precisely what I did. The codebase includes an implementation for the Ethereum bytecodes, including the corresponding gas calculations, own Keccak-256 implementation, as well as a Solidity wrapper for conveniently calling functions in Solidity contracts.\n\nAs of now, the implementation of the EVM is incomplete, and the API is still likely to change. I reckon the code is good enough to play around with, but it is certainly not ready for production (for instance, there is zero test coverage). The examples below demonstrate what can be done now.\n\n## Features\n- *Minimal external dependencies:* numpy, requests.\n- *Functional style state changes:* each call to a smart contract returns a new view of the blockchain with the respective changes implemented, the initial view from which the call started remains available and unchanged.\n- *Overlay architecture:* data is read from the (public) node if these data have not been previously accessed, further reads are cached, and any changes are kept in an overlay in memory.\n- *Gas calculations:* gas usage is calculated exactly (work in progress).\n\n## Examples\nThe following examples all use the Ethereum blockchain.\n\n### Example I (manual contract calls)\n\n```python\nfrom py_overlay_evm.rpc import Node\nfrom py_overlay_evm.evm import Chain, execute\nfrom py_overlay_evm.keccak import keccak\n\n# Initialize a link to a public node\n# \n# During initialization, the current block number is noted, and all the\n# subsequent requests to the chain are done for the same block number. This\n# behaviour ensures consistency but it also means that all the subsequent\n# requests must be done within a short time interval, because public nodes do\n# not return data for somewhat older blocks.\n#\n# For testing purposes, we can use the public nodes from the Flashbots project.\nurl = \"https://rpc.flashbots.net\"\nchain = Chain(Node(url, verbose=True))\n\n# We consider a Wrapped Ether contract for this example\n#\n# Addresses are always encoded as integers.\nweth = int(\"0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2\", 16)\n\n# We will be calling the contract from a mock address\ncaller = int(\"0xabababababababababababababababababababab\", 16)\n\n# We are going to call the `name()` method on the contract. The method accepts\n# zero arguments. According to the Solidity ABI, we need to compute the Keccak\n# signature of \"name()\", take its first 4 bytes and pass it as the byte input\n# to the contract.\ndata = keccak(b'name()')[:4]\nrslt = execute(\n chain = chain,\n caller = caller,\n address = weth,\n value = 0,\n data = data,\n trace = True,\n)\n\n# `rslt.data` contains the string \"Wrapped Ether\" encoded according to the\n# Solidity ABI\nprint(rslt.data)\n\n# prints an equivalent of:\n# bytearray(b'\n# \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\n# \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x20\n# \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\n# \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x0d\n# Wrapped Ether\\x00\\x00\\x00\n# \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\n# ')\n\n# `rslt.chain` contains the new state of the blockchain after a successful\n# execution of a contract. In this case, given that the method `name()` is\n# read-only, `rslt.chain` will be equivalent to `chain`.\n\n# `rslt.gas` reports the exact gas used by the transaction. Please note that\n# gas calculations have not been extensively tested and could well be\n# erroneous.\nprint(rslt.gas) # prints \"24174\"\n\n# Finally, `rslt.trace` contains the trace of the contract execution.\nprint(len(rslt.trace)) # prints \"228\"\n\nfor op in rslt.trace:\n print(op)\n\n# prints:\n# 0x60 push1() -> 0x60, gas: 3\n# 0x60 push1() -> 0x40, gas: 3\n# 0x52 mstore(0x40, 0x60), gas: 12\n# 0x60 push1() -> 0x4, gas: 3\n# 0x36 calldatasize() -> 0x4, gas: 2\n# ...\n# 0x80 dup1() -> 0xa0, gas: 3\n# 0x91 swap2(), gas: 3\n# 0x03 sub(0x100, 0xa0) -> 0x60, gas: 3\n# 0x90 swap1(), gas: 3\n# 0xf3 op_return(0xa0, 0x60), gas: 0\n\n# In this case there are no external calls to other contracts, but if there are\n# such calls, their traces also get recorded.\n```\n\n### Example II (Solidity wrapper)\n\nFor conveniently calling Solidity contracts, there is a wrapper that handles the encoding of the call signature as well as the translation of the data from Python types to Solidity binary format and back. Here, we repeat the first example but using the Solidity wrapper.\n\n```python\nfrom py_overlay_evm.rpc import Node\nfrom py_overlay_evm.evm import Chain, execute, mkcall\nfrom py_overlay_evm.solidity import solidity, string\n\n# Initialize a link to a public node\nurl = \"https://rpc.flashbots.net\"\nchain = Chain(Node(url, verbose=True))\n\n# We consider a Wrapped Ether contract for this example\nweth = int(\"0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2\", 16)\n\n# We will be calling the contract from a mock address\ncaller = int(\"0xabababababababababababababababababababab\", 16)\n\n# To use the Solidity wrapper, we simply define a Python function with the same\n# name as the Solidity function and we type annotate it using Solidity type\n# names. The body of the function does not matter, so we'll just use `pass`.\n@solidity\ndef name() -> string:\n pass\n\n# A wrapped function accepts a partial call to `execute()` as the first\n# parameter, and the remaining parameters are simply the parameters for the\n# Solidity function, in this case none. \nrslt = name(\n lambda data: execute(chain, caller, weth, 0, data, trace=True),\n)\nprint(rslt.value) # prints \"Wrapped Ether\"\n\n# To simplify making a partial call to `execute()` there is also `mkcall()`:\nrslt = name(\n mkcall(chain, caller, weth, 0, trace=True),\n)\n```\n\n### Example III (Uniswap V3 router)\n\nThis is a longer example that walks through a non-trivial operation, namely depositing mock WETH coins to a mock account, and then changing them into USDT coins via a Uniswap V3 router. This example also demonstrates more extensive usage of the Solidity wrapper.\n\n```python\nfrom dataclasses import dataclass\n\nfrom py_overlay_evm.evm import Chain, mkcall, save_trace\nfrom py_overlay_evm.rpc import Node\nfrom py_overlay_evm.solidity import solidity, address\nfrom py_overlay_evm.solidity import uint24, uint160, uint256\n\n\n## Firstly, we define all the Solidity functions that we will need\n\n@solidity\ndef deposit():\n pass\n\n@solidity\ndef balanceOf(address: address) -> uint256:\n pass\n\n@solidity\ndef approve(\n guy: address,\n wad: uint256,\n) -> None:\n pass\n\n# A dataclass can be used if a Solidity function wants a structure as a\n# parameter. The dataclass can have an arbitrary name, because the name does\n# not form a part of the function's signature.\n@dataclass\nclass SwapParams:\n tokenIn: address\n tokenOut: address\n fee: uint24\n recipient: address\n deadline: uint256\n amountIn: uint256\n amountOutMinimum: uint256\n sqrtPriceLimitX96: uint160\n\n@solidity\ndef exactInputSingle(params: SwapParams) -> uint256:\n pass\n\n\n## Secondly, we define all the required addresses\n\n# Tokens\nweth = int(\"0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2\", 16)\nusdt = int(\"0xdac17f958d2ee523a2206206994597c13d831ec7\", 16)\n\n# Uniswap V2 router\nrouter = int(\"0xe592427a0aece92de3edee1f18e0157c05861564\", 16)\n\n# Mock holder address\nholder = int(\"0xabababababababababababababababababababab\", 16)\n\n## Finally, we prepare mock WETH and change them into USDT\n\n# Initialize a link to a public node\nurl = \"https://rpc.flashbots.net\"\nchain = Chain(Node(url, verbose=True))\n\n# Create an unlimited supply of ETH at the mock address :3\nchain[holder].balance = (1 << 256) - 1\n\n# Deposit 100 ETH to WETH\namount = 100*10**18\nrslt1 = deposit(\n mkcall(chain, holder, weth, amount),\n)\n\n# Check that the deposit operation was successful\nx = balanceOf(\n mkcall(rslt1.chain, holder, weth, 0),\n address = holder,\n).value\nprint(f'WETH balance: {x/10**18}')\n\n# Approve withdrawal of 100 WETH\nrslt2 = approve(\n mkcall(rslt1.chain, holder, weth, 0),\n guy = router,\n wad = amount,\n)\n\n# We use the block's timestamp plus 60 seconds as a deadline\ndeadline = chain.node.block_timestamp() + 60\n\n# Exchange 100 WETH to USDT\nrslt3 = exactInputSingle(\n mkcall(rslt2.chain, holder, router, 0, trace=True),\n params = SwapParams(\n tokenIn = weth,\n tokenOut = usdt,\n fee = 500, # 0.05%\n recipient = holder,\n deadline = deadline,\n amountIn = amount,\n amountOutMinimum = 0,\n sqrtPriceLimitX96 = 0,\n ),\n)\nprint(f'USDT amount out: {rslt3.value/10**6:.2f}')\n# prints \"USDT amount out: 256474.98\" at the time or writing\n\n# The trace is large (8,402 operations at the time of writing, but that will\n# differ depending on how many ticks get traversed), so we save it to a file for\n# viewing in an external editor\nsave_trace(rslt3.trace, 'swap.trace')\n\n# Check final WETH balance\nx = balanceOf(\n mkcall(rslt3.chain, holder, weth, 0),\n address = holder,\n).value\nprint(f'WETH balance: {x/10**18:.2f}') # prints \"0.00\"\n\n# Check final USDT balance\nx = balanceOf(\n mkcall(rslt3.chain, holder, usdt, 0),\n address = holder,\n).value\nprint(f'USDT balance: {x/10**6:.2f}')\n# prints \"USDT balance: 256474.98\" at the time or writing\n```\n\n## Roadmap\n\nIn case I happen to have time to further work on this project, the overall roadmap is as follows:\n- [x] Gas calculations\n- [x] Python package\n- [ ] Full coverage of EVM bytecodes\n- [ ] Tests for EVM bytecodes\n- [ ] Full coverage of Solidity datatypes\n- [ ] Tests for Solidity datatypes\n- [ ] Documentation\n",
"bugtrack_url": null,
"license": "",
"summary": "Ethereum VM aimed at a mock execution of smart contracts",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/andrei-dubovik/py-overlay-evm",
"Repository": "https://github.com/andrei-dubovik/py-overlay-evm.git"
},
"split_keywords": [
"evm",
"ethereum virtual machine",
"smart contract",
"simulation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "db9472b5e14e5e0882cc1eea52072735e794f12f1a6cd9ae604f790e57d1f0ca",
"md5": "301ac2f7d7daa72d2d3a6194914f86d4",
"sha256": "cc84f2478c776d608ba3f6a045542c314cf166649365b9bcc6208faf7a29775f"
},
"downloads": -1,
"filename": "py_overlay_evm-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "301ac2f7d7daa72d2d3a6194914f86d4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 29137,
"upload_time": "2024-01-14T17:47:15",
"upload_time_iso_8601": "2024-01-14T17:47:15.896155Z",
"url": "https://files.pythonhosted.org/packages/db/94/72b5e14e5e0882cc1eea52072735e794f12f1a6cd9ae604f790e57d1f0ca/py_overlay_evm-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3a2c49297312167090285462313883a9c74061711ac80864159956411e56c430",
"md5": "184bb9a4d1de8cda9913d7aebcaec605",
"sha256": "75511cd86e6a5c5947fcdf6d57f056c2048141826959f0502bdb7d51981c98dd"
},
"downloads": -1,
"filename": "py-overlay-evm-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "184bb9a4d1de8cda9913d7aebcaec605",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 31291,
"upload_time": "2024-01-14T17:47:18",
"upload_time_iso_8601": "2024-01-14T17:47:18.667708Z",
"url": "https://files.pythonhosted.org/packages/3a/2c/49297312167090285462313883a9c74061711ac80864159956411e56c430/py-overlay-evm-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-14 17:47:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "andrei-dubovik",
"github_project": "py-overlay-evm",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "py-overlay-evm"
}
JSON
