## **Optimal Portfolios Backtester** <a name="analytics"></a>
optimalportfolios package implements analytics for backtesting of optimal portfolios including:
1. computing of inputs (covariance matrices, returns) for roll forward
computations (to avoid hindsight bias) and for generation of rolling optimal portfolios
2. implementation of core optimisation solvers:
1. Minimum variance
2. Maximum quadratic utility
3. Equal risk contribution
4. Maximum diversification
5. Maximum Sharpe ratio
6. Maximum Cara utility under Gaussian mixture model
7. Tracking error minimisation
3. the rolling backtests are compatible with incomplete time series
for roll forward analysis so that the portfolio universe can include instruments with different price histories
4. computing performances of simulated portfolios
5. reporting of backtested portfolios and cross-sectional analysis
OptimalPortfolios package is split into 5 main modules with the
dependecy path increasing sequentially as follows.
1. ```optimisation``` with sub-package ```solvers``` contains implementation of
various quadratic and nonlinear solvers. Each solver is implemented
in a module independently from other solvers.
2. ```reports``` is module for computing performance statistics and performance attribution including returns, volatilities, etc.
3. ```examples.solvers``` provides example of running all implemented solvers.
4. ```examples.crypto_allocation``` is module for computations and visualisations for
paper "Optimal Allocation to Cryptocurrencies in Diversified Portfolios" [https://ssrn.com/abstract=4217841](https://ssrn.com/abstract=4217841)
(see paper for description of the rolling-forward methodology and estimation of inputs)
# Table of contents
1. [Analytics](#analytics)
2. [Installation](#installation)
3. [Portfolio Optimisers](#optimisers)
1. [Implementation structure](#structure)
2. [Example of implementation for Maximum Diversification Solver](#example_structure)
3. [Constraints](#constraints)
4. [Wrapper for implemented rolling portfolios](#wrapper)
5. [Adding an optimiser](#adding)
6. [Default parameters](#params)
7. [Price time series data](#ts)
4. [Examples](#examples)
1. [Optimal Portfolio Backtest](#optimal)
2. [Customised reporting](#report)
3. [Parameters sensitivity backtest](#sensitivity)
4. [Multi optimisers cross backtest](#cross)
5. [Optimal allocation to cryptocurrencies](#crypto)
5. [Contributions](#contributions)
6. [Updates](#updates)
7. [Disclaimer](#disclaimer)
## **Installation** <a name="installation"></a>
install using
```python
pip install optimalportfolios
```
upgrade using
```python
pip install --upgrade optimalportfolios
```
close using
```python
git clone https://github.com/ArturSepp/OptimalPortfolios.git
```
Core dependencies:
python = ">=3.8,<3.11",
numba = ">=0.56.4",
numpy = ">=1.22.4",
scipy = ">=1.9.0",
pandas = ">=2.2.2",
matplotlib = ">=3.2.2",
seaborn = ">=0.12.2",
scikit_learn = ">=1.3.0",
cvxpy = ">=1.3.2",
qis = ">=2.1.1",
Optional dependencies:
yfinance ">=0.2.3" (for getting test price data),
pybloqs ">=1.2.13" (for producing html and pdf factsheets)
To use pybloqs for pandas > 2.x,
locate file "...\Lib\site-packages\pybloqs\jinja\table.html" and change line 44 from:
{% for col_name, cell in row.iteritems() %}
to:
{% for col_name, cell in row.items() %}
## **Portfolio optimisers** <a name="optimisers"></a>
### 1. Implementation structure <a name="structure"></a>
The implementation of each solver is split into the 3 layers:
1) Mathematical layer which takes clean inputs, formulates the optimisation
problem and solves the
optimisation problem using Scipy or CVXPY solvers.
The logic of this layer is to solve the problem algorithmically by taking clean inputs.
2) Wrapper level which takes inputs potentially containing nans,
filters them out and calls the solver in layer 1). The output weights of filtered out
assets are set to zero.
3) Rolling level function with takes price time series as inputs and implements
the estimation of covariance matrix and other inputs on roll-forward basis.
For each update date the rolling layer call the wrapper layer 2) with estimated
inputs as of the update date.
The recommended usage is as follows.
Layer 2) is used for live portfolios or for backtests which are implemented using
data augmentation
Layer 3) is applied for roll forward backtests where all available data is processed
using roll forward analysis
For implementation of different estimation methods for covariance matrices
and other inputs, we recommend to implement specific layers 2) or 3) with the
implementation of the estimation logic.
Layer 1 works with provided covariance matrices and inputs.
### 2. Example of implementation for Maximum Diversification Solver <a name="example_structure"></a>
Using example of ```optimization.solvers.max_diversification.py```
1. Scipy solver ```opt_maximise_diversification()``` which takes "clean" inputs of the
covariance matrix of type ```np.ndarray``` without nans and
```Constraints``` dataclass which implements constraints for the solver.
The lowest level of each optimisation method is ```opt_...``` or ```cvx_...``` function taking clean inputs and producing the optimal weights.
The logic of this layer is to implement pure quant logic for the optimiser with cvx solver.
2. Wrapper function ```wrapper_maximise_diversification()``` which takes inputs
covariance matrix of type ```pd.Dataframe```
potentially containing nans or assets with zero variance (when their time series are missing in the
estimation period) and filters out non-nan "clean" inputs and
updated constraints for OPT/CVX solver in layer 1.
The intermediary level of each optimisation method is ```wrapper_...``` function taking
"dirty" inputs, filtering inputs, and producing the optimal weights. This wrapper can be called either
by rolling backtest simulations or by live portfolios for rebalancing.
The logic of this layer is to filter out data and to be an interface for portfolio implementations.
3. Rolling optimiser function ```rolling_maximise_diversification()``` takes the time series of data
and sliced these accordingly and at each rebalancing step call the wrapper in layer 2.
In the end, the function outputs the time series of optiomal weight of assets in the universe.
Price data of assets may have gaps and nans which is taken care of in the wrapper level.
The backtesting of each optimisation method is implemented with ```rolling_...``` method with produces the time series of
optimal portfolio weights.
The logic of this layer is to faciliate the backtest of portfolio optimisation method and to produce
time series of portfolio weights using a Markovian setup. These weights are applied for the backtest
of the optimal portfolio and the underlying strategy
Each module in ```optimization.solver``` implements specific optimisers and estimators for their inputs.
### 3. Constraints <a name="constraints"></a>
Dataclass ```Constraints``` in ```optimization.constraints``` implements
optimisation constraints in solver independent way.
The following inputs for various constraints are implemented.
```python
@dataclass
class Constraints:
is_long_only: bool = True # for positive allocation weights
min_weights: pd.Series = None # instrument min weights
max_weights: pd.Series = None # instrument max weights
max_exposure: float = 1.0 # for long short portfolios: for long_portfolios = 1
min_exposure: float = 1.0 # for long short portfolios: for long_portfolios = 1
benchmark_weights: pd.Series = None # for minimisation of tracking error
tracking_err_vol_constraint: float = None # annualised sqrt tracking error
weights_0: pd.Series = None # for turnover constraints
turnover_constraint: float = None # for turnover constraints
target_return: float = None # for optimisation with target return
asset_returns: pd.Series = None # for optimisation with target return
max_target_portfolio_vol_an: float = None # for optimisation with maximum portfolio volatility target
min_target_portfolio_vol_an: float = None # for optimisation with maximum portfolio volatility target
group_lower_upper_constraints: GroupLowerUpperConstraints = None # for group allocations constraints
```
Dataclass ```GroupLowerUpperConstraints``` implements asset class loading and min and max allocations
```python
@dataclass
class GroupLowerUpperConstraints:
"""
add constraints that each asset group is group_min_allocation <= sum group weights <= group_max_allocation
"""
group_loadings: pd.DataFrame # columns=instruments, index=groups, data=1 if instrument in indexed group else 0
group_min_allocation: pd.Series # index=groups, data=group min allocation
group_max_allocation: pd.Series # index=groups, data=group max allocation
```
Constraints are updated on the wrapper level to include the valid tickers
```python
def update_with_valid_tickers(self, valid_tickers: List[str]) -> Constraints:
```
On the solver layer, the constants for the solvers are requested as follows.
For Scipy: ```set_scipy_constraints(self, covar: np.ndarray = None) -> List```
For CVXPY: ```set_cvx_constraints(self, w: cvx.Variable, covar: np.ndarray = None) -> List```
### 4. Wrapper for implemented rolling portfolios <a name="wrapper"></a>
Module ```optimisation.wrapper_rolling_portfolios.py``` wraps implementation of
of the following solvers enumerated in ```config.py```
Using the wrapper function allows for cross-sectional analysis of different
backtest methods and for sensitivity analysis to parameters of
estimation and solver methods.
```python
class PortfolioObjective(Enum):
"""
implemented portfolios in rolling_engine
"""
# risk-based:
MAX_DIVERSIFICATION = 1 # maximum diversification measure
EQUAL_RISK_CONTRIBUTION = 2 # implementation in risk_parity
MIN_VARIANCE = 3 # min w^t @ covar @ w
# return-risk based
QUADRATIC_UTILITY = 4 # max means^t*w- 0.5*gamma*w^t*covar*w
MAXIMUM_SHARPE_RATIO = 5 # max means^t*w / sqrt(*w^t*covar*w)
# return-skeweness based
MAX_CARA_MIXTURE = 6 # carra for mixture distributions
```
See examples for [Parameters sensitivity backtest](#sensitivity) and
[Multi optimisers cross backtest](#cross)
### 5. Adding an optimiser <a name="adding"></a>
1. Add analytics for computing rolling weights using a new estimator in
subpackage ```optimization.solvers```. Any third-party packages can be used
2. For cross-sectional analysis, add new optimiser type
to ```config.py``` and link implemented
optimiser in wrapper function ```compute_rolling_optimal_weights()``` in
```optimisation.wrapper_rolling_portfolios.py```
### 6. Default parameters <a name="params"></a>
Key parameters include the specification of the estimation sample.
1. ```returns_freq``` defines the frequency of returns for covariance matrix estimation. This parameter affects all methods.
The default (assuming daily price data) is weekly Wednesday returns ```returns_freq = 'W-WED'```.
For price data with monthly observations
(such us hedged funds), monthly returns should be used ```returns_freq = 'ME'```.
2. ```span``` defines the estimation span for ewma covariance matrix. This parameter affects all methods which use
EWMA covariance matrix:
```
PortfolioObjective in [MAX_DIVERSIFICATION, EQUAL_RISK_CONTRIBUTION, MIN_VARIANCE]
```
and
```
PortfolioObjective in [QUADRATIC_UTILITY, MAXIMUM_SHARPE_RATIO]
```
The span is defined as the number of returns
for the half-life of EWMA filter: ```ewma_lambda = 1 - 2 / (span+1)```. ```span=52``` with weekly returns means that
last 52 weekly returns (one year of data) contribute 50% of weight to estimated covariance matrix
The default (assuming weekly returns) is 52: ```span=52```.
For monthly returns, I recommend to use ```span=12``` or ```span=24```.
3. ```rebalancing_freq``` defines the frequency of weights update. This parameter affects all methods.
The default value is quarterly rebalancing ```rebalancing_freq='QE'```.
For the following methods
```
PortfolioObjective in [QUADRATIC_UTILITY, MAXIMUM_SHARPE_RATIO, MAX_CARA_MIXTURE]
```
Rebalancing frequency is also the rolling sample update frequency when mean returns and mixture distributions are estimated.
4. ```roll_window``` defines the number of past returns applied for estimation of rolling mean returns and mixture distributions.
This parameter affects the following optimisers
```
PortfolioObjective in [QUADRATIC_UTILITY, MAXIMUM_SHARPE_RATIO, MAX_CARA_MIXTURE]
```
and it is linked to ```rebalancing_freq```.
Default value is ```roll_window=20``` which means that data for past 20 (quarters) are used in the sample
with ```rebalancing_freq='QE'```
For monthly rebalancing, I recomend to use ```roll_window=60``` which corresponds to using past 5 years of data
### 7. Price time series data <a name="ts"></a>
The input to all optimisers is dataframe prices which contains dividend and split adjusted prices.
The price data can include assets with prices starting an ending at different times.
All optimisers will set maximum weight to zero for assets with missing prices in the estimation sample period.
## **Examples** <a name="examples"></a>
### 1. Optimal Portfolio Backtest <a name="optimal"></a>
See script in ```optimalportfolios.examples.optimal_portfolio_backtest.py```
```python
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import yfinance as yf
from typing import Tuple
import qis as qis
# package
from optimalportfolios import compute_rolling_optimal_weights, PortfolioObjective, Constraints
def fetch_universe_data() -> Tuple[pd.DataFrame, pd.DataFrame, pd.Series]:
"""
fetch universe data for the portfolio construction:
1. dividend and split adjusted end of day prices: price data may start / end at different dates
2. benchmark prices which is used for portfolio reporting and benchmarking
3. universe group data for portfolio reporting and risk attribution for large universes
this function is using yfinance to fetch the price data
"""
universe_data = dict(SPY='Equities',
QQQ='Equities',
EEM='Equities',
TLT='Bonds',
IEF='Bonds',
LQD='Credit',
HYG='HighYield',
GLD='Gold')
tickers = list(universe_data.keys())
group_data = pd.Series(universe_data)
prices = yf.download(tickers, start=None, end=None, ignore_tz=True)['Adj Close']
prices = prices[tickers] # arrange as given
prices = prices.asfreq('B', method='ffill') # refill at B frequency
benchmark_prices = prices[['SPY', 'TLT']]
return prices, benchmark_prices, group_data
# 2. get universe data
prices, benchmark_prices, group_data = fetch_universe_data()
time_period = qis.TimePeriod('31Dec2004', '16Aug2024') # period for computing weights backtest
# 3.a. define optimisation setup
portfolio_objective = PortfolioObjective.MAX_DIVERSIFICATION # define portfolio objective
returns_freq = 'W-WED' # use weekly returns
rebalancing_freq = 'QE' # weights rebalancing frequency: rebalancing is quarterly on WED
span = 52 # span of number of returns_freq-returns for covariance estimation = 12y
constraints0 = Constraints(is_long_only=True,
min_weights=pd.Series(0.0, index=prices.columns),
max_weights=pd.Series(0.5, index=prices.columns))
# 3.b. compute solvers portfolio weights rebalanced every quarter
weights = compute_rolling_optimal_weights(prices=prices,
portfolio_objective=portfolio_objective,
constraints0=constraints0,
time_period=time_period,
rebalancing_freq=rebalancing_freq,
span=span)
# 4. given portfolio weights, construct the performance of the portfolio
funding_rate = None # on positive / negative cash balances
rebalancing_costs = 0.0010 # rebalancing costs per volume = 10bp
weight_implementation_lag = 1 # portfolio is implemented next day after weights are computed
portfolio_data = qis.backtest_model_portfolio(prices=prices.loc[weights.index[0]:, :],
weights=weights,
ticker='MaxDiversification',
funding_rate=funding_rate,
weight_implementation_lag=weight_implementation_lag,
rebalancing_costs=rebalancing_costs)
# 5. using portfolio_data run the reporting with strategy factsheet
# for group-based reporting set_group_data
portfolio_data.set_group_data(group_data=group_data, group_order=list(group_data.unique()))
# set time period for portfolio reporting
figs = qis.generate_strategy_factsheet(portfolio_data=portfolio_data,
benchmark_prices=benchmark_prices,
time_period=time_period,
**qis.fetch_default_report_kwargs(time_period=time_period))
# save report to pdf and png
qis.save_figs_to_pdf(figs=figs,
file_name=f"{portfolio_data.nav.name}_portfolio_factsheet",
orientation='landscape',
local_path="C://Users//Artur//OneDrive//analytics//outputs")
qis.save_fig(fig=figs[0], file_name=f"example_portfolio_factsheet1", local_path=f"figures/")
qis.save_fig(fig=figs[1], file_name=f"example_portfolio_factsheet2", local_path=f"figures/")
```
### 2. Customised reporting <a name="report"></a>
Portfolio data class ```PortfolioData``` is implemented in [QIS package](https://github.com/ArturSepp/QuantInvestStrats)
```python
# 6. can create customised reporting using portfolio_data custom reporting
def run_customised_reporting(portfolio_data) -> plt.Figure:
with sns.axes_style("darkgrid"):
fig, axs = plt.subplots(3, 1, figsize=(12, 12), tight_layout=True)
perf_params = qis.PerfParams(freq='W-WED', freq_reg='ME')
kwargs = dict(x_date_freq='YE', framealpha=0.8, perf_params=perf_params)
portfolio_data.plot_nav(ax=axs[0], **kwargs)
portfolio_data.plot_weights(ncol=len(prices.columns)//3,
legend_stats=qis.LegendStats.AVG_LAST,
title='Portfolio weights',
freq='QE',
ax=axs[1],
**kwargs)
portfolio_data.plot_returns_scatter(benchmark_price=benchmark_prices.iloc[:, 0],
ax=axs[2],
**kwargs)
return fig
# run customised report
fig = run_customised_reporting(portfolio_data)
# save png
qis.save_fig(fig=fig, file_name=f"example_customised_report", local_path=f"figures/")
```
### 3. Parameters sensitivity backtest <a name="sensitivity"></a>
Cross-sectional backtests are applied to test the sensitivity of
optimisation method to a parameter of estimation or solver methods.
See script in ```optimalportfolios.examples.parameter_sensitivity_backtest.py```
### 4. Multi optimisers cross backtest <a name="cross"></a>
Multiple optimisation methods can be analysed
using the wrapper function ```compute_rolling_optimal_weights()```
See example script in ```optimalportfolios.examples.multi_optimisers_backtest.py```
### 5. Optimal allocation to cryptocurrencies <a name="crypto"></a>
Computations and visualisations for
paper "Optimal Allocation to Cryptocurrencies in Diversified Portfolios" [https://ssrn.com/abstract=4217841](https://ssrn.com/abstract=4217841)
are implemented in module ```optimalportfolios.examples.crypto_allocation```,
see [README in this module](https://github.com/ArturSepp/OptimalPortfolios/blob/master/optimalportfolios/examples/crypto_allocation/README.md)
## **Updates** <a name="updates"></a>
#### 8 July 2023, Version 1.0.1 released
Implementation of optimisation methods and data considered in
"Optimal Allocation to Cryptocurrencies in
Diversified Portfolios" by A. Sepp published in Risk Magazine, October 2023, 1-6. The draft is available at SSRN: https://ssrn.com/abstract=4217841
#### 2 September 2023, Version 1.0.8 released
Added subpackage ```optimisation.rolling_engine``` with optimisers grouped by the type of inputs and
data they require.
#### 18 August 2024, Version 2.1.1 released
Refactor the implementation of solvers with the 3 layers.
Add new solvers for tracking error and target return optimisations.
Add exmples of running all solvers
## **Disclaimer** <a name="disclaimer"></a>
OptimalPortfolios package is distributed FREE & WITHOUT ANY WARRANTY under the GNU GENERAL PUBLIC LICENSE.
See the [LICENSE.txt](https://github.com/ArturSepp/OptimalPortfolios/blob/master/LICENSE.txt) in the release for details.
Please report any bugs or suggestions by opening an [issue](https://github.com/ArturSepp/OptimalPortfolios/issues).
Raw data
{
"_id": null,
"home_page": "https://github.com/ArturSepp/OptimalPortfolios",
"name": "optimalportfolios",
"maintainer": "Artur Sepp",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "artursepp@gmail.com",
"keywords": "quantitative, investing, portfolio optimization, systematic strategies, volatility",
"author": "Artur Sepp",
"author_email": "artursepp@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/63/f8/9d21b4bca21cdd0cc94c3097e515bb7140a06f8bf8ea97e75570db1ab358/optimalportfolios-2.1.8.tar.gz",
"platform": null,
"description": "## **Optimal Portfolios Backtester** <a name=\"analytics\"></a>\n\noptimalportfolios package implements analytics for backtesting of optimal portfolios including:\n1. computing of inputs (covariance matrices, returns) for roll forward\ncomputations (to avoid hindsight bias) and for generation of rolling optimal portfolios\n2. implementation of core optimisation solvers:\n 1. Minimum variance\n 2. Maximum quadratic utility\n 3. Equal risk contribution\n 4. Maximum diversification\n 5. Maximum Sharpe ratio\n 6. Maximum Cara utility under Gaussian mixture model\n 7. Tracking error minimisation \n3. the rolling backtests are compatible with incomplete time series\nfor roll forward analysis so that the portfolio universe can include instruments with different price histories\n4. computing performances of simulated portfolios\n5. reporting of backtested portfolios and cross-sectional analysis\n\n\n\nOptimalPortfolios package is split into 5 main modules with the \ndependecy path increasing sequentially as follows.\n\n1. ```optimisation``` with sub-package ```solvers``` contains implementation of\nvarious quadratic and nonlinear solvers. Each solver is implemented \nin a module independently from other solvers.\n\n2. ```reports``` is module for computing performance statistics and performance attribution including returns, volatilities, etc.\n\n3. ```examples.solvers``` provides example of running all implemented solvers.\n\n4. ```examples.crypto_allocation``` is module for computations and visualisations for \npaper \"Optimal Allocation to Cryptocurrencies in Diversified Portfolios\" [https://ssrn.com/abstract=4217841](https://ssrn.com/abstract=4217841)\n (see paper for description of the rolling-forward methodology and estimation of inputs)\n\n\n# Table of contents\n1. [Analytics](#analytics)\n2. [Installation](#installation)\n3. [Portfolio Optimisers](#optimisers)\n 1. [Implementation structure](#structure)\n 2. [Example of implementation for Maximum Diversification Solver](#example_structure)\n 3. [Constraints](#constraints)\n 4. [Wrapper for implemented rolling portfolios](#wrapper)\n 5. [Adding an optimiser](#adding)\n 6. [Default parameters](#params)\n 7. [Price time series data](#ts)\n4. [Examples](#examples)\n 1. [Optimal Portfolio Backtest](#optimal)\n 2. [Customised reporting](#report)\n 3. [Parameters sensitivity backtest](#sensitivity)\n 4. [Multi optimisers cross backtest](#cross)\n 5. [Optimal allocation to cryptocurrencies](#crypto)\n5. [Contributions](#contributions)\n6. [Updates](#updates)\n7. [Disclaimer](#disclaimer)\n\n## **Installation** <a name=\"installation\"></a>\ninstall using\n```python \npip install optimalportfolios\n```\nupgrade using\n```python \npip install --upgrade optimalportfolios\n```\n\nclose using\n```python \ngit clone https://github.com/ArturSepp/OptimalPortfolios.git\n```\n\n\nCore dependencies:\n python = \">=3.8,<3.11\",\n numba = \">=0.56.4\",\n numpy = \">=1.22.4\",\n scipy = \">=1.9.0\",\n pandas = \">=2.2.2\",\n matplotlib = \">=3.2.2\",\n seaborn = \">=0.12.2\",\n scikit_learn = \">=1.3.0\",\n cvxpy = \">=1.3.2\",\n qis = \">=2.1.1\",\n\nOptional dependencies:\n yfinance \">=0.2.3\" (for getting test price data),\n pybloqs \">=1.2.13\" (for producing html and pdf factsheets)\n\n\nTo use pybloqs for pandas > 2.x, \nlocate file \"...\\Lib\\site-packages\\pybloqs\\jinja\\table.html\" and change line 44 from:\n\n{% for col_name, cell in row.iteritems() %}\n\nto:\n\n{% for col_name, cell in row.items() %}\n\n\n\n## **Portfolio optimisers** <a name=\"optimisers\"></a>\n\n### 1. Implementation structure <a name=\"structure\"></a>\n\nThe implementation of each solver is split into the 3 layers:\n\n1) Mathematical layer which takes clean inputs, formulates the optimisation\nproblem and solves the \noptimisation problem using Scipy or CVXPY solvers.\nThe logic of this layer is to solve the problem algorithmically by taking clean inputs.\n\n2) Wrapper level which takes inputs potentially containing nans, \nfilters them out and calls the solver in layer 1). The output weights of filtered out\nassets are set to zero.\n\n3) Rolling level function with takes price time series as inputs and implements\nthe estimation of covariance matrix and other inputs on roll-forward basis. \nFor each update date the rolling layer call the wrapper layer 2) with estimated\ninputs as of the update date.\n\nThe recommended usage is as follows.\n\nLayer 2) is used for live portfolios or for backtests which are implemented using \ndata augmentation\n\nLayer 3) is applied for roll forward backtests where all available data is processed\nusing roll forward analysis\n\nFor implementation of different estimation methods for covariance matrices\nand other inputs, we recommend to implement specific layers 2) or 3) with the \nimplementation of the estimation logic. \n\nLayer 1 works with provided covariance matrices and inputs.\n\n\n### 2. Example of implementation for Maximum Diversification Solver <a name=\"example_structure\"></a>\n\nUsing example of ```optimization.solvers.max_diversification.py```\n\n1. Scipy solver ```opt_maximise_diversification()``` which takes \"clean\" inputs of the \ncovariance matrix of type ```np.ndarray``` without nans and\n```Constraints``` dataclass which implements constraints for the solver.\n\nThe lowest level of each optimisation method is ```opt_...``` or ```cvx_...``` function taking clean inputs and producing the optimal weights. \n\nThe logic of this layer is to implement pure quant logic for the optimiser with cvx solver.\n\n2. Wrapper function ```wrapper_maximise_diversification()``` which takes inputs\ncovariance matrix of type ```pd.Dataframe``` \npotentially containing nans or assets with zero variance (when their time series are missing in the \nestimation period) and filters out non-nan \"clean\" inputs and \nupdated constraints for OPT/CVX solver in layer 1.\n\nThe intermediary level of each optimisation method is ```wrapper_...``` function taking \n\"dirty\" inputs, filtering inputs, and producing the optimal weights. This wrapper can be called either \nby rolling backtest simulations or by live portfolios for rebalancing.\n\nThe logic of this layer is to filter out data and to be an interface for portfolio implementations.\n\n3. Rolling optimiser function ```rolling_maximise_diversification()``` takes the time series of data \nand sliced these accordingly and at each rebalancing step call the wrapper in layer 2.\nIn the end, the function outputs the time series of optiomal weight of assets in the universe.\nPrice data of assets may have gaps and nans which is taken care of in the wrapper level.\n\nThe backtesting of each optimisation method is implemented with ```rolling_...``` method with produces the time series of\noptimal portfolio weights.\n\nThe logic of this layer is to faciliate the backtest of portfolio optimisation method and to produce\ntime series of portfolio weights using a Markovian setup. These weights are applied for the backtest \nof the optimal portfolio and the underlying strategy\n\nEach module in ```optimization.solver``` implements specific optimisers and estimators for their inputs.\n\n\n\n### 3. Constraints <a name=\"constraints\"></a>\n\nDataclass ```Constraints``` in ```optimization.constraints``` implements \noptimisation constraints in solver independent way.\n\nThe following inputs for various constraints are implemented.\n```python \n@dataclass\nclass Constraints:\n is_long_only: bool = True # for positive allocation weights\n min_weights: pd.Series = None # instrument min weights \n max_weights: pd.Series = None # instrument max weights\n max_exposure: float = 1.0 # for long short portfolios: for long_portfolios = 1\n min_exposure: float = 1.0 # for long short portfolios: for long_portfolios = 1\n benchmark_weights: pd.Series = None # for minimisation of tracking error \n tracking_err_vol_constraint: float = None # annualised sqrt tracking error\n weights_0: pd.Series = None # for turnover constraints\n turnover_constraint: float = None # for turnover constraints\n target_return: float = None # for optimisation with target return\n asset_returns: pd.Series = None # for optimisation with target return\n max_target_portfolio_vol_an: float = None # for optimisation with maximum portfolio volatility target\n min_target_portfolio_vol_an: float = None # for optimisation with maximum portfolio volatility target\n group_lower_upper_constraints: GroupLowerUpperConstraints = None # for group allocations constraints\n```\n\nDataclass ```GroupLowerUpperConstraints``` implements asset class loading and min and max allocations\n```python \n@dataclass\nclass GroupLowerUpperConstraints:\n \"\"\"\n add constraints that each asset group is group_min_allocation <= sum group weights <= group_max_allocation\n \"\"\"\n group_loadings: pd.DataFrame # columns=instruments, index=groups, data=1 if instrument in indexed group else 0\n group_min_allocation: pd.Series # index=groups, data=group min allocation \n group_max_allocation: pd.Series # index=groups, data=group max allocation \n```\n\nConstraints are updated on the wrapper level to include the valid tickers\n```python \n def update_with_valid_tickers(self, valid_tickers: List[str]) -> Constraints:\n```\n\n\nOn the solver layer, the constants for the solvers are requested as follows.\n\nFor Scipy: ```set_scipy_constraints(self, covar: np.ndarray = None) -> List```\n\nFor CVXPY: ```set_cvx_constraints(self, w: cvx.Variable, covar: np.ndarray = None) -> List```\n\n\n\n### 4. Wrapper for implemented rolling portfolios <a name=\"wrapper\"></a>\n\nModule ```optimisation.wrapper_rolling_portfolios.py``` wraps implementation of \nof the following solvers enumerated in ```config.py```\n\nUsing the wrapper function allows for cross-sectional analysis of different\nbacktest methods and for sensitivity analysis to parameters of\nestimation and solver methods.\n\n```python\nclass PortfolioObjective(Enum):\n \"\"\"\n implemented portfolios in rolling_engine\n \"\"\"\n # risk-based:\n MAX_DIVERSIFICATION = 1 # maximum diversification measure\n EQUAL_RISK_CONTRIBUTION = 2 # implementation in risk_parity\n MIN_VARIANCE = 3 # min w^t @ covar @ w\n # return-risk based\n QUADRATIC_UTILITY = 4 # max means^t*w- 0.5*gamma*w^t*covar*w\n MAXIMUM_SHARPE_RATIO = 5 # max means^t*w / sqrt(*w^t*covar*w)\n # return-skeweness based\n MAX_CARA_MIXTURE = 6 # carra for mixture distributions\n```\n\nSee examples for [Parameters sensitivity backtest](#sensitivity) and \n[Multi optimisers cross backtest](#cross)\n\n\n### 5. Adding an optimiser <a name=\"adding\"></a>\n\n1. Add analytics for computing rolling weights using a new estimator in\nsubpackage ```optimization.solvers```. Any third-party packages can be used\n\n2. For cross-sectional analysis, add new optimiser type \nto ```config.py``` and link implemented\noptimiser in wrapper function ```compute_rolling_optimal_weights()``` in \n```optimisation.wrapper_rolling_portfolios.py```\n\n\n### 6. Default parameters <a name=\"params\"></a>\n\nKey parameters include the specification of the estimation sample.\n\n1. ```returns_freq``` defines the frequency of returns for covariance matrix estimation. This parameter affects all methods. \n\nThe default (assuming daily price data) is weekly Wednesday returns ```returns_freq = 'W-WED'```.\n\nFor price data with monthly observations \n(such us hedged funds), monthly returns should be used ```returns_freq = 'ME'```.\n\n\n2. ```span``` defines the estimation span for ewma covariance matrix. This parameter affects all methods which use \nEWMA covariance matrix:\n```\nPortfolioObjective in [MAX_DIVERSIFICATION, EQUAL_RISK_CONTRIBUTION, MIN_VARIANCE]\n``` \nand \n```\nPortfolioObjective in [QUADRATIC_UTILITY, MAXIMUM_SHARPE_RATIO]\n``` \n\nThe span is defined as the number of returns\nfor the half-life of EWMA filter: ```ewma_lambda = 1 - 2 / (span+1)```. ```span=52``` with weekly returns means that \nlast 52 weekly returns (one year of data) contribute 50% of weight to estimated covariance matrix\n\nThe default (assuming weekly returns) is 52: ```span=52```.\n\nFor monthly returns, I recommend to use ```span=12``` or ```span=24```.\n\n\n3. ```rebalancing_freq``` defines the frequency of weights update. This parameter affects all methods.\n\nThe default value is quarterly rebalancing ```rebalancing_freq='QE'```.\n\nFor the following methods \n```\nPortfolioObjective in [QUADRATIC_UTILITY, MAXIMUM_SHARPE_RATIO, MAX_CARA_MIXTURE]\n``` \nRebalancing frequency is also the rolling sample update frequency when mean returns and mixture distributions are estimated.\n\n\n4. ```roll_window``` defines the number of past returns applied for estimation of rolling mean returns and mixture distributions.\n\nThis parameter affects the following optimisers \n```\nPortfolioObjective in [QUADRATIC_UTILITY, MAXIMUM_SHARPE_RATIO, MAX_CARA_MIXTURE]\n``` \nand it is linked to ```rebalancing_freq```. \n\nDefault value is ```roll_window=20``` which means that data for past 20 (quarters) are used in the sample\nwith ```rebalancing_freq='QE'```\n\nFor monthly rebalancing, I recomend to use ```roll_window=60``` which corresponds to using past 5 years of data\n\n### 7. Price time series data <a name=\"ts\"></a>\n\nThe input to all optimisers is dataframe prices which contains dividend and split adjusted prices.\n\nThe price data can include assets with prices starting an ending at different times.\n\nAll optimisers will set maximum weight to zero for assets with missing prices in the estimation sample period. \n\n\n\n## **Examples** <a name=\"examples\"></a>\n\n### 1. Optimal Portfolio Backtest <a name=\"optimal\"></a>\n\nSee script in ```optimalportfolios.examples.optimal_portfolio_backtest.py```\n\n```python\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport yfinance as yf\nfrom typing import Tuple\nimport qis as qis\n\n# package\nfrom optimalportfolios import compute_rolling_optimal_weights, PortfolioObjective, Constraints\n\ndef fetch_universe_data() -> Tuple[pd.DataFrame, pd.DataFrame, pd.Series]:\n \"\"\"\n fetch universe data for the portfolio construction:\n 1. dividend and split adjusted end of day prices: price data may start / end at different dates\n 2. benchmark prices which is used for portfolio reporting and benchmarking\n 3. universe group data for portfolio reporting and risk attribution for large universes\n this function is using yfinance to fetch the price data\n \"\"\"\n universe_data = dict(SPY='Equities',\n QQQ='Equities',\n EEM='Equities',\n TLT='Bonds',\n IEF='Bonds',\n LQD='Credit',\n HYG='HighYield',\n GLD='Gold')\n tickers = list(universe_data.keys())\n group_data = pd.Series(universe_data)\n prices = yf.download(tickers, start=None, end=None, ignore_tz=True)['Adj Close']\n prices = prices[tickers] # arrange as given\n prices = prices.asfreq('B', method='ffill') # refill at B frequency\n benchmark_prices = prices[['SPY', 'TLT']]\n return prices, benchmark_prices, group_data\n\n\n# 2. get universe data\nprices, benchmark_prices, group_data = fetch_universe_data()\ntime_period = qis.TimePeriod('31Dec2004', '16Aug2024') # period for computing weights backtest\n\n# 3.a. define optimisation setup\nportfolio_objective = PortfolioObjective.MAX_DIVERSIFICATION # define portfolio objective\nreturns_freq = 'W-WED' # use weekly returns\nrebalancing_freq = 'QE' # weights rebalancing frequency: rebalancing is quarterly on WED\nspan = 52 # span of number of returns_freq-returns for covariance estimation = 12y\nconstraints0 = Constraints(is_long_only=True,\n min_weights=pd.Series(0.0, index=prices.columns),\n max_weights=pd.Series(0.5, index=prices.columns))\n\n# 3.b. compute solvers portfolio weights rebalanced every quarter\nweights = compute_rolling_optimal_weights(prices=prices,\n portfolio_objective=portfolio_objective,\n constraints0=constraints0,\n time_period=time_period,\n rebalancing_freq=rebalancing_freq,\n span=span)\n\n# 4. given portfolio weights, construct the performance of the portfolio\nfunding_rate = None # on positive / negative cash balances\nrebalancing_costs = 0.0010 # rebalancing costs per volume = 10bp\nweight_implementation_lag = 1 # portfolio is implemented next day after weights are computed\nportfolio_data = qis.backtest_model_portfolio(prices=prices.loc[weights.index[0]:, :],\n weights=weights,\n ticker='MaxDiversification',\n funding_rate=funding_rate,\n weight_implementation_lag=weight_implementation_lag,\n rebalancing_costs=rebalancing_costs)\n\n# 5. using portfolio_data run the reporting with strategy factsheet\n# for group-based reporting set_group_data\nportfolio_data.set_group_data(group_data=group_data, group_order=list(group_data.unique()))\n# set time period for portfolio reporting\nfigs = qis.generate_strategy_factsheet(portfolio_data=portfolio_data,\n benchmark_prices=benchmark_prices,\n time_period=time_period,\n **qis.fetch_default_report_kwargs(time_period=time_period))\n# save report to pdf and png\nqis.save_figs_to_pdf(figs=figs,\n file_name=f\"{portfolio_data.nav.name}_portfolio_factsheet\",\n orientation='landscape',\n local_path=\"C://Users//Artur//OneDrive//analytics//outputs\")\nqis.save_fig(fig=figs[0], file_name=f\"example_portfolio_factsheet1\", local_path=f\"figures/\")\nqis.save_fig(fig=figs[1], file_name=f\"example_portfolio_factsheet2\", local_path=f\"figures/\")\n```\n\n\n\n\n### 2. Customised reporting <a name=\"report\"></a>\n\nPortfolio data class ```PortfolioData``` is implemented in [QIS package](https://github.com/ArturSepp/QuantInvestStrats)\n\n```python\n# 6. can create customised reporting using portfolio_data custom reporting\ndef run_customised_reporting(portfolio_data) -> plt.Figure:\n with sns.axes_style(\"darkgrid\"):\n fig, axs = plt.subplots(3, 1, figsize=(12, 12), tight_layout=True)\n perf_params = qis.PerfParams(freq='W-WED', freq_reg='ME')\n kwargs = dict(x_date_freq='YE', framealpha=0.8, perf_params=perf_params)\n portfolio_data.plot_nav(ax=axs[0], **kwargs)\n portfolio_data.plot_weights(ncol=len(prices.columns)//3,\n legend_stats=qis.LegendStats.AVG_LAST,\n title='Portfolio weights',\n freq='QE',\n ax=axs[1],\n **kwargs)\n portfolio_data.plot_returns_scatter(benchmark_price=benchmark_prices.iloc[:, 0],\n ax=axs[2],\n **kwargs)\n return fig\n\n\n# run customised report\nfig = run_customised_reporting(portfolio_data)\n# save png\nqis.save_fig(fig=fig, file_name=f\"example_customised_report\", local_path=f\"figures/\")\n```\n\n\n\n### 3. Parameters sensitivity backtest <a name=\"sensitivity\"></a>\n\nCross-sectional backtests are applied to test the sensitivity of\noptimisation method to a parameter of estimation or solver methods.\n\nSee script in ```optimalportfolios.examples.parameter_sensitivity_backtest.py```\n\n\n\n\n\n### 4. Multi optimisers cross backtest <a name=\"cross\"></a>\n\nMultiple optimisation methods can be analysed \nusing the wrapper function ```compute_rolling_optimal_weights()``` \n\nSee example script in ```optimalportfolios.examples.multi_optimisers_backtest.py```\n\n\n\n\n### 5. Optimal allocation to cryptocurrencies <a name=\"crypto\"></a>\n\nComputations and visualisations for \npaper \"Optimal Allocation to Cryptocurrencies in Diversified Portfolios\" [https://ssrn.com/abstract=4217841](https://ssrn.com/abstract=4217841)\n are implemented in module ```optimalportfolios.examples.crypto_allocation```,\nsee [README in this module](https://github.com/ArturSepp/OptimalPortfolios/blob/master/optimalportfolios/examples/crypto_allocation/README.md)\n\n\n## **Updates** <a name=\"updates\"></a>\n\n#### 8 July 2023, Version 1.0.1 released\n\nImplementation of optimisation methods and data considered in \n\"Optimal Allocation to Cryptocurrencies in\nDiversified Portfolios\" by A. Sepp published in Risk Magazine, October 2023, 1-6. The draft is available at SSRN: https://ssrn.com/abstract=4217841\n\n\n#### 2 September 2023, Version 1.0.8 released\nAdded subpackage ```optimisation.rolling_engine``` with optimisers grouped by the type of inputs and\ndata they require.\n\n#### 18 August 2024, Version 2.1.1 released\nRefactor the implementation of solvers with the 3 layers.\n\nAdd new solvers for tracking error and target return optimisations.\n\nAdd exmples of running all solvers\n\n\n## **Disclaimer** <a name=\"disclaimer\"></a>\n\nOptimalPortfolios package is distributed FREE & WITHOUT ANY WARRANTY under the GNU GENERAL PUBLIC LICENSE.\n\nSee the [LICENSE.txt](https://github.com/ArturSepp/OptimalPortfolios/blob/master/LICENSE.txt) in the release for details.\n\nPlease report any bugs or suggestions by opening an [issue](https://github.com/ArturSepp/OptimalPortfolios/issues).\n\n\n\n\n",
"bugtrack_url": null,
"license": "LICENSE.txt",
"summary": "Simulation and backtesting of optimal portfolios",
"version": "2.1.8",
"project_urls": {
"Documentation": "https://github.com/ArturSepp/OptimalPortfolios",
"Homepage": "https://github.com/ArturSepp/OptimalPortfolios",
"Issues": "https://github.com/ArturSepp/OptimalPortfolios/issues",
"Personal website": "https://artursepp.com",
"Repository": "https://github.com/ArturSepp/OptimalPortfolios"
},
"split_keywords": [
"quantitative",
" investing",
" portfolio optimization",
" systematic strategies",
" volatility"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "63f89d21b4bca21cdd0cc94c3097e515bb7140a06f8bf8ea97e75570db1ab358",
"md5": "5d6b498081929a9fe979eef6a4ca8f9a",
"sha256": "779578d7f4ebaebd883ebc78715f1384c759e860fbea96b2b8f23e69d0e386da"
},
"downloads": -1,
"filename": "optimalportfolios-2.1.8.tar.gz",
"has_sig": false,
"md5_digest": "5d6b498081929a9fe979eef6a4ca8f9a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 72469,
"upload_time": "2024-11-02T22:41:35",
"upload_time_iso_8601": "2024-11-02T22:41:35.116810Z",
"url": "https://files.pythonhosted.org/packages/63/f8/9d21b4bca21cdd0cc94c3097e515bb7140a06f8bf8ea97e75570db1ab358/optimalportfolios-2.1.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-02 22:41:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ArturSepp",
"github_project": "OptimalPortfolios",
"github_fetch_exception": true,
"lcname": "optimalportfolios"
}
JSON
