python-pour-finance/11-Quantopian-Avancé/.ipynb_checkpoints/00-Pipeline-Example-Walkthr...

717 lines
23 KiB
Plaintext

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Pipeline Example"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from quantopian.pipeline import Pipeline\n",
"from quantopian.research import run_pipeline\n",
"from quantopian.pipeline.data.builtin import USEquityPricing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Getting the Securities we want.\n",
"\n",
"### The Q500US and Q1500US\n",
"\n",
"These gropus of tradeable stocks are refered to as \"universes\", because all your trades will use these stocks as their \"Universe\" of available stock, they won't be trading with anything outside these groups."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from quantopian.pipeline.filters import Q1500US"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are two main benefits of the Q500US and Q1500US. Firstly, they greatly reduce the risk of an order not being filled. Secondly, they allow for more meaningful comparisons between strategies as now they will be used as the standard universes for algorithms."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"universe = Q1500US()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Filtering the universe further with Classifiers\n",
"\n",
"Let's only grab stocks in the energy sector: https://www.quantopian.com/help/fundamentals#industry-sector"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from quantopian.pipeline.data import morningstar"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sector = morningstar.asset_classification.morningstar_sector_code.latest"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternative:"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"#from quantopian.pipeline.classifiers.morningstar import Sector\n",
"#morningstar_sector = Sector()"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"energy_sector = sector.eq(309)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Masking Filters\n",
"\n",
"Masks can be also be applied to methods that return filters like top, bottom, and percentile_between.\n",
"\n",
"Masks are most useful when we want to apply a filter in the earlier steps of a combined computation. For example, suppose we want to get the 50 securities with the highest open price that are also in the top 10% of dollar volume. \n",
"\n",
"Suppose that we then want the 90th-100th percentile of these securities by close price. We can do this with the following:"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from quantopian.pipeline.factors import SimpleMovingAverage, AverageDollarVolume"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Dollar volume factor\n",
"dollar_volume = AverageDollarVolume(window_length=30)\n",
"\n",
"# High dollar volume filter\n",
"high_dollar_volume = dollar_volume.percentile_between(90,100)\n",
"\n",
"# Top open price filter (high dollar volume securities)\n",
"top_open_price = USEquityPricing.open.latest.top(50, mask=high_dollar_volume)\n",
"\n",
"# Top percentile close price filter (high dollar volume, top 50 open price)\n",
"high_close_price = USEquityPricing.close.latest.percentile_between(90, 100, mask=top_open_price)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Applying Filters and Factors\n",
"\n",
"Let's apply our own filters, following along with some of the examples above. Let's select the following securities:\n",
"\n",
"* Stocks in Q1500US\n",
"* Stocks that are in the energy Sector\n",
"* They must be relatively highly traded stocks in the market (by dollar volume traded, need to be in the top 5% traded)\n",
"\n",
"Then we'll calculate the percent difference as we've done previously. Using this percent difference we'll create an unsophisticated strategy that shorts anything with negative percent difference (the difference between the 10 day mean and the 30 day mean)."
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def make_pipeline():\n",
" \n",
" # Base universe filter.\n",
" base_universe = Q1500US()\n",
" \n",
" # Sector Classifier as Filter\n",
" energy_sector = sector.eq(309)\n",
" \n",
" # Masking Base Energy Stocks\n",
" base_energy = base_universe & energy_sector\n",
" \n",
" # Dollar volume factor\n",
" dollar_volume = AverageDollarVolume(window_length=30)\n",
"\n",
" # Top half of dollar volume filter\n",
" high_dollar_volume = dollar_volume.percentile_between(95,100)\n",
" \n",
" # Final Filter Mask\n",
" top_half_base_energy = base_energy & high_dollar_volume\n",
" \n",
" # 10-day close price average.\n",
" mean_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10, mask=top_half_base_energy)\n",
"\n",
" # 30-day close price average.\n",
" mean_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30, mask=top_half_base_energy)\n",
"\n",
" # Percent difference factor.\n",
" percent_difference = (mean_10 - mean_30) / mean_30\n",
" \n",
" # Create a filter to select securities to short.\n",
" shorts = percent_difference < 0\n",
" \n",
" # Create a filter to select securities to long.\n",
" longs = percent_difference > 0\n",
" \n",
" # Filter for the securities that we want to trade.\n",
" securities_to_trade = (shorts | longs)\n",
" \n",
" return Pipeline(\n",
" columns={\n",
" 'longs': longs,\n",
" 'shorts': shorts,\n",
" 'percent_diff':percent_difference\n",
" },\n",
" screen=securities_to_trade\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th></th>\n",
" <th>longs</th>\n",
" <th>percent_diff</th>\n",
" <th>shorts</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th rowspan=\"38\" valign=\"top\">2015-05-05 00:00:00+00:00</th>\n",
" <th>Equity(216 [HES])</th>\n",
" <td>True</td>\n",
" <td>0.036528</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(448 [APA])</th>\n",
" <td>True</td>\n",
" <td>0.035926</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(455 [APC])</th>\n",
" <td>True</td>\n",
" <td>0.049153</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(858 [BHI])</th>\n",
" <td>True</td>\n",
" <td>0.033807</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(1746 [COG])</th>\n",
" <td>True</td>\n",
" <td>0.058078</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(2368 [DVN])</th>\n",
" <td>True</td>\n",
" <td>0.046264</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(2564 [EOG])</th>\n",
" <td>True</td>\n",
" <td>0.032102</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(2621 [ESV])</th>\n",
" <td>True</td>\n",
" <td>0.060197</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(3443 [HAL])</th>\n",
" <td>True</td>\n",
" <td>0.049257</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(3647 [HP])</th>\n",
" <td>True</td>\n",
" <td>0.040991</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(5035 [MRO])</th>\n",
" <td>True</td>\n",
" <td>0.061598</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(5213 [NBL])</th>\n",
" <td>True</td>\n",
" <td>0.010443</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(5214 [NBR])</th>\n",
" <td>True</td>\n",
" <td>0.064133</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(5249 [NE])</th>\n",
" <td>True</td>\n",
" <td>0.037559</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(5729 [OXY])</th>\n",
" <td>True</td>\n",
" <td>0.029776</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(6928 [SLB])</th>\n",
" <td>True</td>\n",
" <td>0.046555</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(7244 [SWN])</th>\n",
" <td>True</td>\n",
" <td>0.070788</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(7612 [ANDV])</th>\n",
" <td>True</td>\n",
" <td>0.005997</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(7990 [VLO])</th>\n",
" <td>False</td>\n",
" <td>-0.017145</td>\n",
" <td>True</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(8214 [WMB])</th>\n",
" <td>True</td>\n",
" <td>0.018876</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(8347 [XOM])</th>\n",
" <td>True</td>\n",
" <td>0.017343</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(8461 [CHK])</th>\n",
" <td>True</td>\n",
" <td>0.014265</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(9038 [RIG])</th>\n",
" <td>True</td>\n",
" <td>0.048180</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(13176 [CAM])</th>\n",
" <td>True</td>\n",
" <td>0.082110</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(17436 [PXD])</th>\n",
" <td>True</td>\n",
" <td>0.010248</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(19249 [RRC])</th>\n",
" <td>True</td>\n",
" <td>0.087062</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(19336 [WFT])</th>\n",
" <td>True</td>\n",
" <td>0.049141</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(22784 [FTI])</th>\n",
" <td>True</td>\n",
" <td>0.054529</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(23112 [CVX])</th>\n",
" <td>True</td>\n",
" <td>0.018972</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(23998 [COP])</th>\n",
" <td>True</td>\n",
" <td>0.023902</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(24809 [NOV])</th>\n",
" <td>True</td>\n",
" <td>0.024940</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(25707 [WLL])</th>\n",
" <td>True</td>\n",
" <td>0.048205</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(33856 [CLR])</th>\n",
" <td>True</td>\n",
" <td>0.064304</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(34440 [CXO])</th>\n",
" <td>True</td>\n",
" <td>0.042184</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(39797 [OAS])</th>\n",
" <td>True</td>\n",
" <td>0.042388</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(40852 [KMI])</th>\n",
" <td>True</td>\n",
" <td>0.023016</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(41636 [MPC])</th>\n",
" <td>True</td>\n",
" <td>0.011952</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Equity(42788 [PSX])</th>\n",
" <td>True</td>\n",
" <td>0.020911</td>\n",
" <td>False</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" longs percent_diff shorts\n",
"2015-05-05 00:00:00+00:00 Equity(216 [HES]) True 0.036528 False\n",
" Equity(448 [APA]) True 0.035926 False\n",
" Equity(455 [APC]) True 0.049153 False\n",
" Equity(858 [BHI]) True 0.033807 False\n",
" Equity(1746 [COG]) True 0.058078 False\n",
" Equity(2368 [DVN]) True 0.046264 False\n",
" Equity(2564 [EOG]) True 0.032102 False\n",
" Equity(2621 [ESV]) True 0.060197 False\n",
" Equity(3443 [HAL]) True 0.049257 False\n",
" Equity(3647 [HP]) True 0.040991 False\n",
" Equity(5035 [MRO]) True 0.061598 False\n",
" Equity(5213 [NBL]) True 0.010443 False\n",
" Equity(5214 [NBR]) True 0.064133 False\n",
" Equity(5249 [NE]) True 0.037559 False\n",
" Equity(5729 [OXY]) True 0.029776 False\n",
" Equity(6928 [SLB]) True 0.046555 False\n",
" Equity(7244 [SWN]) True 0.070788 False\n",
" Equity(7612 [ANDV]) True 0.005997 False\n",
" Equity(7990 [VLO]) False -0.017145 True\n",
" Equity(8214 [WMB]) True 0.018876 False\n",
" Equity(8347 [XOM]) True 0.017343 False\n",
" Equity(8461 [CHK]) True 0.014265 False\n",
" Equity(9038 [RIG]) True 0.048180 False\n",
" Equity(13176 [CAM]) True 0.082110 False\n",
" Equity(17436 [PXD]) True 0.010248 False\n",
" Equity(19249 [RRC]) True 0.087062 False\n",
" Equity(19336 [WFT]) True 0.049141 False\n",
" Equity(22784 [FTI]) True 0.054529 False\n",
" Equity(23112 [CVX]) True 0.018972 False\n",
" Equity(23998 [COP]) True 0.023902 False\n",
" Equity(24809 [NOV]) True 0.024940 False\n",
" Equity(25707 [WLL]) True 0.048205 False\n",
" Equity(33856 [CLR]) True 0.064304 False\n",
" Equity(34440 [CXO]) True 0.042184 False\n",
" Equity(39797 [OAS]) True 0.042388 False\n",
" Equity(40852 [KMI]) True 0.023016 False\n",
" Equity(41636 [MPC]) True 0.011952 False\n",
" Equity(42788 [PSX]) True 0.020911 False"
]
},
"execution_count": 74,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-05')\n",
"result"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'pandas.core.frame.DataFrame'>\n",
"MultiIndex: 38 entries, (2015-05-05 00:00:00+00:00, Equity(216 [HES])) to (2015-05-05 00:00:00+00:00, Equity(42788 [PSX]))\n",
"Data columns (total 3 columns):\n",
"longs 38 non-null bool\n",
"percent_diff 38 non-null float64\n",
"shorts 38 non-null bool\n",
"dtypes: bool(2), float64(1)\n",
"memory usage: 684.0+ bytes\n"
]
}
],
"source": [
"result.info()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Executing this Strategy in the IDE"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from quantopian.algorithm import attach_pipeline,pipeline_output\n",
"from quantopian.pipeline import Pipeline\n",
"from quantopian.pipeline.data.builtin import USEquityPricing\n",
"from quantopian.pipeline.factors import AverageDollarVolume,SimpleMovingAverage\n",
"from quantopian.pipeline.filters.morningstar import Q1500US\n",
"from quantopian.pipeline.data import morningstar\n",
"\n",
"def initialize(context):\n",
" \n",
" schedule_function(my_rebalance,date_rules.week_start(),time_rules.market_open(hours=1))\n",
" \n",
" my_pipe = make_pipeline()\n",
" attach_pipeline(my_pipe,'my_pipeline')\n",
" \n",
"def my_rebalance(context,data):\n",
" for security in context.portfolio.positions:\n",
" if security not in context.longs and security not in context.shorts and data.can_trade(security):\n",
" order_target_percent(security,0)\n",
" \n",
" for security in context.longs:\n",
" if data.can_trade(security):\n",
" order_target_percent(security,context.long_weight)\n",
"\n",
" for security in context.shorts:\n",
" if data.can_trade(security):\n",
" order_target_percent(security,context.short_weight)\n",
"\n",
"\n",
"\n",
"\n",
"def my_compute_weights(context):\n",
" \n",
" if len(context.longs)==0:\n",
" long_weight = 0\n",
" else:\n",
" long_weight = 0.5 / len(context.longs)\n",
" \n",
" if len(context.shorts)==0:\n",
" short_weight = 0\n",
" else:\n",
" short_weight = 0.5 / len(context.shorts)\n",
" \n",
" return (long_weight,short_weight)\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"def before_trading_start(context,data):\n",
" context.output = pipeline_output('my_pipeline')\n",
" \n",
" # LONG\n",
" context.longs = context.output[context.output['longs']].index.tolist()\n",
" \n",
" # SHORT\n",
" context.shorts = context.output[context.output['shorts']].index.tolist()\n",
"\n",
"\n",
" context.long_weight,context.short_weight = my_compute_weights(context)\n",
"\n",
"\n",
"\n",
"def make_pipeline():\n",
" \n",
" # Universe Q1500US\n",
" base_universe = Q1500US()\n",
" \n",
" # Energy Sector\n",
" sector = morningstar.asset_classification.morningstar_sector_code.latest\n",
" energy_sector = sector.eq(309)\n",
" \n",
" # Make Mask of 1500US and Energy\n",
" base_energy = base_universe & energy_sector\n",
" \n",
" # Dollar Volume (30 Days) Grab the Info\n",
" dollar_volume = AverageDollarVolume(window_length=30)\n",
" \n",
" # Grab the top 5% in avg dollar volume\n",
" high_dollar_volume = dollar_volume.percentile_between(95,100)\n",
" \n",
" # Combine the filters\n",
" top_five_base_energy = base_energy & high_dollar_volume\n",
" \n",
" # 10 day mean close\n",
" mean_10 = SimpleMovingAverage(inputs=[USEquityPricing.close],window_length=10,mask=top_five_base_energy)\n",
" \n",
" # 30 day mean close\n",
" mean_30 = SimpleMovingAverage(inputs=[USEquityPricing.close],window_length=30,mask=top_five_base_energy)\n",
" \n",
" # Percent Difference\n",
" percent_difference = (mean_10-mean_30)/mean_30\n",
" \n",
" # List of Shorts\n",
" shorts = percent_difference < 0\n",
" \n",
" # List of Longs\n",
" longs = percent_difference > 0\n",
" \n",
" # Final Mask/Filter for anything in shorts or longs\n",
" securities_to_trade = (shorts | longs)\n",
" \n",
" # Return Pipeline\n",
" return Pipeline(columns={\n",
" 'longs':longs,\n",
" 'shorts':shorts,\n",
" 'perc_diff':percent_difference\n",
" },screen=securities_to_trade)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}