This post is the first in a two-part series on stock data analysis using Python, based on a lecture I gave on the subject for MATH Data Science at the University of Utah. In these posts, I will discuss basics such as obtaining the data from Yahoo! Finance using pandas , visualizing stock data, moving averages, developing a moving-average crossover strategy, backtesting, and benchmarking. The final post will include practice problems.

This first post discusses topics up to introducing moving averages. None of the content of this post should be considered financial advice.

Furthermore, any code written here is provided without any form of guarantee. Individuals who choose to use it do so at their own risk. Advanced mathematics and statistics has been present in finance for some time. For example one of the biggest recent achievements of mathematics was the derivation of the Black-Scholes formula , which facilitated the pricing of stock options a contract giving the holder the right to purchase or sell a stock at a particular price to the issuer of the option.

That said, bad statistical models, including the Black-Scholes formula, hold part of the blame for the financial crisis. In recent years, computer science has joined advanced mathematics in revolutionizing finance and trading , the practice of buying and selling of financial assets for the purpose of making a profit.

In recent years, trading has become dominated by computers; algorithms are responsible for making rapid split-second trading decisions faster than humans could make so rapidly, the speed at which light travels is a limitation when designing systems.

Additionally, machine learning and data mining techniques are growing in popularity in the financial sector, and likely will continue to do so. In fact, a large part of algorithmic trading is high-frequency trading HFT.

While algorithms may outperform humans, the technology is still new and playing in a famously turbulent, high-stakes arena. HFT was responsible for phenomena such as the flash crash and a flash crash prompted by a hacked Associated Press tweet about an attack on the White House. This lecture, however, will not be about how to crash the stock market with bad mathematical models or trading algorithms.

Instead, I intend to provide you with basic tools for handling and analyzing stock market data with Python. I will also discuss moving averages, how to construct trading strategies using moving averages, how to formulate exit strategies upon entering a position, and how to evaluate a strategy with backtesting. THIS IS NOT FINANCIAL ADVICE!!! Furthermore, I have ZERO experience as a trader a lot of this knowledge comes from a one-semester course on stock trading I took at Salt Lake Community College!

This is purely introductory knowledge, not enough to make a living trading stocks. People can and do lose money trading stocks, and you do so at your own risk! Before we play with stock data, we need to get it in some workable format.

Stock data can be obtained from Yahoo! Finance , Google Finance , or a number of other sources, and the pandas package provides easy access to Yahoo! Finance and Google Finance data, along with other sources. In this lecture, we will get our data from Yahoo! The following code demonstrates how to create directly a DataFrame object containing stock information.

You can read more about remote data access here. Open is the price of the stock at the beginning of the trading day it need not be the closing price of the previous trading day , high is the highest price of the stock on that trading day, low the lowest price of the stock on that trading day, and close the price of the stock at closing time. Volume indicates how many stocks were traded.

Adjusted close is the closing price of the stock that adjusts the price of the stock for corporate actions. While stock prices are considered to be set mostly by traders, stock splits when the company makes each extant stock worth two and halves the price and dividends payout of company profits per share also affect the price of a stock and should be accounted for.

Now that we have stock data we would like to visualize it. I first demonstrate how to do so using the matplotlib package. Notice that the apple DataFrame object has a convenience method, plot , which makes creating plots easier.

A linechart is fine, but there are at least four variables involved for each date open, high, low, and close , and we would like to have some visual way to see all four variables that does not require plotting four separate lines. Financial data is often plotted with a Japanese candlestick plot , so named because it was first created by 18th century Japanese rice traders. Such a chart can be created with matplotlib , though it requires considerable effort.

I have made a function you are welcome to use to more easily create candlestick charts from pandas data frames, and use it to plot our stock data. Code is based off this example , and you can read the documentation for the functions involved here.

With a candlestick chart, a black candlestick indicates a day where the closing price was higher than the open a gain , while a red candlestick indicates a day where the open was higher than the close a loss. The wicks indicate the high and the low, and the body the open and close hue is used to determine which end of the body is the open and which the close.

Candlestick charts are popular in finance and some strategies in technical analysis use them to make trading decisions, depending on the shape, color, and position of the candles. I will not cover such strategies today.

We may wish to plot multiple financial instruments together; we may want to compare stocks, compare them to the market, or look at other securities such as exchange-traded funds ETFs. Later, we will also want to see how to plot a financial instrument against some indicator, like a moving average. For this you would rather use a line chart than a candlestick chart.

How would you plot multiple candlestick charts on top of one another without cluttering the chart? While absolute price is important pricy stocks are difficult to purchase, which affects not only their volatility but your ability to trade that stock , when trading, we are more concerned about the relative change of an asset rather than its absolute price. One solution would be to use two different scales when plotting the data; one scale will be used by Apple and Microsoft stocks, and the other by Google.

This involves transforming the data into something more useful for our purposes. There are multiple transformations we could apply. In other words, we plot:. This is a much more useful plot. We can now see how profitable each stock was since the beginning of the period. Furthermore, we see that these stocks are highly correlated; they generally move in the same direction, a fact that was difficult to see in the other charts. Alternatively, we could plot the change of each stock per day.

These formulas are not the same and can lead to differing conclusions, but there is another way to model the growth of a stock: Here, is the natural log, and our definition does not depend as strongly on whether we use or. The advantage of using log differences is that this difference can be interpreted as the percentage change in a stock but does not depend on the denominator of a fraction.

Which transformation do you prefer? Looking at returns since the beginning of the period make the overall trend of the securities in question much more apparent. Changes between days, though, are what more advanced methods actually consider when modelling the behavior of a stock. Charts are very useful. Moving averages smooth a series and helps identify trends. The larger is, the less responsive a moving average process is to short-term fluctuations in the series. Fast moving averages have smaller and more closely follow the stock, while slow moving averages have larger , resulting in them responding less to the fluctuations of the stock and being more stable.

I demonstrate its use by creating a day one month moving average for the Apple data, and plotting it alongside the stock. Notice how late the rolling average begins. It cannot be computed until 20 days have passed.

This limitation becomes more severe for longer moving averages. That said, we will still largely focus on You will notice that a moving average is much smoother than the actua stock data.

Thus, crossing a moving average signals a possible change in trend, and should draw attention. Traders are usually interested in multiple moving averages, such as the day, day, and day moving averages. The day moving average is the most sensitive to local changes, and the day moving average the least. Here, the day moving average indicates an overall bearish trend: The day moving average is at times bearish and at other times bullish , where a positive swing is expected.

You can also see that the crossing of moving average lines indicate changes in trend. These crossings are what we can use as trading signals , or indications that a financial security is changing direction and a profitable trade might be made.

Python library for real-time stock and option data, any feedback welcome. : Python

Visit next week to read about how to design and test a trading strategy using moving averages. An earlier version of this article suggested that algorithmic trading was synonymous as high-frequency trading. As pointed out in the comments by dissolved, this need not be the case; algorithms can be used to identify trades without necessarily being high frequency.

While HFT is a large subset of algorithmic trading, it is not equal to it. Did you get to fix the weekend gaps in your candlestick charts? I want to remove the gaps — weekends and public holidays when the market is closed. These are not addressed in my charts. Great article, thanks for writing! There is absolutely no reason a trading algorithm has to have high turnover. You need to install pandas-datareader package https: Yes, I was aware, but for whatever reason the new code did not work in Jupyter when I tried it, so I left it as is since it still worked for now.

I find another one called candlestick there. However, the generated chart is only black in color. But how do we get the ticker symbols in the first place? If we check the market today we are introducing survivor bias in our analysis. You may need to go to the exchange of interest I. HI How will I get the prices for gold, crudeoil and other commodities. Can we get it from yahoo. Look for ETFs that track commodities.

It did take a few hours of experimenting to get those six lines working, however. Thank you very much for making my quest for writing financial strategies SO much easier. Please post more on this subject whenever you have time, or if you already have more information posted, could you leave me a link pointing me towards the site s???

Like Liked by 1 person. I REALLY want to write more for my blog and I probably would write more on financial topics, if I could think up some; requests are welcome , but I tend to be very busy these days. But thank you for your kind words. I am following the tutorial you put up for R here: I got to this part:. Anyway, there was a bug in the original code.

The post has been updated. You mat delete the comment sorry. This is a great tutorial, thank you for sharing. For anyone getting the following error: Wow, awesome code there, just had to copy it to python and try to run it.

As soon as I ran the code I got this error message: Can someone please help me out here, would definitely like to use this as example for other prediction models for stocks and commodities. The line is an IPython magic function, which does not work in vanilla Python.

Run in an IPython environment, like a Jupyter Notebook, or erase the calls to magic functions but I make no promises about how the program will function if you do.

You are commenting using your WordPress.

Yahoo Options Scraper in Python : investing

You are commenting using your Twitter account. You are commenting using your Facebook account. Notify me of new comments via email. Curtis Miller's Personal Website Curtis Miller's personal website, with resume, portfolio, blog, etc. Tags activism apple backtesting cormania debugging democrats donald trump election finance financial sector game art game design gamemaker: Posted on September 19, September 26, Economics and Finance , Python , Statistics and Data Science.

Introduction Advanced mathematics and statistics has been present in finance for some time. Getting and Visualizing Stock Data Getting Data from Yahoo! Finance with pandas Before we play with stock data, we need to get it in some workable format. DataReader "AAPL", "yahoo", start, end type apple C: After installing the pandas-datareader package https: AAPL GOOG MSFT Date AAPL GOOG MSFT Date 1. AAPL GOOG MSFT Date NaN NaN NaN Twitter Facebook Email Reddit More Print LinkedIn Google Tumblr Pinterest Pocket Telegram WhatsApp Skype.

Planet Python — Cloud Data Architect. I suppose you have a point. Visto nel Web — Ok, panico. An Introduction to Stock Market Data Analysis with Python Part 2 Curtis Miller's Personal Website. October — Data Science News. Kind regards Like Like. Where to Go from Here? Tips for Building Up R Experience Curtis Miller's Personal Website.

Tips for Building Up R Experience - Use-R! Tips for Building Up R Experience — Mubashir Qasim. I was wondering the same question… Like Like.

I would not know. An Introduction to Stock Market Data Analysis with Python — thoughts…. An Introduction to Stock Market Data Analysis with Python from Curtis Miller — Joe the Data Guy. An Introduction to Stock Market Data Analysis with R Part 1 Curtis Miller's Personal Website. An Introduction to Stock Market Data Analysis with R Part 1 — thoughts…. Hi Curtis, I am following the tutorial you put up for R here: I got to this part: Thanks for the post!

An Introduction to Stock Market Data Analysis with R Part 2 Curtis Miller's Personal Website. Hi, I have still not been able to work around this…. An Introduction to Stock Market Data Analysis with Python Learn for Master. Wed, Sep 28th, - Blendo. Leave a Reply Cancel reply Enter your comment here Fill in your details below or click an icon to log in: Email required Address never made public.

Create a free website or blog at WordPress. Send to Email Address Your Name Your Email Address document. Post was not sent - check your email addresses! Sorry, your blog cannot share posts by email.