Extract stock sentiment on financial news headlines, plot hourly/daily sentiment in a Flask web application and deploy it online
In a previous article, I explained how extract news headlines from the FinViz website for any stock before analyze data and marking his feeling using the NLTK Vader library. We also plotted the sentiment scores for different stock symbols. Feel free to refer to the article below to see how everything works in detail.
Today we will expand the article to create a dashboard web application showing the stock sentiment scores using Python Flask. The final product is shown below. Gaps in the table mean that no articles were published during these periods. Feel free to play around with the app via this link (it may take some time to load).
The code for this article is on the following GitHub repository.
Sentiment analysis of Jupyter Notebook financial news headlines is similar to the code in my previous article, but neatly packed in a few functions. Make sure to do the pip installs in the Jupyter notebook. I will give a brief summary of each function and its example of use, for a complete explanation, do not hesitate to refer to the previous article.
get news()
The get news() The function takes a ticker name, uses the Python library urllib3 to get the relevant news headlines from the FinViz website, it outputs the raw html.
ticker = 'AMZN'
news_table = get_news(ticker)
news_table # pure html code, data is not parsed yet
parse_news()
The parse_news() The function takes the raw HTML code above and uses the BeautifulSoup library to parse titles and corresponding datetimes into a Pandas DataFrame.
parsed_news_df = parse_news(news_table)
parsed_news_df.head()
score_news()
The score_news() The function takes the DataFrame from above and gives a sentiment score for each headline using the NLTK Vader library. It produces sentiment scores (negative, neutral, positive, aggregate score) as additional columns in the DataFrame.
parsed_and_scored_news = score_news(parsed_news_df)
parsed_and_scored_news.head()
plot_hourly_sentiment() and plot_daily_sentiment()
Finally, the plot_hourly_sentiment() and plot_daily_sentiment() The functions resample the respective hourly and daily sentiment scores in the above DataFrame and plot them in an interactive graph using the Plotly package.
plot_hourly_sentiment(parsed_and_scored_news, ticker)
plot_daily_sentiment(parsed_and_scored_news, ticker)
Sentiments are not always positive, for example in the last two days of writing this article, ‘TSLA’ happened to have a lot of negative headlines.
We will now create a Flask web application that
- gets user input for stock symbol
- calls the above functions in order
- displays plots and titles on a web page
Each of the titles analyzed and the corresponding sentiment scores will also be indicated on the web page in the form of a table for the user to refer to.
If you’re new to Flask, feel free to walk through the basics of creating a “Hello World” Flask app here.
Note: Don’t forget to make a flask pip setup before moving on.
First, we need to create the following directory structure and files, also listed in my GitHub repository. We will now examine each folder.
app.py
The app.py file allows us to start the Flask web application. We import the relevant packages and copy the 5 functions previously explained into the file.
The Flask application is simply created in this line.
app = Flask(__name__)
When the user enters the Flask app URL, they get road at ‘/’ by default. Therefore, the following code segment tells the Flask app to load the code into the ‘index.html’ drop under the models folder when the user accesses the default (main) page of the application.
@app.route('/')
def index():
return render_template('index.html')
index.html
We want our main page to consist of a few instructions and a text field for the user to enter the stock symbol before submitting.
let’s look at it ‘index.html’ case. We will ignore everything between the tags as it consists of CSS code to style the web page and make it look better. Without this code, the application will still work (but without the colors).
Instructions and informative text
An html file tells the browser what to display on the web page and also its formatting. For example, we see the following code segment for print the header and instructions at the top of the web page. The and are on your mind tags that make the attached text appear large and bold on the web page.
Bohmian's Stock News Sentiment Analyzer
Enter a Stock Ticker Symbol to See the Hourly and Daily Sentiment Scores of the Stock
We also print some information at the bottom of the page, as well as a small financial disclaimer. 🙂 The tags are paragraph tags that enclose tags within a paragraph, while tags and href allows us to embed links.
The sentiment is obtained by performing sentimental analysis on recent news headlines using the
NLTK Sentiment Vader Python library.
The recent news headlines are obtained from FinViz website.
Disclaimer: This page is not meant to represent any form of financial advice.
Any investments you make will carry risk so remember to do your due diligence and research before doing so.
Form to post user input
This is the most important part of the html code. We construct a form by enclosing it in the Key words. In the first line, we see action=“sentiment”this tells the browser to direct the user to ‘/feeling’ after submitting the form. The method=“POST” means that a POST request will be made to the application when the form is submitted, because the user is posting the data (i.e. the ticker entry they are interested in) to the web application for a further processing.
The creates the text field for the user to enter input. Importantly, the name=”ticker” The attribute allows the Flask application to identify this input field by name and later extract the text entered by the user.
The just creates the submit button.
What happens after the user submits an entry?
We mentioned earlier that the user will be redirected to ‘/feeling’ after the form is submitted and a POST request is made. Let us therefore return to the relevant segment of our app.py file to see how this is handled. The first line of the code below tells the application to prepare to handle a POST method request in the ‘/feeling’ itinerary.
The ticker = flask.request.form[‘ticker’].upper() tells the app to ask for the text the user entered in the field we named ‘ticker’ earlier (remember name=‘ticker’) in the html code. It converts input to uppercase text because stock symbols are usually in uppercase.
The next 5 lines of code (lines 5-9 above) apply our 5 functions in order to get and analyze news data before giving Plotly chart output as fig_hourly and fig_daily objects.
The next 2 lines then (lines 11 and 12) encode Plotly charts into objects in JSON format graphJSON_hourly and graphJSON_daily in JavaScript, which is necessary for the web page to display graphics later.
The last line tells the Flask app to render the given template in the ‘sentiment.html’ deposit and pass the graphJSON objectsthe on your mind and the description strings we generated, as well as the news_analyzed_and_rated Data frame to her. These objects will eventually be interpreted in the ‘sentiment.html’ code to display on the web page.
return render_template('sentiment.html',graphJSON_hourly=graphJSON_hourly, graphJSON_daily=graphJSON_daily, header=header,table=parsed_and_scored_news.to_html(classes='data'),description=description)
feeling.html
We want this last page to show the following:
- a on your mind: “Hourly and daily sentiment of ____ Stock”
- the hourly feelings chart
- the daily feelings chart
- some descriptive text say what the graphs show above
- detailed news headlines table and corresponding sentiment scores for the user to refer
Let’s examine the relevant parts of the code in the ‘sentiment.html’ case.
Earlier from our ‘app.py’we spent a on your mind and the description string to the model. This is accessible via Between the header and the description, we also create 2 sections of the webpage with 2 graphics, After the All that remains is to populate the We are now ready to test this Flask application. We have to go to the command line and Walk in ‘app python.py’ under the folder ‘stock_sentiment_webapp’. This will launch the running Flask app. If everything is working correctly, you should see this output. Then go to your browser and enter Now that the application is running locally, we will deploy it on the web so anyone can access it, just like my app here. We will use the Heroku Cloud Platform. Then link your GitHub accountto research your repository on GitHub and click ‘Relate’. Finally, after clicking “Deploy Branch”Herokuapp will start installing the requirements and building the app. Once everything is done, you would see it and be able to see your app on the web! Congratulations! You should be able to get something similar to my deployed application! If you liked this article, please check out my other articles. 🙂 Feel free to follow me for updates as I write more articles!{{header}} and {{description}} code below and included in header beacons and chart and chart2. We will use Javascript to populate these graphs with the respective sentiment graphs later.{{description}} code, we print the table from the DataFrame we passed earlier using the {{ table|safe }} coded. Recall that this is the parsed_and_scored_news DataFrame which includes date and time, news and their sentiment scores.chartand chart2 graphics. The last part of our code is in JavaScript, shown below. First we pass a link to a script on the Plotly page so that we can use the Plotly.plot functions. Then we pass in the Plotly.plot functions our graphJSON_hourly and graphJSON_daily objects we defined in the app.py file earlier and display them in chart and chart2 respectively.
http://127.0.0.1:5000/. Your Flask app should load and you have successfully deployed this app locally.
