Creating a Market Trading Bot Using Open AI Gym Anytrading
The major thrust of AI is the development of computer functions associated with human intelligence, such as reasoning, learning, and problem-solving, which can be particularly very helpful for the markets. Trading and investing in the market takes nothing but a series of reasoning and number crunching, based on the data and solving the problem of forecasting the future direction of the current stock prices. Manual fundamental and technical analysis is getting out of fashion nowadays.  The application of Machine learning technology in trading or stock market is being utilized so that the system automatically learns the trade complexity and improves its algorithms to attend the best trade present. In the past decade, there seemed to be a usage of the portfolio of traders, so that everyone could earn their profits. But, With the help of AI, one can perfectly analyze the underlying data points presented very fastly and accurately. Using such data points, we can analyze the current trends in the market and form patterns at high speed, which are the two necessary things generally being used for smart trading. Using the headlines in news channels and news sources, social media reviews, and comments present on other platforms, AI can analyse the stock by performing sentiment analysis on such data. Machine Learning usually stores the outcomes and the parameters that gave those outcomes and can better analyze the stock market. Data often aids in reaching a better solution, particularly in probability-based, sentiment-led activities like stock market trading. But to this day, financial engineers also believe that it’s impossible for a machine, left to its own devices, to beat the stock market. With the rise in technology, incredibly powerful computers can crunch almost countless data points in a matter of minutes. This means they are also very capable to detect historical and replicating patterns for smart market trading that are often hidden to the mere human investors. We humans simply aren’t capable of processing such data or seeing these patterns at the same rate as a technologically capable machine. AI can evaluate and analyze thousands of stocks in moments, and hence this technology adds even more speed to trading. Today every millisecond counts, and with AI as a means of automated trading, it is a wonder. AI is already learning to improve upon its own mistakes continuously. It deploys automated trading assistant bots and constantly works to improve its performance by fine-tuning programming and to input huge masses of new data. What Is Open AI Gym Anytrading? AnyTrading is an Open Source collection of OpenAI Gym environments for reinforcement learning-based trading algorithms. Trading algorithms are mostly implemented based on two of the biggest markets present: FOREX and Stock. AnyTrading aims to provide Gym environments to improve upon and facilitate the procedure of developing and testing Reinforcement Learning based algorithms in the area of Market Trading. This is obtained by implementing it on three Gym environments: TradingEnv, ForexEnv, and StocksEnv. AnyTrading can help learn about stock market trends and help with powerful analysis, providing deep insights for data-driven decisions. Getting Started With Code In this article, we will implement a Reinforcement Learning Based Market Trading Model, where we will be creating a Trading environment using OpenAI Gym AnyTrading. We will use historical GME price data, then we will train and evaluate our model using Reinforcement Learning Agents and Gym Environment. The following code is partially inspired by a video tutorial on Gym Anytrading, whose link can be found here. Installing the Library The first essential step would be to install the necessary library. To do so, you can run the following lines of code, !pip install tensorflow-gpu==1.15.0 tensorflow==1.15.0 stable-baselines gym-anytrading gym Stable-Baselines will give us the reinforcement learning algorithm and Gym Anytrading will give us our trading environment Importing the Dependencies Now let us install the required dependencies to create a base framework for our model, and we are going to use the A2C reinforcement learning algorithm to create our Market Trading Model. # Importing Dependencies import gym import gym_anytrading # Stable baselines - rl stuff from stable_baselines.common.vec_env import DummyVecEnv from stable_baselines import A2C # Processing libraries import numpy as np import pandas as pd from matplotlib import pyplot as plt Processing our Dataset Now with our pipeline setup, let’s load our GME Market data. You can download the dataset using the link here. You can also use other relevant datasets such as Bitcoin data to perform this model upon. #loading our dataset df = pd.read_csv('/content/gmedata.csv') #viewing first 5 columns df.head() #converting Date Column to DateTime Type df['Date'] = pd.to_datetime(df['Date']) df.dtypes Output : Date datetime64[ns] Open float64 High float64 Low float64 Close float64 Volume object dtype: object #setting the column as index df.set_index('Date', inplace=True) df.head() We will now pass the data and create our gym environment for our Agent to later train upon. #passing the data and creating our environment env = gym.make('stocks-v0', df=df, frame_bound=(5,100), window_size=5) Setting the window size parameter will specify how many previous prices references our trading bot will have so that it can decide upon to place a trade. Testing our Environment Now with our model setup, let’s test our basic environment and deploy our reinforcement learning agent. #running the test environment state = env.reset() while True: action = env.action_space.sample() n_state, reward, done, info = env.step(action) if done: print("info", info) break plt.figure(figsize=(15,6)) plt.cla() env.render_all() plt.show() As we can see, our RL agent has randomly bought and sold stocks. Our profit margin seems to be more than 1, so we can determine that our bot has earned us profits through the trades it made. But these were random steps taken, now let’s properly train our model to achieve better trades. Training our Environment Setting up our environment to train our reinforcement learning agent upon, #setting up our environment for training env_maker = lambda: gym.make('stocks-v0', df=df, frame_bound=(5,100), window_size=5) env = DummyVecEnv([env_maker]) #Applying the Trading RL Algorithm model = A2C('MlpLstmPolicy', env, verbose=1) #setting the learning timesteps model.learn(total_timesteps=1000) --------------------------------- | explained_variance | 0.0016 | | fps | 3 | | nupdates | 1 | | policy_entropy | 0.693 | | total_timesteps | 5 | | value_loss | 111 | --------------------------------- --------------------------------- | explained_variance | -2.6e-05 | | fps | 182 | | nupdates | 100 | | policy_entropy | 0.693 | | total_timesteps | 500 | | value_loss | 2.2e+04 | --------------------------------- --------------------------------- | explained_variance | 0.0274 | | fps | 244 | | nupdates | 200 | | policy_entropy | 0.693 | | total_timesteps | 1000 | | value_loss | 0.0663 | #Setting up the Agent Environment env = gym.make('stocks-v0', df=df, frame_bound=(90,110), window_size=5) obs = env.reset() while True: obs = obs[np.newaxis, ...] action, _states = model.predict(obs) obs, rewards, done, info = env.step(action) if done: print("info", info) break #Plotting our Model for Trained Trades plt.figure(figsize=(15,6)) plt.cla() env.render_all() plt.show() As we can see here, our trained agent is now placing much better trades and a lot less random trades, giving us profit at the same time with a lot more conscience about when to buy and when it is the right time to sell the stock. End Notes In this article, we tried to understand how artificial intelligence can be applied to market trading to help leverage the art of buying and selling. We also created a Reinforcement Learning model through which our trained agent can buy and sell stocks, booking us profit simultaneously. The following implementation above can be found as a colab notebook, which can be accessed using the link here. Happy Learning! References AnyTrading Github Repository Open AI Gym Website
AnyTrading is an Open Source collection of OpenAI Gym environments for reinforcement learning-based trading algorithms. AnyTrading aims to provide Gym environments to improve upon and facilitate the procedure of developing and testing Reinforcement Learning based algorithms in the area of Market Trading.
AnyTrading is an Open Source collection of OpenAI Gym environments for reinforcement learning-based trading algorithms.
["AI (Artificial Intelligence)", "Data Science", "Data Scientist", "Deep Learning", "Machine Learning", "OpenAI", "Python", "video processing ai"]
["NumPy", "artificial intelligence", "machine learning", "OpenAI", "AI", "TensorFlow", "ML", "Machine Learning", "video processing ai", "Python", "Colab", "Aim", "Deep Learning", "Data Science", "Data Scientist", "AI (Artificial Intelligence)", "Pandas"]
["AI", "artificial intelligence", "machine learning", "ML", "OpenAI", "Aim", "TensorFlow", "Colab", "Pandas", "NumPy"]
Victor is an aspiring Data Scientist &amp; is a Master of Science in Data Science &amp; Big Data Analytics. He is a Researcher, a Data Science Influencer and also an Ex-University Football Player. A keen learner of new developments in Data Science and Artificial Intelligence, he is committed to growing the Data Science community.