Solving the option Forecast problem by a numerical method for the Black-Scholes Equation with Machine Learning Classification Model

The Black-Scholes equation is a backward parabolic partial differential equation used in mathematical finance. The proposed model uses initial boundary conditions for the stock of interest to simulate a more accurate options trading strategy. Since it is an ill-posed problem, we introduce the variable $\tau$ to solve the equation backward, where at any given time $t$, we have:

The Black-Scholes equation is given by:

Assuming that the function $u(s,\tau)$ satisfies the Black-Scholes equation with a volatility coefficient $\sigma$, we can rewrite the equation as:

with the initial condition:

where $f(x)$ is the payoff function, $x$ is the stock price, and at maturity time $\tau=0$. We assume the risk-free interest rate to be zero, and thus, $u(x,\tau)$ represents the price of an option at time $t$ [6].

We introduce this method to solve Black-Scholes equations with certain approximations. We aim to solve the following equation:

with the Dirichlet boundary conditions:

and initial conditions:

where:

with $t\in[0,2\tau]$, and $x\in(x_{b}(0),x_{a}(0))$. $P$ represents the partial differential operator for the Black-Scholes equation, where $x$ is the stock price, $\sigma$ is the volatility of the option, $o_{b}$ and $o_{a}$ are the bid and ask prices of the option, and $x_{b}$ and $x_{a}$ are the bid and ask prices of the stock. We assume that the bid price is always less than the ask price.

The method involves five steps to solve the Black-Scholes equations:

• •

  First, solve for the dimensionless variable, namely $x$.

• •

  Secondly, perform interpolation and extrapolation to forecast the option price.

• •

  Thirdly, apply the initial boundary conditions to the PDE: Partial Differential Equation.

• •

  Fourthly, apply regularization to solve the method numerically.

• •

  Lastly, solve the minimizing problem using finite difference and regularization [6].

Assuming $Y_{2\tau}=(x_{b}(0),x_{a}(0))\times(0,2\tau)$, the functions:

allow us to find the solution $u\in H^{2}(Y_{2\tau})$ that satisfies the following conditions:

We will perform a general convergence analysis based on a new Carleman estimate, assuming the time interval is within a short period. Let’s consider a general case with the given properties. Given a non-negative constant $L$ and let $Z_{L}$ be $(x,t)\in(0,1)\times(0,L)$. Suppose there are two non-negative constants $\gamma_{1},\gamma_{2}$ with $\gamma_{2}>\gamma_{1}$. Suppose the functions $\Gamma(x,t)\in C^{1}(\bar{Z}_{L})$ satisfy:

Then, the general version of the original problem is formulated as:

with the boundary condition:

and the initial condition:

We are trying to find a convergent solution to the general solutions [11]. Our previous initial results using neural networks (NN) were presented in the paper [6], where we tested three different methods: the quasi-reversibility method for solving ill-posed problems, and two models using multilayer fully connected NNs—classification and regression.

Classification algorithms are a subset of supervised machine learning techniques that aim to categorize input data into predefined classes or labels. These algorithms learn from labeled training data, where the input features are associated with known output labels, to build a model that can predict the class of new, unseen data. Common classification algorithms include decision trees, random forests, k-nearest neighbors (KNN), and neural networks. Each algorithm has its strengths and is chosen based on the nature of the dataset and the specific requirements of the problem. For instance, decision trees are easy to interpret and visualize, while neural networks can model complex relationships in data. The performance of classification algorithms is typically evaluated using metrics such as accuracy, precision, recall, and the F1 score. These metrics provide insights into how well the model distinguishes between different classes. Properly selecting and tuning these algorithms is crucial for developing effective predictive models in various domains, from medical diagnosis to stock market forecasting [4].

A decision tree is a classification algorithm that utilizes a tree-like model. This model includes a root node with no incoming edges, decision nodes with exactly one incoming edge and at least one outgoing edge, and leaf nodes with only incoming edges and no outgoing edges. The incoming edge represents the result of a previous decision, while the outgoing edges represent possible outcomes of current decisions. It is also possible to have branches consisting of multiple nodes. Note that the decision tree classifies data into final categories based on these decision paths [5].

The concept behind a decision tree is straightforward. By using specific features of the data, the tree splits the data at each node based on the values of these features, with each outgoing edge from a node representing a possible outcome from the decision made. While a decision tree does not have to be binary—meaning nodes can have more than two outgoing edges—we will use a binary tree in our case because we want to capture the binary movement of options as either increasing or decreasing. We will feed the tree with data derived from the QRM minimizer and market features, splitting the data into training, validation, and test sets. Subsequently, we will create trees with different combinations of features to generate outcomes for our trading strategy. Below is an illustration showing a basic binary decision tree, which we refer to as a classification tree (see Figure 1).

Data splitting is the process used to determine the decisions at each node of a decision tree. Generally, we select a feature that divides the data roughly in half to serve as the root node, which helps to keep the tree balanced. At the same time, we design the leaf nodes to have a bias towards one of the classifying classes, enabling us to make a final decision.

Hyperparameters define a model much like how the rules of a sport define that sport. One can differentiate between models through their hyperparameters, as these address questions related to model design. Such questions might include determining the maximum depth of a tree, the maximum number of nodes, or the learning rate for gradient descent. Hyperparameters are not features of the model but rather constitute its structure. They determine how the algorithm behaves during execution, making tuning these hyperparameters essential to minimize noise and enhance accuracy. The selection process can vary; one might manually adjust hyperparameters hoping to achieve better results, or use a method such as Grid Search. Grid Search involves building a model for each possible combination of hyperparameters, evaluating each model, and selecting the one with the most promising results. Another method, Random Search, generates random hyperparameter values from a data distribution for each parameter. This process is called Random Search due to the randomness introduced by not using a predefined set of values, but rather randomly sampling the values from a distribution [10].

The ensemble method utilizes groups of base learners or models to form a final decision by averaging the outputs of different learners or models. This approach is akin to voting, where the decision is based on the majority opinion. Since individual models might exhibit high bias or variance, combining models can reduce these issues by either taking random samples from the same model or using different models to train the data. This principle underlies the bagging and boosting methods, where bagging trains weaker learners in parallel and boosting trains them sequentially. We employ these methods to mitigate the potential for high bias.

Gradient descent is a first-order iterative optimization method that seeks to find the local minima of an algorithm. It is a boosting method that aims to reduce inaccuracies in our model by transforming a weak model into a stronger one by addressing its shortcomings. The concept involves observing the gradient at the current position and then taking a step in the opposite direction, which results in the steepest descent towards the minimum. This method is analogous to descending a hill along the steepest path. Additionally, there is a parameter known as the learning rate, which determines the size of each step taken during the descent. A larger learning rate might result in overshooting the minimum and ascending the opposite slope, while a smaller step ensures greater accuracy but may be inefficient and susceptible to saddle points, which can mislead the algorithm into thinking a local minimum has been reached (2. In this method, we use a loss function to gauge our progress down the hill; essentially, it represents the difference between the actual output of our model and our prediction, which we refer to as the error. We strive to minimize the loss function until it approaches zero, ideally reaching the global minimum. However, there are challenges associated with using this algorithm. It is a greedy method that focuses solely on the current location, potentially leading to a local minimum that may not represent the optimal solution [8].

Here, we aim to apply gradient descent to our decision tree to enhance accuracy. First, we introduce the mean squared error, which evaluates the quality of our splits. Since we are addressing an optimization problem—specifically, aiming for a classification to determine the labels of different classes—we will utilize the log-loss function and strive to minimize it for improved accuracy.

The idea is to use some input data along with a differentiable loss function. For classification, we employ the log-loss function, which is defined as follows:

The logarithmic loss is a metric that evaluates the prediction probability corresponding to the true label. With this loss function, we can proceed with the iterations:

1. 1.

   Initialize our model with a constant value:

   $\displaystyle F_{0}(x)=\arg\min_{\gamma}\sum_{i=1}^{n}L(y_{i},\gamma)$

   (22)

2. 2.

   Compute the pseudo-residuals for each data point:

   $\displaystyle r_{im}=-\left[\frac{\partial L(y_{i},F(x_{i}))}{\partial F(x_{i})}\right]{F(x)=F{m-1}(x)},\forall i=1,2,...,n$

   (23)

3. 3.

   Fit a regression tree $h_{m}(x)$ to the pseudo-residuals.

4. 4.

   Compute the multiplier $\gamma_{m}$ by solving the one-dimensional optimization problem:

   $\displaystyle\gamma_{m}=\arg\min_{\gamma}\sum_{i=1}^{n}L(y_{i},F_{m-1}(x_{i})+\gamma h_{m}(x_{i}))$

   (24)

5. 5.

   Update the model:

   $\displaystyle F_{m}(x)=F_{m-1}(x)+\gamma_{m}h_{m}(x)$

   (25)

The final output, $F_{m}(x)$, will serve as our new decision tree model [10]. By differentiating the log-loss function at every iteration, we can significantly improve the accuracy of our decision tree. Additionally, employing cross-validation helps mitigate overfitting in deeper trees that show higher training accuracy.

We employ a Bagging method, which selects a subset of the data to create multiple diverse training sets. These sets are then trained independently, and the majority of the predictions are averaged to obtain a more accurate estimation while reducing overfitting [9]. In this project, we intend to use the random forest algorithm, which applies the bagging method to develop numerous random, uncorrelated decision trees—a ”forest.” This process, known as ”the random subspace method,” aims to generate trees with low correlation among them. Unlike decision trees that use all provided features, random forests consider only a fraction of the features. The data split is performed based on Gini, entropy, or log-loss criteria, determined through hyperparameter tuning. Additionally, to identify important features, we calculate the importance of a feature by assessing the decrease in node impurity weighted by the probability of reaching that node:

where:

• •

  $w_{i}$ = weighted samples that reach node $i$,

• •

  $N_{i}$ = impurity of node $i$,

• •

  $left$ = left split on node $i$,

• •

  $right$ = right split on node $i$ [10].

Although random forest is known for its accuracy, it can be an extremely time-consuming process because it trains many trees simultaneously. Moreover, interpreting the results from such a large collection of decision trees can be challenging. This is why we utilize feature importance metrics to aid in interpreting the results. Finally, we choose to apply the random forest algorithm to our data because we believe that the market exhibits certain patterns. By correctly identifying and acting on these patterns, we can enhance the predictions made from the QRM. Furthermore, since our primary goal is to classify future options as either increasing or decreasing, having majority votes from different trees can reduce overfitting and bias within the short period of option price predictions.

Neural networks are a fundamental component of artificial intelligence, modeled after the human brain’s architecture. They consist of layers of interconnected nodes, or neurons, that process input data and adjust their connections to minimize prediction errors through a process known as backpropagation. This iterative learning process allows neural networks to improve their accuracy over time. Neural networks are particularly effective for complex tasks involving large amounts of data, such as image and speech recognition. Advances in deep learning, which involves neural networks with multiple hidden layers, have significantly improved their ability to model intricate patterns and relationships in data [1].

The k-nearest neighbors (KNN) algorithm is a simple, yet powerful, machine learning technique used for classification and regression. KNN classifies a new data point based on the majority class of its k-nearest neighbors, determined by a distance metric such as Euclidean distance. As a non-parametric method, KNN does not make assumptions about the underlying data distribution, making it flexible and easy to implement. However, it can be computationally intensive with large datasets due to the necessity of calculating distances for all training samples. KNN is particularly effective for small datasets with clear class boundaries and is frequently employed in pattern recognition and recommendation systems [2].

The results were generated using a robust combination of machine learning methods, integrating the Quasi-Reversibility Method (QRM) as a significant feature in our analysis. We ensured the integrity and reproducibility of our data by setting the random state to 42. This approach provides consistency across different runs of the experiment. Furthermore, we leveraged real market data sourced from Bloomberg, ensuring that our findings are grounded in actual market conditions. By incorporating diverse methodologies and high-quality data, our results offer a comprehensive and reliable analysis of options pricing and market prediction.

We define the option price of a certain day to be the average of the bid and the asking price. All the results below were calculated based on the following matrix:

• •

  Predicted label of Up represents that we predicted the option price would increase the next day.

• •

  Predicted label of Down represents that we predicted the option price would decrease the next day.

• •

  True label of Up represents that the real option price increases the next day.

• •

  True label of Down represents that the real option price decreases the next day.

Since the tree structure is too complex for demonstration purposes, we will only show a partial tree representing the first few layers 4.

Since the tree structure is too complex for demonstration purposes, we will only show a partial tree representing the first few layers 6.

Since the tree structure is too complex for demonstration purposes, we will only show a partial tree representing the first few layers 8.

Since the KNN boundary is too complex for demonstration purposes, we will only show a partial boundary representing the first few boundaries 10.

Since the Neural Network full layer is too complex for demonstration purposes, we will only show partial neuron graphs 12.

The data used for this project were sourced from Bloomberg.com [7]. The dataset comprises stock prices, implied volatility, option prices, QRM estimators, and estimator errors across five different days for 23,548 unique stocks. This extensive dataset was selected to ensure a comprehensive representation of the entire stock market, avoiding biases toward any specific sector. We believe this diverse dataset provides a robust foundation for achieving more reliable and promising results. Additionally, the chosen features are instrumental in predicting future movements in options, which are subsequently utilized as inputs for our classification models.