In the realm of algorithm analysis, it is easy to become preoccupied with testing on actual datasets where variables remain unknown. However, to assess algorithms for speed and accuracy, one can generate known synthetic data and evaluate how closely the algorithms approximate the actual values and how quickly the calculations are performed. This approach serves as a litmus test for algorithms before implementation in production, and also aids in identifying syntax errors or issues in the underlying mathematical models.<\/p>\n\n\n\n
The synthetic data for this experiment is generated using the Ornstein-Uhlenbeck (OU) process, a mean-reverting stochastic process frequently employed in finance and other fields. The following code demonstrates how to generate synthetic data using the OU process:<\/p>\n\n\n\n
import numpy as np\n\nnp.random.seed(42)\n\n# Synthetic data generation\nmu_true = 109\ntheta_true = 22\nsigma_true = 23\ndt = 0.01\nN = 10000\n\nX = np.zeros(N)\nX[0] = mu_true\nnoise = np.random.normal(0, sigma_true * np.sqrt(dt), N - 1)\n\nfor i in range(1, N):\n X[i] = X[i - 1] * np.exp(-theta_true * dt) + mu_true * (1 - np.exp(-theta_true * dt)) + noise[i - 1]\n<\/code><\/pre>\n\n\n\n\n- Evaluating Optimization Algorithms:<\/li>\n<\/ol>\n\n\n\n
In this experiment, we will compare the performance of different optimization algorithms on the synthetic dataset generated above. The algorithms under evaluation include:<\/p>\n\n\n\n
\n- Maximum Likelihood Estimation (MLE)<\/li>\n\n\n\n
- Least Squares Estimation (LSE)<\/li>\n\n\n\n
- Bayesian Estimation<\/li>\n\n\n\n
- Method of Moments (MOM)<\/li>\n\n\n\n
- Kalman Filter Estimation<\/li>\n<\/ul>\n\n\n\n
\n- Defining the Objective Function:<\/li>\n<\/ol>\n\n\n\n
An objective function is required to evaluate the accuracy of the optimization algorithms. In this case, we use the negative log-likelihood function, which is commonly employed for maximum likelihood estimation (MLE).<\/p>\n\n\n\n
\n- Implementing the Experiment:<\/li>\n<\/ol>\n\n\n\n
To compare the accuracy and speed of the optimization algorithms, we will run each algorithm on the synthetic dataset and measure their performance. We will use the scipy.optimize.minimize<\/code> function from the SciPy library to implement the algorithms.<\/p>\n\n\n\n\n- Analyzing the Results:<\/li>\n<\/ol>\n\n\n\n
We will analyze the results by comparing the estimated parameters and the true parameters of the synthetic data. We will also compare the speed of each algorithm by measuring the time taken to converge to the optimal solution.<\/p>\n\n\n\n
\n- Key Insights and Recommendations:<\/li>\n<\/ol>\n\n\n\n
Based on the comparison, we will discuss the key insights and recommendations for selecting the most suitable optimization algorithm for a particular problem.<\/p>\n\n\n\n
Conclusion:<\/p>\n\n\n\n
By using synthetic data, we can effectively evaluate the performance of different optimization algorithms in terms of accuracy and speed. This analysis can help data scientists and researchers make informed decisions when choosing an optimization algorithm for their specific problem. Furthermore, synthetic data can be a valuable tool for improving existing algorithms or developing new ones.<\/p>\n\n\n\n
Here is my complete code with examples.<\/p>\n\n\n\n
This code is not meant to be conclusive but rather to show an example. For instance, many of these algorithms need parameter adjustments to be more accurate or beneficial in any way. <\/p>\n\n\n\n
import os\nimport sys\nimport pandas as pd\nimport numpy as np\nfrom scipy.stats import norm\nimport matplotlib.pyplot as plt\nfrom datetime import datetime, timedelta\nimport plotly.graph_objs as go\nimport plotly.offline as pyo\nfrom plotly.subplots import make_subplots\nimport pickle\nfrom tqdm import tqdm\nfrom typing import Union, List\nfrom utils import filtered_df\nfrom scipy.optimize import minimize, fmin\nfrom statsmodels.base.model import LikelihoodModel, GenericLikelihoodModel\n\n\nparent_dir = os.path.abspath(os.path.join(os.getcwd(), '..'))\ndata_dir = os.path.join(parent_dir, 'data\/')\nsys.path.append(data_dir)\n\nfile_path = os.path.join(data_dir, 'shortable_active_adj_close.pickle')\ndf = pd.read_pickle(file_path)\n\ndef absolute_relative_difference_errors(params, mu, observations, dt):\n theta, sigma = params # Remove mu from here\n y_true, y_pred = np.array(observations), np.zeros(len(observations))\n y_pred[0] = y_true[0]\n\n random_noise = np.random.normal(0, np.sqrt(dt), size=len(observations) - 1)\n y_pred[1:] = y_true[:-1] + theta * (mu - y_true[:-1]) * dt + sigma * random_noise\n\n abs_diff_y_true = np.abs(np.diff(y_true, prepend=y_true[0]))\n abs_y_error = np.abs(y_true - y_pred)\n\n # Safely divide, avoiding division by zero\n relative_difference_errors = np.divide(abs_y_error, abs_diff_y_true, where=(abs_diff_y_true != 0))\n return np.sum(relative_difference_errors)\n\n\ndef optimize_mean_absolute_relative_difference_error(params, observations, dt):\n theta, mu, sigma = params\n n = len(observations)\n ou_predictions = np.zeros(n)\n ou_predictions[0] = observations[0]\n\n random_noise = np.random.normal(0, np.sqrt(dt), size=n - 1)\n ou_predictions[1:] = observations[:-1] + theta * (mu - observations[:-1]) * dt + sigma * random_noise\n\n errors = absolute_relative_difference_errors(observations, ou_predictions)\n return np.sum(errors)\n\n\n# Update optimization functions to pass mu as an additional argument\ndef optimize_nelder_mead(observations, dt, theta_init, mu, sigma_init):\n initial_guess = [theta_init, sigma_init] # Remove mu from here\n result = minimize(absolute_relative_difference_errors, initial_guess, args=(mu, observations, dt), method='Nelder-Mead')\n return result.x\n\ndef optimize_bfgs(observations, dt, theta_init, mu, sigma_init):\n initial_guess = [theta_init, sigma_init] # Remove mu from here\n result = minimize(absolute_relative_difference_errors, initial_guess, args=(mu, observations, dt), method='BFGS')\n return result.x\n\ndef optimize_powell(observations, dt, theta_init, mu, sigma_init):\n initial_guess = [theta_init, sigma_init] # Remove mu from here\n result = minimize(absolute_relative_difference_errors, initial_guess, args=(mu, observations, dt), method='Powell')\n return result.x\n\ndef optimize_lbfgsb(observations, dt, theta_init, mu, sigma_init):\n initial_guess = [theta_init, sigma_init] # Remove mu from here\n result = minimize(absolute_relative_difference_errors, initial_guess, args=(mu, observations, dt), method='L-BFGS-B')\n return result.x\n\ndef optimize_tnc(observations, dt, theta_init, mu, sigma_init):\n initial_guess = [theta_init, sigma_init] # Remove mu from here\n result = minimize(absolute_relative_difference_errors, initial_guess, args=(mu, observations, dt), method='TNC')\n return result.x\n\ndef mean_squared_error(y_true, y_pred):\n return np.mean((y_true - y_pred) ** 2)\n\ndef mean_absolute_error(y_true, y_pred):\n return np.mean(np.abs(y_true - y_pred))\n\ndef mean_absolute_percentage_error(y_true, y_pred):\n y_true, y_pred = np.array(y_true), np.array(y_pred)\n return np.mean(np.abs((y_true - y_pred) \/ y_true)) * 100\n\ndef mean_absolute_relative_difference_error(y_true: Union[List[float], np.ndarray], y_pred: Union[List[float], np.ndarray]) -> float:\n \"\"\"\n Calculate the mean absolute relative difference error for the given true and predicted values.\n\n :param y_true: List or numpy array of true values\n :param y_pred: List or numpy array of predicted values\n :return: Mean absolute volatility error (in percentage)\n \"\"\"\n # Convert input lists to numpy arrays\n y_true, y_pred = np.array(y_true), np.array(y_pred)\n\n # Check if there are less than two elements in y_true\n if len(y_true) < 2:\n return np.nan\n\n # Calculate the absolute differences between consecutive elements in y_true\n if len(y_true) == 2:\n abs_diff_y_true = np.abs(y_true[1] - y_true[0])\n else:\n abs_diff_y_true = np.abs(np.diff(y_true))\n # Insert the first difference value at the beginning of the array\n abs_diff_y_true = np.insert(abs_diff_y_true, 0, abs_diff_y_true[0])\n\n # Replace any zero values with a small value (epsilon) to prevent division by zero\n abs_diff_y_true = np.where(abs_diff_y_true == 0, np.finfo(float).eps, abs_diff_y_true)\n\n # Calculate the absolute differences between y_true and y_predict\n abs_y_error = np.abs(y_true - y_pred)\n\n # Calculate mean absolute volatility error (in percentage)\n return np.mean(abs_y_error \/ abs_diff_y_true)\n\ndef compute_direction_accuracy(y_true, y_pred):\n direction_actual = np.sign(np.array(y_true[1:]) - np.array(y_true[:-1]))\n direction_predicted = np.sign(np.array(y_pred[1:]) - np.array(y_pred[:-1]))\n return np.mean(direction_actual == direction_predicted)\n\n\nimport numpy as np\nimport pandas as pd\nfrom scipy.optimize import minimize\nfrom sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score\nimport plotly.graph_objs as go\nfrom plotly.subplots import make_subplots\nimport plotly.offline as pyo\nimport os\n\ndef plot_best_method_chart(symbol, method_name, metrics, data, predictions, rolling_mse, rolling_rmse, rolling_mae, rolling_marde, rolling_mape, rolling_r2, rolling_dir, best_theta, best_mu, best_sigma):\n fig = make_subplots(rows=4, cols=1, shared_xaxes=True,\n subplot_titles=(\"Original and Predicted Values\", \"Difference between Actual and Predicted\",\n \"Metrics\", \"R2\"),\n vertical_spacing=0.025,\n row_heights=[0.5, 0.1, 0.2, 0.2],\n )\n\n fig.add_trace(go.Scatter(x=data.index[252:], y=data[252:], mode=\"lines\", name=\"Original\"), row=1, col=1)\n fig.add_trace(go.Scatter(x=predictions.index[252:], y=predictions[252:], mode=\"markers\", name=\"Predicted\"),\n row=1, col=1)\n\n differences = data[252:] - predictions[252:]\n fig.add_trace(go.Scatter(x=differences.index, y=differences, mode=\"lines\", name=\"Difference\"), row=2, col=1)\n\n # Add metric subplots\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_mse, mode=\"lines\", name=\"MSE\"), row=3,\n col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_rmse, mode=\"lines\", name=\"RMSE\"), row=3,\n col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_mae, mode=\"lines\", name=\"MAE\"), row=3,\n col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_marde, mode=\"lines\", name=\"MARDE\"), row=3,\n col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_mape, mode=\"lines\", name=\"MAPE\"), row=3,\n col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_dir, mode=\"lines\", name=\"Direction Accuracy\"), row=3, col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_r2, mode=\"lines\", name=\"R2\"), row=4, col=1)\n\n\n fig.add_annotation(xref=\"paper\", yref=\"paper\", xanchor='left', yanchor='top', x=0.0, y=1,\n text=f\"<b>{method_name} Metrics<\/b><br>\"\n f\"MSE: {metrics[method_name]['MSE']:.4f}<br>\"\n f\"RMSE: {metrics[method_name]['RMSE']:.4f}<br>\"\n f\"MAE: {metrics[method_name]['MAE']:.4f}<br>\"\n f\"MARDE: {metrics[method_name]['MARDE']:.4f}<br>\"\n f\"MAPE: {metrics[method_name]['MAPE']:.4f}<br>\"\n f\"R-squared: {metrics[method_name]['R2']:.4f}<br>\"\n f\"Direction Accuracy: {metrics[method_name]['Direction Accuracy']:.4f}<br>\"\n f\"Best theta: {best_theta:.4f}<br>\"\n f\"Best mu: {best_mu:.4f}<br>\"\n f\"Best sigma: {best_sigma:.4f}\",\n font=dict(size=12), align=\"center\", showarrow=False)\n\n fig.update_layout(\n title=f\"{symbol} and MARDE: {metrics[method_name]['MARDE']:.4f}\",\n title_x=0.5,\n title_y=0.95,\n title_xanchor=\"center\",\n title_yanchor=\"top\",\n )\n\n if not os.path.exists(\"plots\"):\n os.makedirs(\"plots\")\n\n plot_file = f\"plots\/{symbol}_{method_name}_marde{metrics[method_name]['MARDE']:.4f}_theta{best_theta:.4f}_mu{best_mu:.4f}_sigma{best_sigma:.4f}.html\"\n pyo.plot(fig, filename=plot_file, auto_open=False)\n\n print(f\"Plot saved as {plot_file}\")\n\ndef save_analysis_to_dataframe(symbol, method_name, metrics, theta, mu, sigma, filename=\"ornstein_uhlenbeck_pairs_analysis\"):\n # Create a DataFrame with the specified index and columns\n columns = [\"Theta\", \"Mu\", \"Sigma\", \"Best Method\", \"MSE\", \"RMSE\", \"MAE\", \"MAPE\", \"MARDE\", \"R2\"]\n analysis_df = pd.DataFrame(index=[symbol], columns=columns)\n\n # Add the data to the DataFrame\n analysis_df.loc[symbol] = [theta, mu, sigma, method_name, metrics[method_name]['MSE'],\n metrics[method_name]['RMSE'], metrics[method_name]['MAE'],\n metrics[method_name]['MARDE'], metrics[method_name]['MARDE'],\n metrics[method_name]['R2'], metrics[method_name]['Direction Accuracy']]\n\n # Save the DataFrame to disk using the pickle library\n with open(filename + \".pkl\", \"wb\") as file:\n pickle.dump(analysis_df, file)\n\n print(f\"Analysis saved as {filename}.pkl\")\n\ndef ornstein_uhlenbeck(filename='cointegrated_etf_pairs_historical_ratios.pkl', make_stationary=False, plot_all_charts=True, output_file=\"ornstein_uhlenbeck_pairs_analysis.pkl\"):\n # Load the DataFrame from the specified pickle file\n file_path = os.path.join(data_dir, filename)\n with open(file_path, 'rb') as file:\n df = pickle.load(file)\n\n # Create a DataFrame to store the analysis for all symbols\n columns = [\"Theta\", \"Mu\", \"Sigma\", \"Best Method\", \"MSE\", \"RMSE\", \"MAE\", \"MARDE\", \"MAPE\", \"R2\", \"Direction Accuracy\"]\n all_symbols_analysis = pd.DataFrame(columns=columns)\n\n # Iterate through each column in the DataFrame\n for symbol in tqdm(df.columns, desc=\"Processing symbols\", leave=False):\n # Select the symbol column and drop NaNs\n data = df[symbol].dropna()\n original_first_value = data.iloc[0] # Save the first value before differencing\n\n # Apply first-differencing if make_stationary is True\n if make_stationary:\n data = data.diff().dropna()\n\n # Calculate dt\n index_diff = data.index[-1] - data.index[-2]\n dt = index_diff.days + index_diff.seconds \/ (60*60*24)\n # dt = dt\/365 # for calendar\n dt = dt\/252 # for trading days\n\n # Create a series to store predictions\n predictions = pd.Series(index=data.index, dtype='float64')\n\n best_method = None\n best_params = None\n best_rmse = float('inf')\n\n optimization_methods = [\n ('Nelder-Mead', optimize_nelder_mead),\n ('BFGS', optimize_bfgs),\n ('L-BFGS-B', optimize_lbfgsb),\n ('Powell', optimize_powell),\n ('TNC', optimize_tnc),\n ]\n metrics = {}\n best_parameters = {}\n rolling_metrics = {}\n method_predictions = {}\n\n\n for method_name, optimization_method in tqdm(optimization_methods, desc=\"Optimizing parameters\", leave=False):\n all_predictions = []\n all_actuals = []\n\n theta_init = 2\n sigma_init = 2.9\n\n last_theta = None\n last_mu = None\n last_sigma = None\n\n rolling_mse = []\n rolling_rmse = []\n rolling_mae = []\n rolling_mape = []\n rolling_marde = []\n rolling_r2 = []\n rolling_dir_acc = []\n rolling_mu = []\n\n for i in range(252, len(data) - 1):\n if last_theta is None:\n # First time running this optimization method, use initial values\n theta_init, mu_init, sigma_init = theta_init, data[:i].mean(), sigma_init\n else:\n # Use last calculated values as initial guess\n # theta_init, mu_init, sigma_init = last_theta, last_mu, last_sigma\n #this method does not allow mu to be optimized\n theta_init, mu_init, sigma_init = last_theta, data[:i].mean(), last_sigma\n\n # Run optimization\n theta, sigma = optimization_method(data[i - 252:i].values, dt, theta_init, mu_init,\n sigma_init) # Unpack two values\n mu = data[i-252:i].mean() # Calculate mu as the mean of the data up to the current index\n rolling_mu.append(mu)\n # Save the last calculated values\n last_theta, last_mu, last_sigma = theta, mu, sigma\n\n # print(\"All actuals:\", all_actuals)\n # print(\"All predictions:\", all_predictions)\n # Calculate prediction and save it to the predictions series\n ou_prediction = data[i] + theta * (mu - data[i]) * dt + sigma * np.random.normal(0, np.sqrt(dt))\n predictions[i + 1] = ou_prediction\n\n # Save actual and predicted values for computing metrics later\n all_predictions.append(ou_prediction)\n all_actuals.append(data[i + 1])\n\n # print(\"All actuals:\", all_actuals)\n # print(\"All predictions:\", all_predictions)\n\n # Calculate the rolling metrics\n mse = mean_squared_error(np.array(all_actuals), np.array(all_predictions))\n rmse = np.sqrt(mse)\n mae = mean_absolute_error(np.array(all_actuals), np.array(all_predictions))\n marde = mean_absolute_relative_difference_error(np.array(all_actuals), np.array(all_predictions))\n mape = mean_absolute_percentage_error(np.array(all_actuals), np.array(all_predictions))\n r2 = r2_score(np.array(all_actuals), np.array(all_predictions))\n dir_acc = compute_direction_accuracy(np.array(data[i-252:i]), np.array(predictions[i-252:i]))\n\n # Append the rolling metrics to their respective lists\n rolling_mse.append(mse)\n rolling_rmse.append(rmse)\n rolling_mae.append(mae)\n rolling_marde.append(marde)\n rolling_mape.append(mape)\n rolling_r2.append(r2)\n rolling_dir_acc.append(dir_acc)\n\n rolling_metrics[method_name] = {\n 'rolling_mse': rolling_mse,\n 'rolling_rmse': rolling_rmse,\n 'rolling_mae': rolling_mae,\n 'rolling_marde': rolling_marde,\n 'rolling_mape': rolling_mape,\n 'rolling_r2': rolling_r2,\n 'rolling_direction_accuracy': rolling_dir_acc,\n\n\n }\n\n method_predictions[method_name] = predictions.copy()\n\n # Compute metrics for this optimization method\n if len(all_actuals) > 0 and len(all_predictions) > 0:\n mse = mean_squared_error(np.array(all_actuals), np.array(all_predictions))\n rmse = np.sqrt(mse)\n mae = mean_absolute_error(np.array(all_actuals), np.array(all_predictions))\n marde = mean_absolute_relative_difference_error(np.array(all_actuals), np.array(all_predictions))\n mape = mean_absolute_percentage_error(np.array(all_actuals), np.array(all_predictions))\n r2 = r2_score(np.array(all_actuals), np.array(all_predictions))\n direction_accuracy = compute_direction_accuracy(np.array(all_actuals), np.array(all_predictions))\n\n metrics[method_name] = {'MSE': mse, 'RMSE': rmse, 'MAE': mae, 'MARDE': marde, 'MAPE': mape, 'R2': r2,\n 'Direction Accuracy': direction_accuracy}\n\n best_parameters[method_name] = {'theta': last_theta, 'mu': last_mu, 'sigma': last_sigma}\n\n print(f\"Optimization method: {method_name}\")\n print(f\"Mean Squared Error: {mse}\")\n print(f\"Root Mean Squared Error: {rmse}\")\n print(f\"Mean Absolute Error: {mae}\")\n print(f\"Mean Absolute Volatility Error: {marde}%\\n\")\n print(f\"Mean Absolute Percentage Error: {mape}%\\n\")\n print(f\"R-squared: {r2}\\n\")\n print(f\"Direction Accuracy: {direction_accuracy}\\n\")\n\n\n\n if plot_all_charts:\n # Add the plotting code here\n fig = make_subplots(rows=3, cols=1, shared_xaxes=True,\n subplot_titles=(\n \"Original and Predicted Values\", \"Difference between Actual and Predicted\",\n \"Metrics\"),\n vertical_spacing=0.05,\n row_heights=[0.5, 0.5, 0.5],\n )\n\n fig.add_trace(go.Scatter(x=data.index[252:], y=data[252:], mode=\"lines\", name=\"Original\"), row=1,\n col=1)\n fig.add_trace(\n go.Scatter(x=predictions.index[252:], y=predictions[252:], mode=\"markers\", name=\"Predicted\"),\n row=1, col=1)\n\n fig.add_trace(\n go.Scatter(x=data.index[252:], y=rolling_mu, mode=\"markers\", name=\"Mu\"),\n row=1, col=1)\n\n differences = data[252:] - predictions[252:]\n fig.add_trace(go.Scatter(x=differences.index, y=differences, mode=\"lines\", name=\"Difference\"),\n row=2, col=1)\n\n # Add metric subplots\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_mse, mode=\"lines\", name=\"MSE\"),\n row=3, col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_rmse, mode=\"lines\", name=\"RMSE\"),\n row=3, col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_mae, mode=\"lines\", name=\"MAE\"),\n row=3,col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_marde, mode=\"lines\", name=\"MARDE\"),\n row=3, col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_mape, mode=\"lines\", name=\"MAPE\"),\n row=3, col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_r2, mode=\"lines\", name=\"R2\"),\n row=3, col=1)\n fig.add_trace(go.Scatter(x=data.index[252:len(data) - 1], y=rolling_dir_acc, mode=\"lines\", name=\"Direction Accuracy\"),\n row=3, col=1)\n\n fig.add_annotation(xref=\"paper\", yref=\"paper\", xanchor='left', yanchor='top', x=0.0, y=1,\n text=f\"<b>{method_name} Metrics<\/b><br>\"\n f\"MSE: {metrics[method_name]['MSE']:.4f}<br>\"\n f\"RMSE: {metrics[method_name]['RMSE']:.4f}<br>\"\n f\"MAE: {metrics[method_name]['MAE']:.4f}<br>\"\n f\"MARDE: {metrics[method_name]['MARDE']:.4f}<br>\"\n f\"MAPE: {metrics[method_name]['MAPE']:.4f}<br>\"\n f\"R-squared: {metrics[method_name]['R2']:.4f}<br>\"\n f\"Direction Accuracy: {metrics[method_name]['Direction Accuracy']:.4f}\",\n font=dict(size=12), align=\"center\", showarrow=False)\n\n if not os.path.exists(\"plots\"):\n os.makedirs(\"plots\")\n\n plot_file = f\"plots\/{symbol}_ornstein_uhlenbeck_{method_name}.html\"\n pyo.plot(fig, filename=plot_file, auto_open=False)\n\n print(f\"Plot saved as {plot_file}\")\n else:\n print(\"Skipping metric computation due to insufficient data.\")\n\n if metrics:\n best_method = min(metrics, key=lambda x: metrics[x]['MAPE'])\n else:\n print(\"No metrics available for this symbol.\")\n continue\n # best_method = max(metrics, key=lambda x: metrics[x]['R2'])\n\n best_mse = metrics[best_method]['MSE']\n best_rmse = metrics[best_method]['RMSE']\n best_mae = metrics[best_method]['MAE']\n best_marde = metrics[best_method]['MARDE']\n best_mape = metrics[best_method]['MAPE']\n best_r2 = metrics[best_method]['R2']\n best_dir_acc = metrics[best_method]['Direction Accuracy']\n\n best_theta = best_parameters[best_method]['theta']\n best_mu = best_parameters[best_method]['mu']\n best_sigma = best_parameters[best_method]['sigma']\n\n print(f\"Best optimization method: {best_method}\")\n print(f\"Best values for theta: {best_theta}, mu: {best_mu}, sigma: {best_sigma}\")\n print(f\"Mean Squared Error: {best_mse}\")\n print(f\"Root Mean Squared Error: {best_rmse}\")\n print(f\"Mean Absolute Error: {best_mae}\")\n print(f\"Mean Absolute Relative Distance Error: {best_marde}\")\n print(f\"Mean Absolute Percentage Error: {best_mape}\")\n print(f\"R-squared: {best_r2}\")\n print(f\"Direction Accuracy: {best_dir_acc}\")\n\n all_symbols_analysis.loc[symbol] = [best_theta, best_mu, best_sigma, best_method,\n metrics[best_method]['MSE'],\n metrics[best_method]['RMSE'], metrics[best_method]['MAE'],\n metrics[best_method]['MARDE'], metrics[best_method]['MAPE'],\n metrics[best_method]['R2'],\n metrics[best_method]['Direction Accuracy']]\n\n # Call the plot_best_method_chart function after determining the best_method\n plot_best_method_chart(\n symbol, best_method, metrics, data, method_predictions[best_method],\n rolling_metrics[best_method]['rolling_mse'],\n rolling_metrics[best_method]['rolling_rmse'],\n rolling_metrics[best_method]['rolling_mae'],\n rolling_metrics[best_method]['rolling_marde'],\n rolling_metrics[best_method]['rolling_mape'],\n rolling_metrics[best_method]['rolling_r2'],\n rolling_metrics[best_method]['rolling_direction_accuracy'],\n best_theta, best_mu, best_sigma\n )\n\n with open(output_file, \"wb\") as file:\n pickle.dump(all_symbols_analysis, file)\n # return best_method, best_theta, best_mu, best_sigma, best_mse, best_rmse, best_mae, best_marde, best_mape,best_r2\n\n\n\nornstein_uhlenbeck()\n\n<\/code><\/pre>\n\n\n\nYour final results will look something like this.
True values: \u03bc = 109.00, \u03b8 = 22.00, \u03c3 = 23.00
MLE estimates: \u03bc = 923764.61, \u03b8 = -4591680.43, \u03c3 = 69817.80
Least Squares: \u03bc = 108.97, \u03b8 = 23.72, \u03c3 = 1.00
Method of Moments: \u03bc = 108.98, \u03b8 = 23.70, \u03c3 = 19.33
Kalman Filter: \u03bc = 0.00, \u03b8 = 2.00, \u03c3 = 2.42
Ensemble prediction: \u03bc = 108.98, \u03b8 = 23.70, \u03c3 = 19.33
MLE estimates (Nelder-Mead): \u03bc = 0.00, \u03b8 = -3255.78, \u03c3 = 18095.76
MLE estimates (Powell): \u03bc = -0.51, \u03b8 = -104.01, \u03c3 = 16.22
MLE estimates (CG): \u03bc = -31431.14, \u03b8 = -159.34, \u03c3 = 8154.10
MLE estimates (BFGS): \u03bc = -59278.98, \u03b8 = -1326.24, \u03c3 = 19548.41
MLE estimates (L-BFGS-B): \u03bc = 0.77, \u03b8 = 4.98, \u03c3 = 5.30<\/strong><\/p>\n\n\n\nProcess finished with exit code 0<\/p>\n","protected":false},"excerpt":{"rendered":"
In the realm of algorithm analysis, it is easy to become preoccupied with testing on actual datasets where variables remain unknown. However, to assess algorithms for speed and…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-4149","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\n
Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed - Jeremy Whittaker<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed - Jeremy Whittaker\" \/>\n<meta property=\"og:description\" content=\"In the realm of algorithm analysis, it is easy to become preoccupied with testing on actual datasets where variables remain unknown. However, to assess algorithms for speed and...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/\" \/>\n<meta property=\"og:site_name\" content=\"Jeremy Whittaker\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/WhittakerJeremy\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/WhittakerJeremy\" \/>\n<meta property=\"article:published_time\" content=\"2023-04-26T16:30:56+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-04-26T16:31:00+00:00\" \/>\n<meta name=\"author\" content=\"JeremyWhittaker\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"JeremyWhittaker\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/\"},\"author\":{\"name\":\"JeremyWhittaker\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\"},\"headline\":\"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed\",\"datePublished\":\"2023-04-26T16:30:56+00:00\",\"dateModified\":\"2023-04-26T16:31:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/\"},\"wordCount\":463,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/\",\"url\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/\",\"name\":\"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed - Jeremy Whittaker\",\"isPartOf\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/#website\"},\"datePublished\":\"2023-04-26T16:30:56+00:00\",\"dateModified\":\"2023-04-26T16:31:00+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/new.jeremywhittaker.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/#website\",\"url\":\"https:\/\/new.jeremywhittaker.com\/\",\"name\":\"Jeremy Whittaker\",\"description\":\"Research, software, markets, housing, and energy\",\"publisher\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/new.jeremywhittaker.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\",\"name\":\"JeremyWhittaker\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\",\"caption\":\"JeremyWhittaker\"},\"logo\":{\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\"},\"sameAs\":[\"http:\/\/www.jeremywhittaker.com\",\"https:\/\/www.facebook.com\/WhittakerJeremy\",\"https:\/\/www.linkedin.com\/in\/jeremywhittaker\/\"],\"url\":\"https:\/\/new.jeremywhittaker.com\/index.php\/author\/jeremywhittaker\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed - Jeremy Whittaker","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed - Jeremy Whittaker","og_description":"In the realm of algorithm analysis, it is easy to become preoccupied with testing on actual datasets where variables remain unknown. However, to assess algorithms for speed and...","og_url":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/","og_site_name":"Jeremy Whittaker","article_publisher":"https:\/\/www.facebook.com\/WhittakerJeremy","article_author":"https:\/\/www.facebook.com\/WhittakerJeremy","article_published_time":"2023-04-26T16:30:56+00:00","article_modified_time":"2023-04-26T16:31:00+00:00","author":"JeremyWhittaker","twitter_card":"summary_large_image","twitter_misc":{"Written by":"JeremyWhittaker","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#article","isPartOf":{"@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/"},"author":{"name":"JeremyWhittaker","@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c"},"headline":"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed","datePublished":"2023-04-26T16:30:56+00:00","dateModified":"2023-04-26T16:31:00+00:00","mainEntityOfPage":{"@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/"},"wordCount":463,"commentCount":0,"publisher":{"@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/","url":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/","name":"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed - Jeremy Whittaker","isPartOf":{"@id":"https:\/\/new.jeremywhittaker.com\/#website"},"datePublished":"2023-04-26T16:30:56+00:00","dateModified":"2023-04-26T16:31:00+00:00","breadcrumb":{"@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2023\/04\/26\/evaluating-optimization-algorithms-for-ornstein-uhlenbeck-with-synthetic-data-a-comparison-of-accuracy-and-speed\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/new.jeremywhittaker.com\/"},{"@type":"ListItem","position":2,"name":"Evaluating Optimization Algorithms for Ornstein-Uhlenbeck with Synthetic Data: A Comparison of Accuracy and Speed"}]},{"@type":"WebSite","@id":"https:\/\/new.jeremywhittaker.com\/#website","url":"https:\/\/new.jeremywhittaker.com\/","name":"Jeremy Whittaker","description":"Research, software, markets, housing, and energy","publisher":{"@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/new.jeremywhittaker.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c","name":"JeremyWhittaker","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g","caption":"JeremyWhittaker"},"logo":{"@id":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g"},"sameAs":["http:\/\/www.jeremywhittaker.com","https:\/\/www.facebook.com\/WhittakerJeremy","https:\/\/www.linkedin.com\/in\/jeremywhittaker\/"],"url":"https:\/\/new.jeremywhittaker.com\/index.php\/author\/jeremywhittaker\/"}]}},"_links":{"self":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/posts\/4149","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/comments?post=4149"}],"version-history":[{"count":0,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/posts\/4149\/revisions"}],"wp:attachment":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/media?parent=4149"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/categories?post=4149"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/tags?post=4149"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}