{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Validation\n",
    "\n",
    "In this example, we will show how to generate an html report to validate the parameter-fitting process. \n",
    "\n",
    "We begin by generating some synthetic data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pybamm\n",
    "import ionworkspipeline as iwp\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Create a model\n",
    "model = pybamm.lithium_ion.SPMe()\n",
    "\n",
    "# Create a parameter set\n",
    "param = pybamm.ParameterValues(\"Chen2020\")\n",
    "param[\"Current function [A]\"] = \"[input]\"\n",
    "simulation = iwp.Simulation(model, parameter_values=param)\n",
    "\n",
    "# Solve the model at 3 different current rates, and save the current, voltage, and time data with some noise added to the voltage\n",
    "synthetic_data = {}\n",
    "for c_rate in [0.5, 1, 2]:\n",
    "    sol = simulation.solve(\n",
    "        np.arange(0, 3000 / c_rate, 100), inputs={\"Current function [A]\": 5 * c_rate}\n",
    "    )\n",
    "    synthetic_data[f\"{c_rate} C\"] = pd.DataFrame(\n",
    "        {\n",
    "            \"Time [s]\": sol[\"Time [s]\"].entries,\n",
    "            \"Current [A]\": sol[\"Current [A]\"].entries,\n",
    "            \"Voltage [V]\": sol[\"Voltage [V]\"].entries\n",
    "            + np.random.randn(len(sol[\"Time [s]\"].entries)) * 0.01,\n",
    "        }\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the validation\n",
    "\n",
    "Begin by creating an objective for each of the sets of interest (in our case, this is the synthetic data). Then, pass it to an ionworks validator object, run the validation, and export the report. Only `CurrentDriven` is supported for validation at the moment. Summary stats should be a list of `ionworkspipeline.calculations.costs` objects. The default is `RMSE`, `MAE`, and `Max`. Note that each of these has a `scale` parameter, which is used to normalize the data. This should be set to 1 to ensure that the data is not scaled.\n",
    "\n",
    "`Validation.run` takes the parameter values, and returns a dictionary of the validation results and one of the summary statistics. The validation results contains the model and data voltage, current, time, error, and log(error) for each objective. You can plot these results using any plot library, or, as shown in the next cell, use the `export_report` method to export an html report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "validation_objectives = {}\n",
    "for name, data in synthetic_data.items():\n",
    "    validation_objectives[name] = iwp.objectives.CurrentDriven(\n",
    "        data,\n",
    "        options={\n",
    "            \"model\": model,\n",
    "        },\n",
    "    )\n",
    "\n",
    "# Create a summary stat object\n",
    "summary_stats = [\n",
    "    iwp.data_fits.costs.RMSE(scale=1),\n",
    "    iwp.data_fits.costs.MAE(scale=1),\n",
    "    iwp.data_fits.costs.Max(scale=1),\n",
    "]\n",
    "\n",
    "# Create a validator object\n",
    "validator = iwp.Validation(validation_objectives, summary_stats=summary_stats)\n",
    "\n",
    "# Run the validation.\n",
    "validator.run(param)\n",
    "\n",
    "# Print the summary statistics by objective. Here is shown the RMSE, MAE, and Max for the 0.5 C objective.\n",
    "# The RMSE should be around 10 mV given the noise added to the voltage had a standard deviation of 10 mV.\n",
    "summary_stats = validator.summary_stats\n",
    "validation_results = validator.validation_results\n",
    "print(summary_stats[\"0.5 C\"])\n",
    "\n",
    "fig, ax = plt.subplots(1, 2, figsize=(12, 5))\n",
    "colors = {\n",
    "    \"0.5 C\": \"tab:blue\",\n",
    "    \"1 C\": \"tab:orange\",\n",
    "    \"2 C\": \"tab:green\",\n",
    "}\n",
    "for key in validation_results.keys():\n",
    "    ax[0].plot(\n",
    "        validation_results[key][\"Time [s]\"],\n",
    "        validation_results[key][\"Error [mV]\"],\n",
    "        label=key,\n",
    "        color=colors[key],\n",
    "    )\n",
    "    ax[1].plot(\n",
    "        validation_results[key][\"Time [s]\"],\n",
    "        validation_results[key][\"Processed data voltage [V]\"],\n",
    "        \"o\",\n",
    "        label=key + \" processed data\",\n",
    "        color=colors[key],\n",
    "        markerfacecolor=\"none\",\n",
    "    )\n",
    "    ax[1].plot(\n",
    "        validation_results[key][\"Time [s]\"],\n",
    "        validation_results[key][\"Model voltage [V]\"],\n",
    "        label=key + \" model\",\n",
    "        color=\"black\",\n",
    "    )\n",
    "ax[0].set_xlabel(\"Time [s]\")\n",
    "ax[0].set_ylabel(\"Error [mV]\")\n",
    "ax[0].legend()\n",
    "\n",
    "\n",
    "ax[1].set_xlabel(\"Time [s]\")\n",
    "ax[1].set_ylabel(\"Voltage [V]\")\n",
    "ax[1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Export the report\n",
    "\n",
    "Finally, we can export the report to an html file which can be opened in any browser. You can pass in a list of plots, which can be any combination of \"model data\", \"error\", and \"log error\". Default is \"model data\" and \"error\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "validator.export_report(\n",
    "    filename=\"validation_report_synthetic_data.html\",\n",
    "    plots=[\"model data\", \"error\", \"log error\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}