From e01ebdd0a4f678f37fd47e0a6af95f3f3668a716 Mon Sep 17 00:00:00 2001 From: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com> Date: Fri, 4 Oct 2024 19:17:38 +0200 Subject: [PATCH 1/3] add aupimo notebook advanced iii (aupimo score of a random model) Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com> --- .../701d_aupimo_advanced_iii.ipynb | 341 ++++++++++++++++++ 1 file changed, 341 insertions(+) create mode 100644 notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb diff --git a/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb b/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb new file mode 100644 index 0000000000..01eb4dfc20 --- /dev/null +++ b/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb @@ -0,0 +1,341 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# AUPIMO Score of a Random Model\n", + "\n", + "If model randomly assigns scores to the pixels -- i.e. no discrimination -- its AUROC score will be 50%. \n", + "\n", + "What would be its AUPIMO score?\n", + "\n", + "> AUPIMO is pronounced \"a-u-pee-mo\".\n", + "\n", + "> For basic usage, please check the notebook [701a_aupimo.ipynb](./701a_aupimo.ipynb).\n", + "\n", + "> For PIMO curve plots, please check the notebook [701c_aupimo_advanced_ii.ipynb](./701c_aupimo_advanced_ii.ipynb)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "# What is AUPIMO?\n", + "\n", + "The `Area Under the Per-Image Overlap [curve]` (AUPIMO) is a metric of recall (higher is better) designed for visual anomaly detection.\n", + "\n", + "Inspired by the [ROC](https://en.wikipedia.org/wiki/Receiver_operating_characteristic) and [PRO](https://link.springer.com/article/10.1007/s11263-020-01400-4) curves, \n", + "\n", + "> AUPIMO is the area under a curve of True Positive Rate (TPR or _recall_) as a function of False Positive Rate (FPR) restricted to a fixed range. \n", + "\n", + "But:\n", + "- the TPR (Y-axis) is *per-image* (1 image = 1 curve/score);\n", + "- the FPR (X-axis) considers the (average of) **normal** images only; \n", + "- the FPR (X-axis) is in log scale and its range is [1e-5, 1e-4]\\* (harder detection task!).\n", + "\n", + "\\* The score (the area under the curve) is normalized to be in [0, 1].\n", + "\n", + "AUPIMO can be interpreted as\n", + "\n", + "> average segmentation recall in an image given that the model (nearly) does not yield false positives in normal images.\n", + "\n", + "References in the last cell.\n", + "\n", + "![AUROC vs. AUPRO vs. AUPIMO](./roc_pro_pimo.svg)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Setup" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Install `anomalib` using `pip`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# TODO(jpcbertoldo): replace by `pip install anomalib` when AUPIMO is released # noqa: TD003\n", + "%pip install ../.." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Imports" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import torch\n", + "from matplotlib import pyplot as plt\n", + "from matplotlib.axes import Axes\n", + "from matplotlib.ticker import FixedLocator, PercentFormatter\n", + "from numpy import ndarray" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Random Model\n", + "\n", + "If a model cannot discriminate between normal and anomalous images, the survival fuctions\\* of the anomaly scores conditioned to each class would be the same.\n", + "\n", + "> \\* https://en.wikipedia.org/wiki/Survival_function\n", + "\n", + "In other words, FPR and TPR would be the same.\n", + "\n", + "Let's simulate this situation." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0cAAADTCAYAAABKpWhLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABLdklEQVR4nO3dd3gUVdsG8Hu2ZJNNJ70nJJSE3nsViYCggBTxFYKIgALSRIpS9FVEQEGq4CsonyIoXXoLSJNehUA6JZX0bPrO90fIypJANiHJpNy/69qL7Jkzs8/shnny7Jw5I4iiKIKIiIiIiKiGk0kdABERERERUWXA4oiIiIiIiAgsjoiIiIiIiACwOCIiIiIiIgLA4oiIiIiIiAgAiyMiIiIiIiIALI6IiIiIiIgAsDgiIiIiIiICwOKIiIiIiIgIAIsjIqqhPD09ERAQIMlrz5s3D4IgSPLazxIeHg5BELBhwwapQ6m0AgMDIQgCAgMDpQ6FiIjKCYsjIqpWrl+/jjfeeAMeHh4wNjaGi4sLXn75ZSxfvlzq0MrEqlWrJC1gCgqEoh5Dhw7V9evataveslq1aqFVq1b48ccfodVqdf0CAgL0+qlUKtStWxdz5sxBZmamFLv4Qp713jz9CAwM1BWkBQ+5XA53d3f0798fV65cee52LSws0KVLF+zZs0eaHSUiqqYUUgdARFRWTp8+jW7dusHd3R2jR4+Go6Mj7t27h7Nnz2LZsmWYMGGCrm9QUBBksqr3/dCqVatga2sr2VmvAhMnTkSrVq302jw9PfWeu7q6YsGCBQCAuLg4/Pzzzxg1ahTu3LmDr776StdPpVLhhx9+AAAkJydj586d+PzzzxESEoJffvmlfHekjG3cuFHv+c8//4xDhw4Vavf19UVGRgYA4M0330Tv3r2Rl5eHW7duYfXq1di3bx/Onj2Lpk2b6tZ5+eWXMXz4cIiiiIiICKxevRp9+/bFvn374O/vX+77RkRUE7A4IqJq44svvoClpSXOnz8PKysrvWWxsbF6z1UqVQVGVv106tQJb7zxxnP7WFpa4j//+Y/u+ZgxY1CvXj2sWLECn3/+OZRKJQBAoVDo9Xv//ffRvn17bNq0Cd988w0cHBzKZyfKwZP7AQBnz57FoUOHCrUD+UMZAaB58+Z6yzt06IB+/fph9erV+P7773XtdevW1es3cOBA+Pn5YdmyZZWiOEpPT4epqanUYRARvZCq97UpEdEzhISEoEGDBoUKIwCwt7fXe/70NUcbNmyAIAg4efIkJk6cCDs7O1hZWWHMmDHIzs5GUlIShg8fDmtra1hbW2P69OkQRVG3/rOuRzH0Wp7169eje/fusLe3h0qlgp+fH1avXl0o5ps3b+L48eO64VVdu3bVLU9KSsKkSZPg5uYGlUoFHx8fLFy4UG8YW0G/gIAAWFpawsrKCiNGjEBSUtJz4ysLarUabdu2RXp6OuLi4p7ZTxAEdOzYEaIoIjQ0tNjt7ty5E3369IGzszNUKhW8vb3x+eefIy8vT69f165d0bBhQ/zzzz/o1q0b1Go1XFxc8PXXXxfa5v379/H666/D1NQU9vb2mDx5MrKyskq+06XQvXt3AEBYWNhz+/n6+sLW1hYhISEGbTczMxPz5s1D3bp1YWxsDCcnJwwYMEC3fkl+hwMCAmBmZoaQkBD07t0b5ubmeOuttzB+/HiYmZlBo9EUev0333wTjo6Oep/Lvn370KlTJ5iamsLc3Bx9+vTBzZs39daLjo7GyJEj4erqCpVKBScnJ7z22mu64pKIqCzxzBERVRseHh44c+YMbty4gYYNG5ZqGxMmTICjoyPmz5+Ps2fPYu3atbCyssLp06fh7u6OL7/8Env37sWiRYvQsGFDDB8+vExiX716NRo0aIB+/fpBoVBg9+7deP/996HVavHBBx8AAJYuXYoJEybAzMwMs2fPBgDdWRWNRoMuXbrgwYMHGDNmDNzd3XH69GnMnDkTUVFRWLp0KQBAFEW89tprOHnyJMaOHQtfX19s374dI0aMKFG8qampiI+P12urVatWsUMVQ0NDIZfLiyxgn1Twh6+1tXWxsWzYsAFmZmaYMmUKzMzMcPToUcyZMwcpKSlYtGiRXt/ExES88sorGDBgAAYPHow//vgDH3/8MRo1aoRevXoBADIyMvDSSy8hMjISEydOhLOzMzZu3IijR48WG0tZKChWbGxsntsvOTkZiYmJ8Pb2LnabeXl5ePXVV3HkyBEMHToUH374IVJTU3Ho0CHcuHHDoG08LTc3F/7+/ujYsSMWL14MtVoNT09PrFy5Env27MGgQYN0fTUaDXbv3o2AgADI5XIA+UMQR4wYAX9/fyxcuBAajQarV69Gx44dcfnyZd0wzYEDB+LmzZuYMGECPD09ERsbi0OHDiEyMrLQUE4iohcmEhFVEwcPHhTlcrkol8vFdu3aidOnTxcPHDggZmdnF+rr4eEhjhgxQvd8/fr1IgDR399f1Gq1uvZ27dqJgiCIY8eO1bXl5uaKrq6uYpcuXXRtx44dEwGIx44d03udsLAwEYC4fv16XdvcuXPFpw+/Go2mUIz+/v5i7dq19doaNGig97oFPv/8c9HU1FS8c+eOXvuMGTNEuVwuRkZGiqIoijt27BABiF9//bXe/nTq1KlQnEUp2M+iHmFhYbp+Xbp0EevXry/GxcWJcXFx4q1bt8SJEyeKAMS+ffvq+o0YMUI0NTXV9QsODhYXL14sCoIgNmzYUO+zeJai3rsxY8aIarVazMzM1IsJgPjzzz/r2rKyskRHR0dx4MCBuralS5eKAMQtW7bo2tLT00UfH58iP+Pn+eCDDwp91gUKfjfmz58vxsXFidHR0WJgYKDYrFkzEYC4detWXV8A4qhRo8S4uDgxNjZWvHDhgvjKK6+IAMRFixYVG8ePP/4oAhC/+eabQssK3uOS/A6PGDFCBCDOmDGj0LZcXFz03k9RFMUtW7aIAMQTJ06IoiiKqampopWVlTh69Gi9ftHR0aKlpaWuPTEx0eB9JCIqCxxWR0TVxssvv4wzZ86gX79+uHr1Kr7++mv4+/vDxcUFu3btMmgbo0aN0ptmu02bNhBFEaNGjdK1yeVytGzZ0qAhX4YyMTHR/ZycnIz4+Hh06dIFoaGhSE5OLnb933//HZ06dYK1tTXi4+N1jx49eiAvLw8nTpwAAOzduxcKhQLjxo3T258nJ6swxJw5c3Do0CG9h6Ojo16f27dvw87ODnZ2dvD19cXy5cvRp08f/Pjjj3r90tPTdf18fHwwbdo0dOjQATt37jRoyvMn37uCM1qdOnWCRqPB7du39fqamZnpXbdjZGSE1q1b632We/fuhZOTk941VWq1Gu+9955hb04JzZ07F3Z2dnB0dETXrl0REhKChQsXYsCAAXr9/ve//8HOzg729vZo2bIljhw5gunTp2PKlCnFvsbWrVtha2tb5Of8ItPKP/l7VLCtQYMGYe/evUhLS9O1b968GS4uLujYsSMA4NChQ0hKSsKbb76p9/sql8vRpk0bHDt2DED+Z2tkZITAwEAkJiaWOk4iIkNxWB0RVSutWrXCtm3bkJ2djatXr2L79u349ttv8cYbb+DKlSvw8/N77vru7u56zy0tLQEAbm5uhdrL8o+1U6dOYe7cuThz5kyh6zWSk5N1cTzL3bt3ce3aNdjZ2RW5vGBCioiICDg5OcHMzExveb169UoUb6NGjdCjR4/n9vH09MS6desgCAKMjY1Rp06dQtd+AYCxsTF2794NIP9an6+//hqxsbF6Rc/z3Lx5E5988gmOHj2KlJQUvWVPF5aurq6FigFra2tcu3ZN9zwiIgI+Pj6F+pX0PTLUe++9h0GDBkEmk8HKygoNGjQocsKQ1157DePHj0d2djbOnz+PL7/8EhqNxqBZF0NCQlCvXj0oFGWX9hUKBVxdXQu1DxkyBEuXLsWuXbswbNgwpKWlYe/evRgzZozuPb179y6Af6+vepqFhQWA/IlTFi5ciKlTp8LBwQFt27bFq6++iuHDhxcqxomIygKLIyKqloyMjNCqVSu0atUKdevWxciRI/H7779j7ty5z12v4HoIQ9rFJyZkeNa3709PClCUkJAQvPTSS6hfvz6++eYbuLm5wcjICHv37sW3335baEKFomi1Wrz88suYPn16kcvr1q1b7DbKmqmpabEFFJD/3j7Zz9/fH/Xr18eYMWOKPeOXlJSELl26wMLCAp999hm8vb1hbGyMS5cu4eOPPy703j3r833ys6xoderUMeh9cnV11fXr3bs3bG1tMX78eHTr1q3QWabSKOnvsEqlKrIwa9u2LTw9PbFlyxYMGzYMu3fvRkZGBoYMGaLrU/C5bNy4scgi58kibtKkSejbty927NiBAwcO4NNPP8WCBQtw9OhRNGvWrET7SERUHBZHRFTttWzZEgAQFRVVbq9RMHHA07O+RUREFLvu7t27kZWVhV27dumduSoYWvSkZ/0B6+3tjbS0tGL/yPbw8MCRI0eQlpamd/YoKCio2DgripOTEyZPnqybFKNt27bP7BsYGIhHjx5h27Zt6Ny5s669uJnensfDwwM3btyAKIp673dleo+A/KnRv/32W3zyySfo37//c4fHeXt74++//0ZOTo5uCvWnvcjv8NMGDx6MZcuWISUlBZs3b4anp6fe51gwAYS9vb1BhaG3tzemTp2KqVOn4u7du2jatCmWLFmC//u//ytxbEREz8Nrjoio2jh27FiRZwD27t0LoPyGRQH5f1DL5XLdtT0FVq1aVey6BWcznow9OTkZ69evL9TX1NS0yGm3Bw8ejDNnzuDAgQOFliUlJSE3NxdA/hmH3NxcvWnC8/LysHz58mLjrEgTJkyAWq3Wu1lsUYp677Kzsw1635+ld+/eePjwIf744w9dm0ajwdq1a0u9zfKgUCgwdepU3Lp1Czt37nxu34EDByI+Ph4rVqwotKzgvXuR3+GnDRkyBFlZWfjpp5+wf/9+DB48WG+5v78/LCws8OWXXyInJ6fQ+gVTvWs0GmRmZuot8/b2hrm5eYVNrU5ENQvPHBFRtTFhwgRoNBr0798f9evXR3Z2Nk6fPq375nrkyJHl9tqWlpYYNGgQli9fDkEQ4O3tjT///LPQzWeL0rNnTxgZGaFv374YM2YM0tLSsG7dOtjb2xc629WiRQusXr0a//3vf+Hj4wN7e3t0794dH330EXbt2oVXX30VAQEBaNGiBdLT03H9+nX88ccfCA8Ph62tLfr27YsOHTpgxowZCA8Ph5+fH7Zt22bQpA8VycbGBiNHjsSqVatw69Yt+Pr6Ftmvffv2sLa2xogRIzBx4kQIgoCNGze+0DC50aNHY8WKFRg+fDguXrwIJycnbNy4EWq1utTbLC8BAQGYM2cOFi5ciNdff/2Z/YYPH46ff/4ZU6ZMwblz59CpUyekp6fj8OHDeP/99/Haa6+90O/w05o3bw4fHx/Mnj0bWVlZekPqgPxrilavXo23334bzZs3x9ChQ2FnZ4fIyEjs2bMHHTp0wIoVK3Dnzh289NJLGDx4MPz8/KBQKLB9+3bExMRg6NChJY6LiKhYks2TR0RUxvbt2ye+8847Yv369UUzMzPRyMhI9PHxESdMmCDGxMTo9X3WVN7nz5/X61cw7XZcXJxee8EU1E+Ki4sTBw4cKKrVatHa2locM2aMeOPGDYOm8t61a5fYuHFj0djYWPT09BQXLlyom375ySmyo6OjxT59+ojm5uYiAL1pvVNTU8WZM2eKPj4+opGRkWhrayu2b99eXLx4sd505o8ePRLffvtt0cLCQrS0tBTffvtt8fLlyyWayvv3339/br8uXbqIDRo0eG4fUSz6fSwQEhIiyuVyvc+pKKdOnRLbtm0rmpiYiM7Ozrop3PHUtNTPimnEiBGih4eHXltERITYr18/Ua1Wi7a2tuKHH34o7t+/v1ym8jZkmmoA4gcffFDksnnz5hkUl0ajEWfPni16eXmJSqVSdHR0FN944w0xJCRE18fQ3+HnfW4FZs+eLQIQfXx8ntnn2LFjor+/v2hpaSkaGxuL3t7eYkBAgHjhwgVRFEUxPj5e/OCDD8T69euLpqamoqWlpdimTRu9adaJiMqSIIoSXoVKRERERERUSfCaIyIiIiIiIrA4IiIiIiIiAsDiiIiIiIiICACLIyIiIiIiIgAsjoiIiIiIiACwOCIiIiIiIgLA4oiIiIiIiAgAiyMiIiIiIiIALI6IiIiIiIgAsDgiIiIiIiICACikDqAy0mq1ePjwIczNzSEIgtThEBHRCxBFEampqXB2doZMZth3gswDRETVi6G5gMVRER4+fAg3NzepwyAiojJ07949uLq6GtSXeYCIqHoqLhewOCqCubk5gPw3z8LCQuJoiIjoRaSkpMDNzU13bDcE8wARUfViaC6o9MXRiRMnsGjRIly8eBFRUVHYvn07Xn/99eeuExgYiClTpuDmzZtwc3PDJ598goCAAINfs2AIhYWFBZMiEVE1UZLhccwDRETVU3G5oNJPyJCeno4mTZpg5cqVBvUPCwtDnz590K1bN1y5cgWTJk3Cu+++iwMHDpRzpEREREREVJVV+jNHvXr1Qq9evQzuv2bNGnh5eWHJkiUAAF9fX5w8eRLffvst/P39yytMIiIiIiKq4ip9cVRSZ86cQY8ePfTa/P39MWnSpGeuk5WVhaysLN3zlJQUAEBOTg5ycnJKHMODpAxM+f061EZyqI3kMDWSw1SlePxcoWtTP263UitRS20Ea7US5sYKzoxERFSGDDmOl3UeAIC1f4Xh+J143bG+4LivywOqf59bGCtgrTZ6nA+UUCnlpXpNIiIqmqHH8mpXHEVHR8PBwUGvzcHBASkpKcjIyICJiUmhdRYsWID58+cXaj948CDUanWJY7iXBlyKLN1bK4MItRIwVQBmCsDCSISlEWCtEmFlBFgbibBSAeZKQMYaiqhGkMlkBk9BXVPl5eVBFMUil2k0mmLXL+s8AADHg2U4F1e6z81IJsJUAZgqATNFQR4ArIxEvX9VrKGIagTmgeJptVpotdpnLjckFwDVsDgqjZkzZ2LKlCm65wWzWfTs2bNUF+ImZ+TAq2ECNNl5SM/OQ3pWLjTZeY8fuUjP+vfntKw8JGmykajJQXp2HrQQkJYDpOUAMQCAoisgY6UMXjamqG1ritp2atS2NUUdezN425lCIed/HqLqICcnBzExMcjIyJA6lEpPEAQ4OTnB1NS00LKCs0DPU9Z5AADcH6QgIkGTf9zPzoPmiWN/+lM5ISUzB4maHCRpcpCrFZGtFZCdDSRmA8/KAwBgb65CbVs1atuZwsvWFN62pvB1MoetmapUMRNR5cI8UDImJiZwcHCAUqkstMyQXABUw+LI0dERMTExem0xMTGwsLAo8qwRAKhUKqhUhROJUqks8s0tjq1SiT5NSv5NY1ZuHpI0OUhIz0aiJhsJ6dmISclCVFIGopIz8TA5A1FJmYhNzURmjha3olNxKzpVbxvGShkaOFuikYslmrhZorWXDVysit5vIqq8tFotQkNDIZfL4eLiAiMjIw65fQZRFBEXF4fo6GjUqVMHcrn+6RRDjuNlnQcAoJmnDZp52pRoHVEUkZqVi8T0/C/NEtOzEZeWhejkTEQlZ+BhUv6/UUmZSM3KRWxqFmJTs3A2LFFvO86WxmjkaonGrlZo6maFFh7WMOZQPaIqhXnAcKIoIjs7G3Fxcbh37x7q1KlT6EybocfyalcctWvXDnv37tVrO3ToENq1aydRRIZTKeRwsJDDwcL4uf1y8rS4n5iBkNg0hMTlP4Jj0xAUnYr07DxcjEjExYh/E6WHjRrtvW3QztsWnevYwkptVN67QkQvKDs7G1qtFm5ubqUe1lWT2NnZITw8HDk5OYWKo6pEEARYGCthYayERzF1VXJGDkLj0hAal67LBXdj0hD2KB0PkzPxMDkTB27mf1loJJehmbsV2nvbomMdGzRzs4aMY7OJKjXmgZIxMTGBUqlEREQEsrOzYWz8/L+nn6XSF0dpaWkIDg7WPQ8LC8OVK1dQq1YtuLu7Y+bMmXjw4AF+/vlnAMDYsWOxYsUKTJ8+He+88w6OHj2KLVu2YM+ePVLtQplTymXwss0fQtED/15fpdWKCI1Px/UHSbh2PxmXI5Nw/UEyIh5pEPFIg03n7kEuE9C2di34N3BETz9HOFqW7heHiCoGx5gbpiZ+m2ppokQzd2s0c7fWa0/NzMHNhym48SAZV+8n43xYAqJTMvF3WAL+DkvAt4cBWzMVXvZzgH8DB7T3toWRgr9nRJUV84DhyuK9qvTF0YULF9CtWzfd84Ix4SNGjMCGDRsQFRWFyMhI3XIvLy/s2bMHkydPxrJly+Dq6ooffvihRkzjLZMJ8LE3g4+9Gfo3cwWQnyTPhyfgTMgjnLgTj6CYVJwKfoRTwY8wd9dNdPSxxeCWbujZwAEqRdX9tpWIiPKZGyvRtrYN2tbOP/UkiiLC4tNxJvQRTgc/wok7cYhPy8Kmc5HYdC4StUyN8HpTFwxp5YZ6js+/czwRUXUniM+a3qcGS0lJgaWlJZKTk6vdndHD49Nx4GY0DtyMxqXIJF27lVqJIS3dENDBE06WvEaJSGqZmZkICwuDl5dXqYcG1CTPe79Kc0yvznkgO1eLM6GPcOBmNA7ejEF82r9TmDdxs8Kojl7o3dCRk/sQSYx5oOTKIhewOCpCdU6KT4p4lI4/Lt7HHxfvIyo5EwCgkAno19QZY7t4o64Dv0EkkgqTYsmwOCqd3DwtTtyNw5bz93H4Vgxytfl/Erham2BURy8MbeUOEyOOKiCSAvNAyZVFLuDXQjWYh40ppvash5Mfd8cPw1uitVct5GpFbLv0AP5LT2DKliu4n2jYnPBERAUCAgIgCEKhR3BwsN4yIyMj+Pj44LPPPkNubi4AIDAwUG8dOzs79O7dG9evX5d4r6onhVyG7vUdsObtFjg76yVM7lEXtUyNcD8xA/N3/4Mui47h178jkZv37HuHEBE9qajj/5OPefPmITw8XK/NxsYGPXv2xOXLl3Xb6dq1q265sbEx6tatiwULFjzznnZlhcURQS4T0MPPAVvGtMP299vjlQaOEEVg26UH6L74OL7Y8w/SsnKlDpOIqpBXXnkFUVFReg8vLy+9ZXfv3sXUqVMxb948LFq0SG/9oKAgREVF4cCBA8jKykKfPn2QnZ0txa7UGLZmKnzYow5Ofdwd/329IVytTRCbmoVZ26+j59ITOHo7pviNEFGN9+Rxf+nSpbCwsNBrmzZtmq7v4cOHdcf6tLQ09OrVC0lJSbrlo0ePRlRUFIKCgjBz5kzMmTMHa9asKdf4K/2EDFSxmrlbY83bLXD1XhK+2ncbZ0IfYd1fYfjzWhTm92uAng0cpQ6RqMYSRREZOXmSvLaJUl6iGeFUKhUcHYs+Xjy5bNy4cdi+fTt27dqFmTNn6vrY29vDysoKjo6OmDRpEvr164fbt2+jcePGL7YjVCwTIzn+09YDg1q64pezkVh+9C5C49LxzoYL6NPICXP6+hV7ywkiKh9VIQ88eey3tLSEIAiF8kF8fDwAwMbGBo6OjnB0dMTixYvRoUMH/P3337qJ1NRqtW7dkSNHYsWKFTh06BDGjRtXVrtVCIsjKlITNyv8OroNAu/EYe7Om4hM0OC9jRfh38ABX/ZvBBvefZ2owmXk5MFvzgFJXvufz/yhNiqflGFiYoJHjx4VuSw5ORm//fYbAMDIiPdoq0gqhRzvdPTCoJauWH40GP87GYY916Nw4k4cZvfxxZBWbjVyCnUiKVXXPADk5wIARY4SEEURJ0+exO3bt1GnTp1yiwHgsDp6DkEQ0K2ePQ5O7oz3u3pDIRNw4GYMXln2F07ciZM6PCKqxP7880+YmZnpHoMGDSrURxRFHD58GAcOHED37t31lrm6usLMzAxWVlb49ddf0a9fP9SvX7+iwqcnmBsrMau3L3aN74AmblZIzcrFjG3X8f4vl5Ck4VBHInpxSUlJ+Pzzz2FmZobWrVvr2letWgUzMzOoVCp07twZWq0WEydOLNdYeOaIimWslGP6K/XxamNnfPjbZdyNTcPwH89hbBdvfORfD3LeZZ2oQpgo5fjnM2nu2WaiLNmMZd26dcPq1at1z01NTXU/FxROOTk50Gq1GDZsGObNm6e3/l9//QW1Wo2zZ8/iyy+/LPcx5lS8Bs6W2DauPX74KxSLDwZh341oXLmXhBXDmqOFh3XxGyCiF1aV8oAh2rdvD5lMhvT0dNSuXRubN2+Gg4ODbvlbb72F2bNnIzExEXPnzkX79u3Rvn37Mo/jSSyOyGB+zhbYPaEjvthzCxvPRmDN8RD8E5WC5UObwVKtlDo8ompPEIRyHdJQlkxNTeHj41PksoLCycjICM7OzlAoCu+Tl5cXrKysUK9ePcTGxmLIkCE4ceJEeYdNxZDLBIzp4o323rb48LfLCI1Px5trz+K//RticEs3qcMjqvaqUh4wxObNm+Hn5wcbGxtYWVkVWm5paanLJVu2bIGPjw/atm2LHj16lFtMHFZHJWKslOPz1xti5bDmMFHKceJOHF5beRLBsWlSh0ZEVURB4eTu7l5kYfS0Dz74ADdu3MD27dsrIDoyRCNXS+ye0BG9GjoiO0+L6X9cw/zdN5Gn5a0Tichwbm5u8Pb2LrIwepqZmRk+/PBDTJs2rVyn82ZxRKXSp7ET/hjXDi5WJgh/pMGgNadx5V6S1GERUTWkVqsxevRozJ07t9zvb0GGM1UpsHJYc0zqkX9x9PpT4Ziw6RKycqWZSYuIqr8xY8bgzp072Lp1a7m9BosjKrUGzpbYOb4DmrhaIlGTg2HrzuLk3XipwyKiamj8+PG4desWfv/9d6lDoSfIZAIm9aiLlcOaw0guw97r0Ri5/jzvjUdE5aJWrVoYPnw45s2bB622fG5OLYj8Gq6QlJQUWFpaIjk5GRYWFlKHU+mlZeVi7MaLOBkcD6VcwJr/tMBLvg7Fr0hEz5SZmYmwsDB4eXnB2Jj3lCnO896v0hzTmQdK7lRwPN77+QLSs/PQxM0KG0e1hoUxr0clKi3mgZIri1xQbmeOUlNTy2vTVMmYqRT4X0BL9G7kiJw8EeP+7xKOc6pvIqIapYOPLTa91xbWaiWu3kviGSQiqpLKvDiKjY3FrFmz4O7uXtabpkpMpZDju6HNdBfnvvfzBZwK5hA7IqKapLGrFf7v3TawMFbgYkQiRm04j4xsXoNERFVHiYqj2NhYLF68GOPGjcOsWbNw8eJF3bIHDx7g/fffh6enJ7766is0a9aszIOlyk0hl2HZ0Gbo4WuPrFwt3v3pAq7dT5I6LCIiqkANnC2xcVQbmKsU+DssAe//chG5eeVzbQARUVkzuDi6ffs2GjZsiI8//hjff/89vvrqK7Rp0wa//fYbNm/ejPr162PdunXo168fzp8/j6NHj5Zn3FRJGSlkWPlWc3T0sUVGTh7e2XAB9xI0UodFREQVqImbFTa80wrGShmOBcVhzq6bnGmQiKoEg4ujTz/9FGlpaVi1ahVu3LiB3bt3o3bt2pg0aRICAgLQq1cvBAUF4bfffkOLFi3KM2aq5FQKOVb/pznqO5ojPi0LIzecR7ImR+qwiKok/kFpGL5PlU8Lj1pYNrQZBAH49e9IrDkeKnVIRFUSj2+GK4v3yuDi6MSJExg3bhzGjBkDPz8/9OnTB8uXL0dsbCyGDh2KLVu2oHbt2i8cEFUP5sZKrB/ZCo4WxgiOTcM4DqsgKhGlMn+WL42GZ14NkZ2dDQCQy+USR0JP8m/giDmv+gEAFu6/jf03oiSOiKjqYB4ouYL3quC9K43ib03+2KNHj9C4cWO9tiZNmgAA+vfvX+oAqPpysjTBjwGt8Maa0zgd8ghfHwjCrN6+UodFVCXI5XJYWVkhNjYWQP6NUAVBkDiqykmr1SIuLg5qtRoKhcFpjSrIyA5eiEzQYP2pcEzdchXedmao42AudVhElR7zgOFEUYRGo0FsbCysrKxe6Isyg7OIVqstVIUVPDczMyt1AFS9+TlbYNEbTfDBr5ew9kQoGrta4tXGzlKHRVQlODo6AoAuMdKzyWQyuLu78w+HSmp2b1/cjkrFmdBHGLPxInaM78B7IBEZgHmgZKysrHTvWWmV6Cu2Cxcu6N1QKTU1FYIg4OTJk0hKSirUf8CAAS8UHFUPfRo74dqD2vj+eCg++v0a6tibo54jvzUkKo4gCHBycoK9vT1ycnjd3vMYGRlBJiu3W/fRC1LIZVgxrBn6Lj+J0Ph0TN1yFWvfbsFilqgYzAOGUyqVZTK0WhANvHKppElHEATk5VXNexvwzuhlLzdPi4D153EyOB71HMyxc3wHGCt5bQARlb/SHNOZB8rHtftJeGPNGWTnajG/XwOMaO8pdUhEVEMYelw3+MzRsWPHyiQwqpkUchmWDm2KV5b+haCYVCzYewvzX2sodVhERFSBGrtaYXZvX8zddRNf7L2FNrVrob4ji08iqjwMLo66dOlSnnFQDWBrpsLiQY0RsP48fjoTgc517fCSr4PUYRERUQUa3s4Dx+/E4ejtWEzcdBm7xnfkSAIiqjRKNFYuMzMTmzdvxldffYV169YhKopTclLJdK1nj1EdvQAAH/1xDbEpmRJHREREFUkQBCx6ozHszFW4E5OGL/bckjokIiIdg4uj2NhYNGzYEMOGDcOsWbMwZswY1KlTB4cPHy7P+Kgamv5KPfg6WSAhPRszt13nzc2IiGoYGzMVlgzKvx3IxrMR+OtunMQRERHlM7g4+vzzzxEeHo7Jkyfjzz//xNKlS2FiYoIxY8aUZ3xUDakUciwb2hRGchmO3I7FzisPpQ6JiIgqWOe6dgh4PCHDjK3XkZaVK21AREQoQXF08OBBDB8+HIsXL0bv3r0xceJErFixAuHh4QgKCirPGKkaqutgjgndfQAA83bfRFxqlsQRERFRRfvIvx5crU3wICkDX++/LXU4RESGF0eRkZHo2LGjXlvHjh0hiiJiYmLKPDCq/sZ29YafkwWSNDmYt+um1OEQEVEFM1UpsHBgYwDAz2ci8HfoI4kjIqKazuDiKCsrS+8GsAB0z3NzeSqcSk4pl+HrNxpDLhOw53oU9t/gBB9ERDVNBx9bvNnaDQDw8dZryMypmvdIJKLqweCpvAEgPDwcly5d0j1PTk4GANy9exdWVlaF+jdv3vzFoqNqr6GLJcZ18caKY8GYt+sfdKpjB1NViX4tiYioipvZ2xfHbsch/JEGqwJDMOXlulKHREQ1lCAaOFWYTCaDIAiF2kVRLNRe0JaXVzW//eGd0StWZk4eXv72OO4lZGBM59qY2dtX6pCIqBopzTGdeaDi7b0ehfd/uQQjhQwHJ3WGp62p1CERUTVi6HHd4K/o169fXyaBET3NWCnH/H4N8M6GC/jfyTC80cIVdRzMpQ6LiIgqUK+GjuhUxxZ/3Y3HvN03sT6gVZFfyhIRlSeDi6MRI0aUZxxUw3Wv74CX/Rxw6J8YfLrzBjaNbsukSERUgwiCgM9eawj/b08gMCgOB27G4JWGjlKHRUQ1jMETMsjlcmzatKk8Y6Eabs6rfjBWynA2NAG7rvLeR0RENY2XrSne61wbAPDZ7pvQZHPCJyKqWAYXR6IowsDLk4hKxa2WGuO75d/76L97biGdNwQkIqpxPujmAxcrEzxMzsSqYyFSh0NENYzBxRFRRRjduTY8bNSIS83C98eZFImIahoTIznm9PUDAKz7KxQPkjIkjoiIapISFUe8BoTKm0ohx8xe9QEAa/8KxUMmRSKiGqennwPaeNVCVq4Wi/bfljocIqpBSjSVt6+vLxwcHAzbsCDgyJEjLxScVDiFq7REUcSQ78/iXHgCBjRzwTdDmkodEhFVYZzKu2q68SAZfVechCgCOz7ogKZuVlKHRERVmKHH9RKdOUpNTUVcXJxBj9jY2BfeCaqZBEHAJ6/m3+to2+UHuHY/SdqAiIiowjV0scSAZq4AgC/2/MPrnomoQhg8lTcAfPXVVxg2bFh5xUKk09jVCgOauWDb5Qf475+3sHkMp/YmIqppPvKvhz3XH+J8eCL234hGr0ZOUodERNUcJ2SgSmuafz2oFDKcC0/A/hvRUodDREQVzNHSGO919gYALNh3G1m5eRJHRETVHYsjqrScrUx097v4+kAQcvK0EkdEREQVbUzn2rA3VyEyQYNfzkZKHQ4RVXMsjqhSG9PFG7VMjRAWn44/Lt6XOhwiIqpgpioFJr9cFwCw8lgw0ngPPCIqRwYXR8eOHUOPHj3KMxaiQsxUCnzw+MawSw/fQWYOh1QQEdU0g1q4wsvWFI/Ss/G/v8KkDoeIqjGDi6MuXbrA3t6+PGMhKtJbbdzhYmWCmJQs/HwmXOpwiIioginkMkztmX/2aN1foUhIz5Y4IiKqrjisjio9Y6Uck3rUAQCsCgxBSmaOxBEREVFF693QCQ1dLJCWlYtVx4KlDoeIqikWR1QlDGjuCh97MyRpcrDuRKjU4RARUQWTyQR85F8fAPDz2Qg8SMqQOCIiqo5YHFGVIJcJmNazHgDgfyfDEJeaJXFERERU0TrXsUXb2rWQnavFssN3pA6HiKqhEhdHmZmZ+O6773DixInyiIfomfwbOKCJmxU02XlYySEVREQ1jiAImP5K/tmjPy7eR3BsmsQREVF1U+LiyNjYGB9//DGCgoLKIx6iZxIEAR/75589+uXvCNxL0EgcERERVbTm7tZ42c8BWhFYcpB/ixBR2SrVsLqGDRsiPDy8jEMhKl57H1t09LFFTp6IZUfuSh0OERFJ4CP/ehAEYN+NaFy/nyx1OERUjZSqOPriiy/w/fff4/Dhw2UdD1Gxpj0+e7Tt0n0Ex6ZKHA0REVW0ug7meL2pCwBgMc8eEVEZUpRmpRUrVqBWrVrw9/eHl5cXvLy8YGJiotdHEATs3LmzTIIkelJTNyu87OeAQ//E4NtDd7HyreZSh0RERBVsUo862H31IY7ficO5sAS09qoldUhEVA2U6szRtWvXkJOTA3d3d+Tl5SE4OBjXr18v9CAqL1N71oUgAHuuR+HGAw6pICKqaTxsTDG4lRsAYPGBIIiiKHFERFQdlKo4Cg8PR1hY2HMfoaFldy+alStXwtPTE8bGxmjTpg3OnTv3zL4bNmyAIAh6D2Nj4zKLhSqH+o4W6NfEGQAvyCUiqqkmdPeBkUKGc+EJOHE3XupwiKgaqPT3Odq8eTOmTJmCuXPn4tKlS2jSpAn8/f0RGxv7zHUsLCwQFRWle0RERFRgxFRRJveoC7lMwLGgOFwIT5A6HCIiqmBOliYY3tYDAM8eEVHZeKHi6Pjx45g+fTqGDBmCIUOGYPr06Th+/HhZxQYA+OabbzB69GiMHDkSfn5+WLNmDdRqNX788cdnriMIAhwdHXUPBweHMo2JKgdPW1MMbpk/pOJrJkUiohppXFdvmBrJcf1BMg7cjJY6HCKq4ko1IUN2djbefPNN7NixA6IowsrKCgCQlJSEJUuWoH///ti0aROUSuULBZednY2LFy9i5syZujaZTIYePXrgzJkzz1wvLS0NHh4e0Gq1aN68Ob788ks0aNDgmf2zsrKQlZWle56SkgIAyMnJQU5OzgvtA5WvcZ09sfXSfZwLS8Cx29Ho5GMrdUhEVMkYchxnHqi6LFQyBLT3wMrAUCw6EISudWwglwlSh0VElYyhx/JSFUfz58/H9u3bMW3aNEydOlV3ZiY2NhZLlizBokWL8Nlnn+Hzzz8vzeZ14uPjkZeXV+jMj4ODA27fvl3kOvXq1cOPP/6Ixo0bIzk5GYsXL0b79u1x8+ZNuLq6FrnOggULMH/+/ELtBw8ehFqtfqF9oPLX3k6GwCgZ5v5xEVMb5UFgTiSiJ2g0xd8wmnmganPLBdRyOULi0vHfjfvRyo4jCYhInyG5AAAEsRRjkby8vNC1a1esX7++yOUBAQEIDAx84RvFPnz4EC4uLjh9+jTatWunay8Yvvf3338Xu42cnBz4+vrizTfffGaxVtQ3hm5uboiPj4eFhcUL7QOVv0dpWej+7UlosvOw8s0m6OnHYZRE9K+UlBTY2toiOTn5mcd05oGq7/sTYVh86C5crU1wYGIHGCkq/WXVRFSBDMkFQCnPHEVFRaFNmzbPXN6mTRv89ttvpdm0HltbW8jlcsTExOi1x8TEwNHR0aBtKJVKNGvWDMHBwc/so1KpoFKpilz3RYcGUvlztFZiVEcvLD8ajGVHQ/BKIxcOqSAiHUOO48wDVd87nWrjp7ORuJ+Yge1Xo/GfxxM1EBEBhuUCoJQTMri6uiIwMPCZy48fP/7MIWwlYWRkhBYtWuDIkSO6Nq1WiyNHjuidSXqevLw8XL9+HU5OTi8cD1Ve73aqDQtjBe7EpGHX1QdSh0NERBVMbaTA+G4+AIDlR+8iMydP4oiIqCoqVXE0YsQIbNmyBWPHjkVQUBDy8vKg1WoRFBSEcePG4ffff0dAQECZBDhlyhSsW7cOP/30E27duoVx48YhPT0dI0eOBAAMHz5cb8KGzz77DAcPHkRoaCguXbqE//znP4iIiMC7775bJvFQ5WRposTYrt4AgG8P3UVOnlbiiIiIqKINbe0GFysTxKRkYeMZ3saDiEquVMPqZs2ahZCQEKxduxbr1q2DTJZfY2m1WoiiiBEjRmDWrFllEuCQIUMQFxeHOXPmIDo6Gk2bNsX+/ft1kzRERkbqXh8AEhMTMXr0aERHR8Pa2hotWrTA6dOn4efnVybxUOUV0N4TP54MR2SCBlsu3MNbbTikgoioJlEp5PiwRx1M/+MaVgUGY2hrN5gbc1gkERmuVBMyFLh27Rr27t2ru8mqh4cHevfujcaNG5dZgFJISUmBpaVlsRdsUeWz4VQY5u3+Bw4WKhz/qBuMlXKpQyIiiZXmmM48UHXl5mnRc+kJhMalY1KPOpjUo67UIRFRJWDocb1UZ44KNG7cuMoXQlS9vNnGHev+CsODpAz839kIvNupttQhERFRBVLIZZjycl2M//UyfvgrDCPaecLa1EjqsIioiuA8l1StqBRyfPhSHQDAqsAQpGXlShwRERFVtN4NneDnZIG0rFysOR4idThEVIWwOKJqZ0BzF9S2NUVCejZ+PBkmdThERFTBZDIB0/zzh9P9dCYcMSmZEkdERFUFiyOqdhRyGSa/nJ8U150IRZImW+KIiIioonWrZ48WHtbIzNFixdFn3+uQiOhJLI6oWurTyAm+ThZIzcrFmuOhUodDREQVTBAEfORfDwDw2/lI3EvQSBwREVUFLI6oWpLJBEzrmX/2aMPpMMRySAURUY3TtrYNOtWxRU6eiKWH70odDhFVAeVSHCUmJuKzzz4rj00TGax7fXs0d7dCZo4WK49xSAURUU00rWf+2aPtl+8jODZV4miIqLIrcXEkiiJiYmKQlZVVaNn9+/cxZcoUeHh4YP78+WUSIFFpCYKAaY+HVPx6jkMqiIhqoiZuVujp5wCtCHxz6I7U4RBRJWdwcSSKIj799FNYW1vD2dkZpqameO2115CQkACNRoNJkyahTp06WLZsGbp06YJjx46VZ9xEBmnvbYuOPvlDKr47wiEVREQ10dSe9SAIwN7r0bjxIFnqcIioEjO4OPruu+/wxRdfwNraGgMHDkSzZs2we/dujBo1Cj179sTKlSsxdOhQ3LhxA7t370bnzp3LM24igxWcPdp66T6CY9MkjoaIiCpaPUdzvNbEGQCw+GCQxNEQUWVmcHH0448/onXr1rh9+za2bNmC8+fPY9q0adi5cyciIyNx6dIlrF+/Hr6+vuUZL1GJNXWzwsuPh1R8e5hDKoiIaqJJPepCIRMQGBSH8+EJUodDRJWUwcXR3bt3MWzYMKhUKl3bu+++CwCYPXs2GjVqVPbREZWRqT3rQhCAPdeiOKSCiKgG8rQ1xaCWbgCARQeCIIqixBERUWVkcHGUmZkJW1tbvTYbGxsAgLe3d9lGRVTG6jtaoN/jIRVfMykSEdVIE1/ygZFChnNhCQgMipM6HCKqhEo0W50gCEW2y+XyMgmGqDxNebkulHIBJ+7E4VhQrNThEBFRBXOyNMHI9p4AgM/3/IPsXK20ARFRpSOIBn6FLpPJ4ObmBktLS11bXl4ebt26BS8vL5iamupvWBBw9erVso22gqSkpMDS0hLJycmwsLCQOhwqQwv23cL3x0PhZWuKA5M6w0jB+yATVXelOaYzD1RfKZk56L44EPFp2fikjy/e7VRb6pCIqAIYelxXGLrBzp07F3nmyN7evnQREklgfDcfbL34AGHx6fjpdDhGd2ZSJCKqSSyMlfjIvx4+3nody47cxevNXGBrpip+RSKqEQwujgIDA8sxDKKKYW6sxPRX6mH6H9fw3eOkaGfOpEhEVJO80cING89G4MaDFCw5eAcLBnBSKSLKxzFFVOO80dwVjV0tkZqViyW83wURUY0jlwmY27cBAOC385G4+ZCzmBJRvhIVR2vXroWvry+MjY3h4uKCyZMnIysrq7xiIyoXsieS4uYL93D9PpMiEVFN08qzFvo1cYYoAvN3/cNZTIkIQAmKox07dmDs2LG4f/8+GjduDK1Wi++++w5jx44tz/iIykULD2u83jQ/Kc7ecR15WiZFIqKaZkav+jBWynAuPAF/XLwvdThEVAkYXBx988038Pb2RnBwMM6dO4d79+5h8ODB+OWXX5CSklKeMRKVi1l9fGFurMC1+8n4+Uy41OEQEVEFc7YywaQedQEAX+69hYT0bIkjIiKpGVwcBQUFYcyYMXBwcAAAKBQKzJw5E7m5ubh161a5BUhUXuzNjTGjV30AwOIDQXiYlCFxREREVNFGdfRCfUdzJGpy8N89/0gdDhFJzODiKC4uDs7OznptLi4uAACNRlO2URFVkDdbuaOFhzXSs/Mwb9dNqcMhIqIKppTLsGBAIwgCsO3SA5wOjpc6JCKSUIkmZCjqPkdEVZlMJuDL/o2gkAk4+E8MDtyMljokIiKqYM3crfF2Ww8AwOwdN5CZkydxREQkFYPvcwQAixcvxqZNm3TPc3JyAACzZ8+Gra2tXl9BELBz584yCJGofNVzNMd7nWtjVWAIPt1xA228asFKbSR1WEREVIGm+dfD/hvRCItPx7eH72BmL1+pQyIiCRhcHLm7uyMhIQEJCQl67R4eHoiKikJUVJReO88yUVUy8aU62H8zGqFx6Ziz8ya+e7OZ1CEREVEFsjBW4r+vN8R7Gy9i7YlQ9PB1QCvPWlKHRUQVzODiKDw8vBzDIJKWsVKObwY3xcDVp7Hr6kP0bOCAVxs7F78iERFVGz0bOGJgc1dsvXQfU7dcxb4PO8FUVaJBNkRUxZXomiOi6qypmxU+6OoNAPhkxw3EpmRKHBEREVW0uf384GxpjMgEDb7cy9l4iWoag4ujWbNm4dq1a+UZC5Hkxnevg4YuFkjS5GD61mvQ8uawREQ1ioWxEosHNQEA/PJ3JI7djpU4IiKqSAYXR1999RVu3Lihe/7o0SPI5XIcPXq0XAIjkoKRQoZvBjeFkUKGwKA4rP0rVOqQiIiogrX3scXIDp4AgClbrvA+eEQ1yAsNqxNFfqtO1U9dB3PM69sAALDoQBDOhycUswYREVU3M3rVR0MXCyRqcjBh02Xk5GmlDomIKgCvOSIqwput3fB6U2fkaUWM//USHqVlSR0SERFVIJVCjlXDWsDcWIGLEYlYdCBI6pCIqAKwOCIqgiAI+KJ/I3jbmSImJQuTNl9BLr81JCKqUdxt1Fj0Rv71R2tPhPJG4UQ1QInmpwwPD8elS5cAAMnJyQCAu3fvwsrKqsj+zZs3f7HoiCRkqlJg9X9a4LUVp/DX3Xh8sfcW5j4ebkdERDXDKw0dMaqjF/53MgyTN1/BH2Pbw8/ZQuqwiKicCKKBFw7JZLJCN3YVRbHIm70WtOfl5ZVNlBUsJSUFlpaWSE5OhoUFD4A13b7rURj3S/6XAl/0b4i32nhIHBERlURpjunMA/SknDwtAtafw6ngR3C2NMaO8R1gb24sdVhEVAKGHtcNPnO0fv36MgmMqKrp1cgJ03rWxeKDdzB350142piig4+t1GEREVEFUcplWDWsBfqvOoXQ+HSM2XgRm0a3hbFSLnVoRFTGDD5zVJPwG0N6miiKmLT5CnZeeQgzlQKbRrdFI1dLqcMiIgPwzBGVldC4NLy+8hRSMnPRw9ceq//TAko5L98mqgoMPa7zfzSRAQRBwMKBjdGutg3SsnIxYv05hMSlSR0WERFVoNp2Zlg3vCVUChkO34rFx3/wZuFE1Q2LIyIDGSvlWDu8BRq5WCIhPRtv//A37idqpA6LiIgqUJvaNlg5rDnkMgHbLj/AZ3/+w/s+ElUjLI6ISsDcWIkNI1uhtp0pHiZnYsj3ZxEeny51WEREVIF6+Dlg8aDGAIANp8MxZ+dNnkEiqiZYHBGVkI2ZCr++2xa17UzxICkDg78/g+DYVKnDIiKiCtS/mSsWDmwEQQA2no3AjG3XkMcCiajKY3FEVAqOlsbY/F471HMwR2xqFgZ/fxaXIhOlDouIiCrQkFbu+GZwE8gEYMuF+5iw6RIyc6rmbUyIKB+LI6JSsjNXYdN7bXXXIL259iz2XIuSOiwiIqpA/Zu5YsWw5lDKBey9Ho2ha88iPi1L6rCIqJRYHBG9gFqmRvjtvbZ4qb49snK1+ODXS1hx9C7HnhMR1SC9Gzlh46g2sDRR4sq9JLy+8hT+eZgidVhEVAosjohekKlKgbXDWyKgvScAYPHBOxj103kkpmdLGxgREVWYtrVtsO399vCwUeN+Ygb6rzqFzecjOZMdURXD4oioDMhlAub1a4CFAxtBpZDhWFAc+nz3F86FJUgdGhERVRBvOzPseL8DutazQ1auFh9vvY6pW64iJTNH6tCIyEAsjojK0JBW7tj+fgd42qjzp/peewbzdt2EJjtX6tCIiKgCWJsa4ccRrfCRfz3IBGDb5Qfw//YEjgXFSh0aERmAxRFRGfNztsDuCR0xtJUbRDH/HhivLP0Lx+/ESR0aERFVAJlMwAfdfLB5TDt42qgRlZyJkevPY+qWq4hL5WQNRJUZiyOicmBurMRXAxvjp3daw9nSGJEJGoz48RxGrj+H4Ng0qcMjIqIK0MqzFvZ92BmjOnpBEICtl+6j2+JArDkegqxcTvlNVBkJIq8ULCQlJQWWlpZITk6GhYWF1OFQFZeamYOlh+/ip9PhyNWKUMgEDGrphve7esOtllrq8IiqvdIc05kHqKxdjEjE/N03ce1+MgDArZYJxnfzQf9mrjBS8LtqovJm6HGdxVERmBSpPITGpeHLvbdw+Fb+uHOFTED/Zi4Y06U2fOzNJY6OqPpicUSVhVYrYtvlB/h6/23EPh5e52JlgrFdvTGwuQvURgqJIySqvlgcvQAmRSpP58ISsPzoXfx1N17X1t7bBm+39UAPPwco5fwGkagssTiiykaTnYtfzkbi+xOhuhvGmhsr8EYLV7zd1gO17cwkjpCo+mFx9AKYFKkiXIxIxJrjIThyKwYF94y1M1ehTyMn9G3ijObuVhAEQdogiaoBFkdUWWXm5GHTuUisPxWOyASNrr2lhzX6NXVGr4ZOsDNXSRghUfXB4ugFMClSRXqQlIFNf0fit/ORiE/798axrtYm6OHrgC717NCutg2MlXIJoySqulgcUWWn1Yo4cTcO/3c2Akdux6LgLzOZkH9z2W717NG1nh187M34pRlRKbE4egFMiiSF7FwtTgbHYdeVhzj4Tww02f/OZKRSyNCmtg1aeVijhac1mrpZcWw6kYFYHFFVEp2ciT+vPcTuqw9x9fHkDQVcrEzQ3tsGrTxroYWnNWrbmrJYIjIQi6MXwKRIUsvIzsOJu3EIDIrD8aBYPEzO1FuukAnwdbKAn5MF6juZo76jBXydzGGlNpIoYqLKi8URVVURj9Jx5FYsjgXF4u+wBGTnavWW1zI1QiMXS/g65eeA+o4WqG1nymtXiYpQrYqjlStXYtGiRYiOjkaTJk2wfPlytG7d+pn9f//9d3z66acIDw9HnTp1sHDhQvTu3dvg12NSpMpEFEXciUnDmZB4XIhIxMWIREQ9VSwVsDVTwcNGDY9aarjbqOFho4Z7LTXszY1hb6GCSsGheVTzsDii6kCTnYu/wxJwPiwBFyIScfVeErKeKpYAQCkX4Gb9OAfUUsPdxhQetdRwsTaBvbkK1mojyGQ820Q1T7UpjjZv3ozhw4djzZo1aNOmDZYuXYrff/8dQUFBsLe3L9T/9OnT6Ny5MxYsWIBXX30Vv/76KxYuXIhLly6hYcOGBr0mkyJVdg+SMnD1XhJuR6fiVlQKbken4F5CRrHrWZooYW+ugr2FCvbmxrBSK2FhrISlyRMPdf6/5sYKmCjlMDGSw0gu49ANqrJYHFF1lJ2rxc2HyfgnKgW3owpyQSrSsnKfu55SLsDWTAV7cxXszI1hZ24EC5On8sDjh4WxEmqVHGqj/HwgZ1FFVVi1KY7atGmDVq1aYcWKFQAArVYLNzc3TJgwATNmzCjUf8iQIUhPT8eff/6pa2vbti2aNm2KNWvWGPSaTIpUFaVm5iA8XoOIhHREPNLgXoIm/99EDWJTswoNxygJmQCojRQwVsqhNpLriiZjpQxKecFDgEIug5FcBoVMgFIhg1Im5C974meZTIBMECCXATJBgCAIkAuATFbwswCZkL8sv29BP0D+eN1/2/ITdUG6LqjfdP9C98NT/Z6x3uMW4an++tsouo9um0WtW4lVlZrXy9YMtUxLN2yUxRHVFKIo4kFSBiIfaRBRkAMS8vNCVFImHqVnF7+R5zBSyGDyVB4wUcphpPg3DxTkBIVcyM8HT7QVLFfIhH9zwePje35eyD++C0//LAiQ6eWMf5cV/AtUQC6opnkAqBq5QCmXobGrVanXN/S4Xqmv6M7OzsbFixcxc+ZMXZtMJkOPHj1w5syZItc5c+YMpkyZotfm7++PHTt2PPN1srKykJWVpXuekpICAMjJyUFOTs4L7AFRxTGWA/Ud1KjvoC60TBRFpGTmIjYlC7FpWYhLzUJcWhZSMnKRnJGT/29mDlIycpCckYuUzBykZeUiJy//uxOtCKRl5Rb7jSRRefluSGP0auhYqnUNOY4zD1B14WCmhIOZJVp5WBZalpOnRXxadn4OSM3PB4/SspGSWZALcpCSmZufCx7/q8nJ082el52rRXauFskZ/D9BFc/OzAinP+5a6vUNPZZX6uIoPj4eeXl5cHBw0Gt3cHDA7du3i1wnOjq6yP7R0dHPfJ0FCxZg/vz5hdoPHjwItbrwH5pEVZ0RAJfHDygAmD9+PCVPC2QXPPKe+FkrIDsPyNECeeITj8fPc3U/C3rLckVARH6xJQIQxSd+Lmgvqg/+bRdFQfccj9d70jPbn9Hv6eXiUw1FnVo3dFuVXVWJEwBuXr0MMbJ0EWs0mmL7MA9QTWT5+AEAMH78eIoo5h/rc7RA1uN/dfkgT0CWtojj/xPP80QBuSKg1T6xTHzymP70MV7/edG5QNB7DlR8LqgueQCoOrGaIgt79+4t9fqG5AKgkhdHFWXmzJl6Z5tSUlLg5uaGnj17cjgFEVEVV3AW6HmYB4iIqjdDcgFQyYsjW1tbyOVyxMTE6LXHxMTA0bHo4RWOjo4l6g8AKpUKKlXhO1ArlUoolcpSRE5ERJWFIcdx5gEiourN0GN5pZ4I38jICC1atMCRI0d0bVqtFkeOHEG7du2KXKddu3Z6/QHg0KFDz+xPREREREQEVPIzRwAwZcoUjBgxAi1btkTr1q2xdOlSpKenY+TIkQCA4cOHw8XFBQsWLAAAfPjhh+jSpQuWLFmCPn364LfffsOFCxewdu1aKXeDiIiIiIgquUpfHA0ZMgRxcXGYM2cOoqOj0bRpU+zfv1836UJkZCRksn9PgLVv3x6//vorPvnkE8yaNQt16tTBjh07DL7HEZA/sxdg+NhEIiKqvAqO5SW5cwXzABFR9WJoLqj09zmSwv379+Hm5iZ1GEREVIbu3bsHV1dXg/oyDxARVU/F5QIWR0XQarV4+PAhzM3NdTf0IiKiqkkURaSmpsLZ2VlvpMHzMA8QEVUvhuYCFkdERERERESo5LPVERERERERVRQWR0RERERERGBxREREREREBIDFEREREREREQAWR0SVVmBgIARBQFJSUoW+7oYNG2BlZfVC2wgPD4cgCLhy5coz+0i1f0REVQlzAVHFYnFEJAFBEJ77mDdvntQhEhFROWMuIKp8FFIHQFQTRUVF6X7evHkz5syZg6CgIF2bmZkZLly4UOLtZmdnw8jIqExiJCKi8sVcQFT58MwRkQQcHR11D0tLSwiCoNdmZmam63vx4kW0bNkSarUa7du310uc8+bNQ9OmTfHDDz/Ay8sLxsbGAICkpCS8++67sLOzg4WFBbp3746rV6/q1rt69Sq6desGc3NzWFhYoEWLFoUS8IEDB+Dr6wszMzO88soreklcq9Xis88+g6urK1QqFZo2bYr9+/c/d5/37t2LunXrwsTEBN26dUN4ePiLvIVERFUecwFzAVU+LI6IKrnZs2djyZIluHDhAhQKBd555x295cHBwdi6dSu2bdumG9c9aNAgxMbGYt++fbh48SKaN2+Ol156CQkJCQCAt956C66urjh//jwuXryIGTNmQKlU6rap0WiwePFibNy4ESdOnEBkZCSmTZumW75s2TIsWbIEixcvxrVr1+Dv749+/frh7t27Re7DvXv3MGDAAPTt2xdXrlzBu+++ixkzZpTxO0VEVH0xFxBVEJGIJLV+/XrR0tKyUPuxY8dEAOLhw4d1bXv27BEBiBkZGaIoiuLcuXNFpVIpxsbG6vr89ddfooWFhZiZmam3PW9vb/H7778XRVEUzc3NxQ0bNjwzHgBicHCwrm3lypWig4OD7rmzs7P4xRdf6K3XqlUr8f333xdFURTDwsJEAOLly5dFURTFmTNnin5+fnr9P/74YxGAmJiYWGQcREQ1CXNBYpFxEFU0njkiquQaN26s+9nJyQkAEBsbq2vz8PCAnZ2d7vnVq1eRlpYGGxsbmJmZ6R5hYWEICQkBAEyZMgXvvvsuevToga+++krXXkCtVsPb21vvdQteMyUlBQ8fPkSHDh301unQoQNu3bpV5D7cunULbdq00Wtr166dwe8BEVFNx1xAVDE4IQNRJffkEAdBEADkj/MuYGpqqtc/LS0NTk5OCAwMLLStgmlZ582bh2HDhmHPnj3Yt28f5s6di99++w39+/cv9JoFryuKYlnsDhERlQJzAVHF4JkjomqmefPmiI6OhkKhgI+Pj97D1tZW169u3bqYPHkyDh48iAEDBmD9+vUGbd/CwgLOzs44deqUXvupU6fg5+dX5Dq+vr44d+6cXtvZs2dLuGdERGQo5gKi0mFxRFTN9OjRA+3atcPrr7+OgwcPIjw8HKdPn8bs2bNx4cIFZGRkYPz48QgMDERERAROnTqF8+fPw9fX1+DX+Oijj7Bw4UJs3rwZQUFBmDFjBq5cuYIPP/ywyP5jx47F3bt38dFHHyEoKAi//vorNmzYUEZ7TERET2MuICodDqsjqmYEQcDevXsxe/ZsjBw5EnFxcXB0dETnzp3h4OAAuVyOR48eYfjw4YiJiYGtrS0GDBiA+fPnG/waEydORHJyMqZOnYrY2Fj4+flh165dqFOnTpH93d3dsXXrVkyePBnLly9H69at8eWXXxaabYmIiMoGcwFR6QgiB48SERERERFxWB0RERERERHA4oiIiIiIiAgAiyMiIiIiIiIALI6IiIiIiIgAsDgiIiIiIiICwOKIiIiIiIgIAIsjIiIiIiIiACyOiIiIiIiIALA4IiIiIiIiAsDiiIiIiIiICACLIyIiIiIiIgDA/wNDastb/9BDfwAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "thresholds = torch.linspace(0, 1, 1001)\n", + "\n", + "# fpr and tpr as a function of the threshold (i.e. the survival functions)\n", + "# generaly look like logistic functions flipped horizontally\n", + "# their actual shapes don't matter much, but rather how they compare to each other\n", + "# in this case, since they're the same, this choice is arbitrary as long as\n", + "# they're monotonically decreasing with the threshold\n", + "fpr = 1 - 1e2 / (1e2 + torch.exp(-20 * (thresholds - 0.5)))\n", + "tpr = fpr.clone()\n", + "\n", + "fig, axes = plt.subplots(1, 2, figsize=(8, 2), constrained_layout=True, sharey=True)\n", + "\n", + "axes[0].plot(thresholds, fpr, label=\"FPR\")\n", + "axes[1].plot(thresholds, tpr, label=\"TPR\")\n", + "\n", + "for ax in axes:\n", + " ax.set_xlabel(\"Threshold\")\n", + " ax.legend(loc=\"upper right\")\n", + " ax.set_yticks([0, 0.5, 1])\n", + " ax.set_xticks([])\n", + " ax.grid()\n", + "\n", + "fig.supylabel(\"FPR or TPR\", x=-0.03)\n", + "fig.suptitle(\"Simulated FPR and TPR curves\")\n", + "fig # noqa: B018, RUF100" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PIMO curve\n", + "\n", + "In the ROC curve, the FPR = TPR looks like a straight line.\n", + "\n", + "What does it look like in the PIMO curve?" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhwAAAI4CAYAAAAhwzBcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAACH+0lEQVR4nO3dd3gUVdsG8Hs32fQeSE8g1BAghN4FpUMQBAUVKYqUj/KCCEgiiChvEERBEBWwAEpRiiAISOgqoYVuSAFCCATSe91k5/sjZl42u5tswm425f5dlxfumZkzz5yd2Tx7zplZiSAIAoiIiIj0SGroAIiIiKjuY8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBpAOTJk1C48aNDR1GjXH//n1IJBJs2bKl0tuePn0aEokEp0+f1nlcRGQ4TDioVtmyZQskEon4n7GxMdzd3TFp0iQ8evTI0OGRnkyaNAl9+/bVSV1Pnz9SqRRubm4YOHCgSoLTuHFjBAQEqN327bffVlv3+++/L66TnJyssvzQoUMYPHgwHB0dYWZmhhYtWmD+/PlISUnRybER1WRMOKhW+uijj/Djjz/im2++wZAhQ/DTTz+hT58+yM/PN3RoVAsMGDAAP/74I7Zu3Yrp06fjxo0beOGFF3DkyJEKtzUzM8PevXtRWFiosmznzp0wMzNTu938+fMxfPhwPHnyBO+99x6+/PJL9O/fH19++SXatWuHyMjIZz4uoprM2NABEFXFkCFD0KlTJwDA22+/jQYNGmDlypX47bffMGbMGANHRzVdixYt8MYbb4ivX3rpJfj5+WHt2rUYMmRIudsOHjwYv/32G44cOYIRI0aI5efOnUNMTAxGjx6NvXv3Km2zc+dOfPbZZxg7diy2b98OIyMjcdmkSZPw/PPP45VXXsGVK1dgbGy4j2VBEJCfnw9zc3ODxUB1F3s4qE7o3bs3AODu3btiWWFhIT744AN07NgRtra2sLS0RO/evXHq1CmlbUvnG6xevRqbNm1C06ZNYWpqis6dO+PSpUsq+9q/fz/atGkDMzMztGnTBr/++qvamHJycvDuu+/C09MTpqamaNmyJVavXo2yP9AskUgwa9Ys7N69G76+vjA3N0f37t1x8+ZNAMDGjRvRrFkzmJmZoW/fvrh//36F7fHhhx9CIpEgKioKb7zxBmxtbdGwYUMsWbIEgiAgLi4OI0aMgI2NDVxcXPDZZ5+p1JGYmIjJkyfD2dkZZmZmaNeuHbZu3aqyXnp6OiZNmgRbW1vY2dlh4sSJSE9PVxtXREQEXn75ZTg4OMDMzAydOnXCb7/9VuHxqLNr1y507NgR1tbWsLGxQdu2bfHFF19Uqa62bduiQYMGiImJqXBdd3d3PPfcc9ixY4dS+fbt29G2bVu0adNGZZtly5bB3t4emzZtUko2AKBLly547733cPPmTezZs6fC/T969AiTJ0+Gm5sbTE1N4e3tjf/7v/8Te1xK3/uySocjnz5/SoeN/vjjD3Tq1Anm5ubYuHEj2rRpg+eff16lDoVCAXd3d7z88stKZWvXrkXr1q1hZmYGZ2dnTJs2DWlpaRUeC9UvTDioTij9ELW3txfLMjMz8e2336Jv375YuXIlPvzwQyQlJWHQoEG4du2aSh07duzAp59+imnTpmH58uW4f/8+Ro0aBblcLq5z7NgxjB49GhKJBCtWrMDIkSPx5ptv4vLly0p1CYKAF198EWvWrMHgwYPx+eefo2XLlliwYAHmzZunsu8///wT7777LiZOnIgPP/wQt2/fRkBAADZs2IB169ZhxowZWLBgAUJDQ/HWW29p3S5jx46FQqHAJ598gq5du2L58uVYu3YtBgwYAHd3d6xcuRLNmjXD/PnzcfbsWXG7vLw89O3bFz/++CPGjRuHTz/9FLa2tpg0aZLSH3VBEDBixAj8+OOPeOONN7B8+XI8fPgQEydOVInln3/+Qbdu3XD79m0sWrQIn332GSwtLTFy5EiNSZsmISEheO2112Bvb4+VK1fik08+Qd++ffH3339Xqp5SaWlpSEtLg6Ojo1brv/766zh48CCys7MBAEVFRdi9ezdef/11lXWjo6MRGRkpJnjqTJgwAUDJHI/yxMfHo0uXLti1axfGjh2LdevWYfz48Thz5gxyc3O1ir2syMhIvPbaaxgwYAC++OIL+Pv7Y+zYsTh79iyePHmitO5ff/2F+Ph4vPrqq2LZtGnTsGDBAvTs2RNffPEF3nzzTWzfvh2DBg1SunaIIBDVIj/88IMAQDh+/LiQlJQkxMXFCXv27BEaNmwomJqaCnFxceK6RUVFQkFBgdL2aWlpgrOzs/DWW2+JZTExMQIAwdHRUUhNTRXLDxw4IAAQDh48KJb5+/sLrq6uQnp6ulh27NgxAYDQqFEjsWz//v0CAGH58uVK+3/55ZcFiUQi3LlzRywDIJiamgoxMTFi2caNGwUAgouLi5CZmSmWBwYGCgCU1lVn6dKlAgBh6tSpSu3h4eEhSCQS4ZNPPlFqE3Nzc2HixIli2dq1awUAwk8//SSWFRYWCt27dxesrKzEmEqPc9WqVUr76d27twBA+OGHH8Tyfv36CW3bthXy8/PFMoVCIfTo0UNo3ry5WHbq1CkBgHDq1CmNxzdnzhzBxsZGKCoqKrcd1AEgTJ48WUhKShISExOFCxcuCP369RMACJ999pm4XqNGjYRhw4apbDtz5kwhNTVVMDExEX788UdBEATh999/FyQSiXD//n2x7ZOSkpTaaM2aNeXGZWNjI3To0KHcdSZMmCBIpVLh0qVLKssUCoUgCP9778sqvXaePncaNWokABCOHj2qtG5kZKQAQFi/fr1S+YwZMwQrKyshNzdXEARB+PPPPwUAwvbt25XWO3r0qNpyqt/Yw0G1Uv/+/dGwYUN4enri5ZdfhqWlJX777Td4eHiI6xgZGcHExARASbdvamoqioqK0KlTJ1y5ckWlzrFjxyr1kJQO09y7dw8A8PjxY1y7dg0TJ06Era2tuN6AAQPg6+urVNfhw4dhZGSE//znP0rl7777LgRBUJmc2K9fP6Xbart27QoAGD16NKytrVXKS2OqyNN3UxgZGaFTp04QBAGTJ08Wy+3s7NCyZUulOg8fPgwXFxe89tprYplMJsN//vMfZGdn48yZM+J6xsbG+L//+z+l/cyePVspjtTUVJw8eRJjxoxBVlYWkpOTkZycjJSUFAwaNAjR0dGVusvIzs4OOTk5CAkJ0Xqbp3333Xdo2LAhnJyc0LVrV/z999+YN28e5s6dq9X29vb2GDx4MHbu3AmgpHesR48eaNSokcq6WVlZAKD0PqpjbW2NzMxMjcsVCgX279+P4cOHi/OXnqZuGEUb3t7eGDRokFJZixYt4O/vj59//lksKy4uxp49ezB8+HBxjsfu3btha2uLAQMGiO9pcnIyOnbsCCsrK5XhS6rfOGmUaqUNGzagRYsWyMjIwPfff4+zZ8/C1NRUZb2tW7fis88+Q0REhFL3rre3t8q6Xl5eSq9Lk4/SsejY2FgAQPPmzVW2bdmypVISExsbCzc3N5U/Mq1atVKqS9O+SxMaT09PteXajo+rq9fMzAwNGjRQKX/61szY2Fg0b94cUqnyd5Ky8cfGxsLV1RVWVlZK67Vs2VLp9Z07dyAIApYsWYIlS5aojTUxMRHu7u5aHdeMGTPwyy+/YMiQIXB3d8fAgQMxZswYDB48WKvtR4wYgVmzZkEikcDa2hqtW7eGpaWlVtuWev311zF+/Hg8ePAA+/fvx6pVq9SuV3oOlCYemmRlZcHJyUnj8qSkJGRmZqqdI/Is1F0LQEkCHhQUhEePHsHd3R2nT59GYmIixo4dK64THR2NjIwMjXEnJibqNFaq3ZhwUK3UpUsX8VveyJEj0atXL7z++uuIjIwU//j99NNPmDRpEkaOHIkFCxbAyckJRkZGWLFihdLk0lJlJ/OVEspM8tQHTft+1pjUbW+I41QoFABKbg0t+226VLNmzbSuz8nJCdeuXcMff/yBI0eO4MiRI/jhhx8wYcIEtRNby/Lw8ED//v213p86L774IkxNTTFx4kQUFBRovDuqNEm7ceOGxrpiY2ORmZmp0lNWFZp6OoqLi9WWa7ojZezYsQgMDMTu3bsxd+5c/PLLL7C1tVVK6hQKBZycnLB9+3a1dTRs2LCS0VNdxoSDar3SJOL555/Hl19+iUWLFgEA9uzZgyZNmmDfvn1KH8JLly6t0n5Ku8ujo6NVlpV9hkKjRo1w/PhxZGVlKfVyREREKNVVUzVq1Ag3btyAQqFQ6uUoG3+jRo1w4sQJZGdnK/VylG2PJk2aACgZlnnWP/SlTExMMHz4cAwfPhwKhQIzZszAxo0bsWTJkkolL1Vlbm6OkSNH4qeffsKQIUNUeo1KtWjRAi1atMD+/fvxxRdfqB1a2bZtGwCoPGjsaQ0bNoSNjQ1u3bpVblylPXPp6emws7MTy8v2qlXE29sbXbp0wc8//4xZs2Zh3759GDlypFJPYtOmTXH8+HH07NmTt9JShTiHg+qEvn37okuXLli7dq348K/Sb/JPf3O/cOECQkNDq7QPV1dX+Pv7Y+vWrcjIyBDLQ0JCEB4errTu0KFDUVxcjC+//FKpfM2aNZBIJBU+68HQhg4diidPniiN4RcVFWH9+vWwsrJCnz59xPWKiorw9ddfi+sVFxdj/fr1SvU5OTmhb9++2LhxIx4/fqyyv6SkpErFV/bJnFKpFH5+fgCAgoKCStX1LObPn4+lS5dqHCYq9cEHHyAtLQ3Tp09X6WkICwvDypUr0aZNG4wePVpjHVKpFCNHjsTBgwdV7ooC/neeN23aFACU7jrKycnRquenrLFjx+L8+fP4/vvvkZycrDScAgBjxoxBcXExPv74Y5Vti4qKNN4eTfUTeziozliwYAFeeeUVbNmyBdOnT0dAQAD27duHl156CcOGDUNMTAy++eYb+Pr6irczVtaKFSswbNgw9OrVC2+99RZSU1Oxfv16tG7dWqnO4cOH4/nnn8f777+P+/fvo127djh27BgOHDiAuXPnin8UaqqpU6di48aNmDRpEsLCwtC4cWPs2bMHf//9N9auXSt+Sx8+fDh69uyJRYsW4f79+/D19cW+ffuUErJSGzZsQK9evdC2bVtMmTIFTZo0QUJCAkJDQ/Hw4UNcv35d6/jefvttpKam4oUXXoCHhwdiY2Oxfv16+Pv7i0MY1aFdu3Zo165dheuNGzcOly5dwhdffIHw8HCMGzcO9vb2uHLlCr7//ns4Ojpiz549kMlk5dYTHByMY8eOoU+fPpg6dSpatWqFx48fY/fu3fjrr79gZ2eHgQMHwsvLC5MnT8aCBQtgZGSE77//Hg0bNsSDBw8qdXxjxozB/PnzMX/+fDg4OKj0TvXp0wfTpk3DihUrcO3aNQwcOBAymQzR0dHYvXs3vvjiC6VndlA9Z7gbZIgqr/TWPnW3BRYXFwtNmzYVmjZtKhQVFQkKhUIIDg4WGjVqJJiamgrt27cXDh06JEycOFHpFtbS22I//fRTlToBCEuXLlUq27t3r9CqVSvB1NRU8PX1Ffbt26dSpyAIQlZWlvDOO+8Ibm5ugkwmE5o3by58+umn4u2LT+9j5syZSmWaYiq9ZXT37t3ltlPZWzNLTZw4UbC0tFRZv0+fPkLr1q2VyhISEoQ333xTaNCggWBiYiK0bdtW6TbXUikpKcL48eMFGxsbwdbWVhg/frxw9epVldtiBUEQ7t69K0yYMEFwcXERZDKZ4O7uLgQEBAh79uxROcbybovds2ePMHDgQMHJyUkwMTERvLy8hGnTpgmPHz8ut10EQX17q1PebbHl0dT2glByi+yAAQMEe3t7wdTUVGjWrJnw7rvvql1Xk9jYWGHChAnireBNmjQRZs6cqXQLeFhYmNC1a1exbT7//HONt8WWPcayevbsKQAQ3n77bY3rbNq0SejYsaNgbm4uWFtbC23bthUWLlwoxMfHa31cVPdJBKEaZsQRERFRvcY5HERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBREREemds6ABqAoVCgfj4eFhbW0MikRg6HCIiolpDEARkZWXBzc0NUqnmfgwmHADi4+Ph6elp6DCIiIhqrbi4OHh4eGhczoQDgLW1NYCSxrKxsVFZLpfLcezYMQwcOBAymay6w6tV5KlyHAs9hoHpAyEzZVtpIi+Q45jdMQzsPhAyB7ZTeXj9aY9tpR22k/a0aavMzEx4enqKf0s1YcIBiMMoNjY2GhMOCwsL2NjY8OSsgNhWEhvILNlWmshz5LAwt4CNtQ1kNmyn8vD60x7bSjtsJ+1Vpq0qmpLASaNERESkd0w4iIiISO+YcBAREZHeGTThOHv2LIYPHw43NzdIJBLs379fabkgCPjggw/g6uoKc3Nz9O/fH9HR0UrrpKamYty4cbCxsYGdnR0mT56M7OzsajwKIiIiqohBE46cnBy0a9cOGzZsULt81apVWLduHb755htcuHABlpaWGDRoEPLz88V1xo0bh3/++QchISE4dOgQzp49i6lTp1bXIRAREZEWDHqXypAhQzBkyBC1ywRBwNq1a7F48WKMGDECALBt2zY4Oztj//79ePXVV3H79m0cPXoUly5dQqdOnQAA69evx9ChQ7F69Wq4ublV27EQERHVFem5hbgalw5Hc2NkywGFQnjmOmvsbbExMTF48uQJ+vfvL5bZ2tqia9euCA0NxauvvorQ0FDY2dmJyQYA9O/fH1KpFBcuXMBLL72ktu6CggIUFBSIrzMzMwGU3P4jl8tV1i8tU7eMlMmL/m0ryIFnPz/rLDn+baciOcDTqly8/rTHttIO26li1x6k4s0tYf++MsbSK8cxvpsXgoa0VFlX23assQnHkydPAADOzs5K5c7OzuKyJ0+ewMnJSWm5sbExHBwcxHXUWbFiBZYtW6ZSfuzYMVhYWGjcLiQkROv467sQ8xAmHOUxL/kn5DzPKW3x+tMe20o7bCfNLiVJABiJr4sUAh7GxuDw4bsq6+bm5mpVZ41NOPQpMDAQ8+bNE1+XPiVt4MCBGh/8FRISggEDBvAhMRWQp8oRcj4EA/IGQGbBttJEnitHiHkIBnQbwCeNVoDXn/bYVtphO1Us7mwMcEf5Jo3u/r4Y2s1LZd3SUYKK1NiEw8XFBQCQkJAAV1dXsTwhIQH+/v7iOomJiUrbFRUVITU1VdxeHVNTU5iamqqUy2Syck++ipYTxDNKBhlkErZVRWTGPKe0xetPe2wr7bCdNEvOUR0mcbO3UNte2rZhjX0Oh7e3N1xcXHDixAmxLDMzExcuXED37t0BAN27d0d6ejrCwsLEdU6ePAmFQoGuXbtWe8xERER1QWJWvkqZk43ZM9Vp0B6O7Oxs3LlzR3wdExODa9euwcHBAV5eXpg7dy6WL1+O5s2bw9vbG0uWLIGbmxtGjhwJAGjVqhUGDx6MKVOm4JtvvoFcLsesWbPw6quv8g4VIiKiKkrILFApc67NCcfly5fx/PPPi69L51VMnDgRW7ZswcKFC5GTk4OpU6ciPT0dvXr1wtGjR2Fm9r+D3r59O2bNmoV+/fpBKpVi9OjRWLduXbUfCxERUV2RkKnaw9HQSnUqQmUYNOHo27cvBEHzrQwSiQQfffQRPvroI43rODg4YMeOHfoIj4iIqN4RBAGJWco9HPYWMpgYP9ssjBo7h4OIiIiqX0aeHIVFCqUyZ+tn690AmHAQERHRU9TN33CyYcJBREREOqRu/oaT9bNNGAWYcBAREdFT1Ccc7OEgIiIiHSo7YRQAnKxNnrleJhxEREQkSuSQChEREekbJ40SERGR3iWoe6w553AQERGRLiWW6eGQQEADK87hICIiIh0pecqocg+HpQyQGT17usCEg4iIiAAAablyyIuVf3LEVrtfn68QEw4iIiICoP4ZHLYmmn/zrDKYcBAREREATQmHbupmwkFEREQAVCeMAoANEw4iIiLSJQ6pEBERkd6pewaHDSeNEhERkS6pG1JhDwcRERHpVIKaH27jpFEiIiLSqbI/3CaVAFYcUiEiIiJdUSgElZ+mb2BlCiOJbupnwkFERERIySlEsUJ5vkZDax2Np4AJBxEREUH9LbG6+JXYUkw4iIiICI8zVBMOV1szndXPhIOIiIjwJCNPpczFhgkHERER6VC8mh4OJhxERESkU0/UJRy2nMNBREREOvSYQypERESkb2p7OJhwEBERka4IgqByl4qdhQzmJkY62wcTDiIionouPVeOgiKFUpkuezcAJhxERET1nr6fwQEw4SAiIqr3nmSqmTBqa67TfTDhICIiqufYw0FERER6p/4ZHEw4iIiISIfYw0FERER6p66HgwkHERER6ZTap4xy0igRERHpirqHflmbGsPK1Fin+2HCQUREVI9lFRQht7BYqczVTrfDKQATDiIionrtcbq6O1R0O5wCMOEgIiKq19TN33DV8WPNASYcRERE9Vp1PIMDYMJBRERUr1XHMzgAJhxERET1Gns4iIiISO8eZ6rr4eCkUSIiItKhJ2of+sUeDiIiItIRQRDwKE054bA0MYKNmW4f+gUw4SAiIqq3MvOKkFPmoV/u9uaQSCQ63xcTDiIionrqYXquSpmbne7nbwBMOIiIiOqteDVPGXVnwkFERES69CiNPRxERESkZ/FqnsHhYc+Eg4iIiHSo7B0qAHs4iIiISMcepasmHJzDQURERDpVNuEwkkrgZG2ql30x4SAiIqqH8uXFSMoqUCpzsTGDsZF+UoManXAUFxdjyZIl8Pb2hrm5OZo2bYqPP/4YgiCI6wiCgA8++ACurq4wNzdH//79ER0dbcCoiYiIaj51P9qmr+EUoIYnHCtXrsTXX3+NL7/8Erdv38bKlSuxatUqrF+/Xlxn1apVWLduHb755htcuHABlpaWGDRoEPLzVRuSiIiISqidv6GnO1QAQPcPS9ehc+fOYcSIERg2bBgAoHHjxti5cycuXrwIoKR3Y+3atVi8eDFGjBgBANi2bRucnZ2xf/9+vPrqqwaLnYiIqCarzgmjQA1POHr06IFNmzYhKioKLVq0wPXr1/HXX3/h888/BwDExMTgyZMn6N+/v7iNra0tunbtitDQUI0JR0FBAQoK/jdulZmZCQCQy+WQy+Uq65eWqVtGyuRF/7YV5IBQwcr1mBz/tlORHOBpVS5ef9pjW2mH7VQiLiVbpczZ2kSpXbRpK23bsUYnHIsWLUJmZiZ8fHxgZGSE4uJi/Pe//8W4ceMAAE+ePAEAODs7K23n7OwsLlNnxYoVWLZsmUr5sWPHYGFhoXG7kJCQqhxGvRRiHsKEozz/fokIOc9zSlu8/rTHttJOfW+ni3ekKDuz4lH0TRxOuqGybnltlZur+rRSdWp0wvHLL79g+/bt2LFjB1q3bo1r165h7ty5cHNzw8SJE6tcb2BgIObNmye+zszMhKenJwYOHAgbGxuV9eVyOUJCQjBgwADIZLIq77c+kKfKEXI+BAPyBkBmwbbSRJ4rR4h5CAZ0GwCZA9upPLz+tMe20g7bqcSuHy4DSalKZSP690YzJyvxtTZtVTpKUJEanXAsWLAAixYtEodG2rZti9jYWKxYsQITJ06Ei4sLACAhIQGurq7idgkJCfD399dYr6mpKUxNVe8zlslk5Z58FS0niGeUDDLIJGyrisiMeU5pi9ef9thW2qnv7aTuseaNGlpDJlNNDcprK23bsEbfpZKbmwupVDlEIyMjKBQKAIC3tzdcXFxw4sQJcXlmZiYuXLiA7t27V2usREREtYVCIeBxmV+KtbeQwcJEf/0QNbqHY/jw4fjvf/8LLy8vtG7dGlevXsXnn3+Ot956CwAgkUgwd+5cLF++HM2bN4e3tzeWLFkCNzc3jBw50rDBExER1VDJ2QUoLFYolenzllighicc69evx5IlSzBjxgwkJibCzc0N06ZNwwcffCCus3DhQuTk5GDq1KlIT09Hr169cPToUZiZmRkwciIioppL3S2xbrb1OOGwtrbG2rVrsXbtWo3rSCQSfPTRR/joo4+qLzAiIqJarLof+gXU8DkcREREpHvx1fzQL4AJBxERUb3zMI0JBxEREelZXKrqw7o87DU/+FIXmHAQERHVM3Fqejg8HdjDQURERDoiCAIepin3cFibGsPWXL8PQWPCQUREVI8kZRcgX678DA5PBwtIJBK97pcJBxERUT0Sl1r9wykAEw4iIqJ6Rd2EUU89TxgFmHAQERHVK2oTDgcmHERERKRDcWnqEg4OqRAREZEOqZ3DwSEVIiIi0iV1PRz6fugXwISDiIio3pAXK/A4I1+prKG1KcxNjPS+byYcRERE9cTj9HwUKwSlMk89/0psKSYcRERE9YT6CaP6H04BmHAQERHVG4Z6BgfAhIOIiKjeMNQtsQATDiIionpD/WPN2cNBREREOqS2h4NDKkRERKRLZedwGEklcLU1q5Z9M+EgIiKqB3ILi5CcXahU5mZnBmOj6kkFmHAQERHVAw/TDPNI81JMOIiIiOqB2BTV+Rte1TRhFGDCQUREVC/EpuSolDVytKy2/TPhICIiqgfuq0k4Gjuyh4OIiIh0SN2QCns4iIiISKfU9XA0Yg8HERER6UphkQKPytyl0tDaFJamxtUWAxMOIiKiOu5hWi7K/Cp9tc7fAJhwEBER1XmGnr8BMOEgIiKq8wx9hwrAhIOIiKjOYw8HERER6Z36Hg4mHERERKRDah9rziEVIiIi0pWiYoXKz9I7WJrA1lxWrXEw4SAiIqrD4tPzUVTmntjqfOBXKSYcREREdVhNmL8BMOEgIiKq09T/Six7OIiIiEiH7quZMMoeDiIiItIp9nAQERGR3sUkcw4HERER6VFRsQIPytwSa2NmDDuL6r0lFmDCQUREVGfFpeVBXqx8S2yThlaQSCTVHgsTDiIiojrqXlK2SlmThtU/nAIw4SAiIqqz7iWpzt9o2tDKAJEw4SAiIqqz7iWr6eFowB4OIiIi0qG7ano4mrCHg4iIiHSp7JCKRGKYZ3AATDiIiIjqpMx8OZKzC5TKPOzNYSYzMkg8TDiIiIjqIHUTRps0MMxwCsCEg4iIqE5Sd0usoe5QAZhwEBER1UlqezgM9AwOgAkHERFRnaT2llgmHERERKRLNemhXwATDiIiojpHoRBUfiXW0sQITtamBoqICQcREVGd8yg9DwVFCqUyQ/1oW6kan3A8evQIb7zxBhwdHWFubo62bdvi8uXL4nJBEPDBBx/A1dUV5ubm6N+/P6Kjow0YMRERkWHdS65ZE0aBGp5wpKWloWfPnpDJZDhy5AjCw8Px2Wefwd7eXlxn1apVWLduHb755htcuHABlpaWGDRoEPLz8w0YORERkeGo/ZVYAz6DAwCMDbr3CqxcuRKenp744YcfxDJvb2/x/wVBwNq1a7F48WKMGDECALBt2zY4Oztj//79ePXVV6s9ZiIiIkO7k1iz7lABanjC8dtvv2HQoEF45ZVXcObMGbi7u2PGjBmYMmUKACAmJgZPnjxB//79xW1sbW3RtWtXhIaGakw4CgoKUFDwv8e9ZmZmAgDkcjnkcrnK+qVl6paRMnnRv20FOSAYOJgaTI5/26lIDvC0KhevP+2xrbRTH9op8kmmSlkTR7NKH7M2baVtnRJBEGrsnwUzMzMAwLx58/DKK6/g0qVLmDNnDr755htMnDgR586dQ8+ePREfHw9XV1dxuzFjxkAikeDnn39WW++HH36IZcuWqZTv2LEDFhaG+VEbIiIiXRAEIOiyEXKL/jdBVAoBn3YthrEeJlLk5ubi9ddfR0ZGBmxsbDSuV6N7OBQKBTp16oTg4GAAQPv27XHr1i0x4aiqwMBAzJs3T3ydmZkJT09PDBw4UG1jyeVyhISEYMCAAZDJZFXeb30gT5Uj5HwIBuQNgMyCbaWJPFeOEPMQDOg2ADIHtlN5eP1pj22lnbreTinZBcg9f0aprHEDK7wY0LPSdWnTVqWjBBWp0QmHq6srfH19lcpatWqFvXv3AgBcXFwAAAkJCUo9HAkJCfD399dYr6mpKUxNVe9Flslk5Z58FS0niGeUDDLIJGyrisiMeU5pi9ef9thW2qmr7XQvNUOlrIWz9TMda3ltpW29NfoulZ49eyIyMlKpLCoqCo0aNQJQMoHUxcUFJ06cEJdnZmbiwoUL6N69e7XGSkREVBOomzDawtmwd6gAWvZwdOjQoVKVSiQS/Pbbb3B3d69SUKXeeecd9OjRA8HBwRgzZgwuXryITZs2YdOmTeJ+5s6di+XLl6N58+bw9vbGkiVL4ObmhpEjRz7TvomIiGqjqIQslbJmztYGiESZVgnHtWvX8O6778LKquIMSRAEfPLJJ0p3gVRV586d8euvvyIwMBAfffQRvL29sXbtWowbN05cZ+HChcjJycHUqVORnp6OXr164ejRo+KEUyIiovokOkG1h6O5Uy3p4QCABQsWwMnJSat1P/vssyoHVFZAQAACAgI0LpdIJPjoo4/w0Ucf6WyfREREtVXZIRWpxPDP4AC0TDhiYmLQsGFDrSsNDw+Hm5tblYMiIiKiykvJLkBKTqFSWWNHS5gaGxkoov/RKuEonaSpLU9PzyoFQ0RERFUXrWbCaLMaMJwCPMNtsUVFRdi4cSNOnz6N4uJi9OzZEzNnzuTcCSIiIgNRl3C0qAETRoFnSDj+85//ICoqCqNGjYJcLse2bdtw+fJl7Ny5U5fxERERkZai1dyh0rwG3BILVCLh+PXXX/HSSy+Jr48dO4bIyEgYGZWMCw0aNAjdunXTfYRERESkFfV3qNSMHg6tH/z1/fffY+TIkYiPjwdQ8myO6dOn4+jRozh48CAWLlyIzp076y1QIiIiKl/ZIZWacocKUImE4+DBg3jttdfQt29frF+/Hps2bYKNjQ3ef/99LFmyBJ6entixY4c+YyUiIiIN0nIKkZyt/AwsLwcLmMkMf4cKUMk5HGPHjsWgQYOwcOFCDBo0CN98841On7lBREREVROp7gmjNWQ4BajCb6nY2dlh06ZN+PTTTzFhwgQsWLAA+fn5+oiNiIiItBTxWPVXW1u51sKE48GDBxgzZgzatm2LcePGoXnz5ggLC4OFhQXatWuHI0eO6DNOIiIiKkfEE9UeDh8XGwNEop7WCceECRMglUrx6aefwsnJCdOmTYOJiQmWLVuG/fv3Y8WKFRgzZow+YyUiIiINbqtLOGpQD4fWczguX76M69evo2nTphg0aBC8vb3FZa1atcLZs2fFX3ElIiKi6lOsEBD5RHlIxdRYisaONeMOFaASCUfHjh3xwQcfYOLEiTh+/Djatm2rss7UqVN1GhwRERFVLDYlB/lyhVJZSxdrGEklBopIldZDKtu2bUNBQQHeeecdPHr0CBs3btRnXERERKQl9fM3as5wClCJHo5GjRphz549+oyFiIiIqkDdHSo1acIooGUPR2am6oGUJytLNdMiIiIi/VA3YbSVay1MOOzt7ZGYmKh1pe7u7rh3716VgyIiIiLtRTxR18NRC4dUBEHAt99+Cysr7X5xTi6XP1NQREREpJ2sfDniUvOUylxszGBvaWKgiNTTKuHw8vLC5s2bta7UxcUFMpmsykERERGRdqLUPNK8Jj1/o5RWCcf9+/f1HAYRERFVxe3HNfsJo6Uq/VsqREREVHOom79Rk35DpRQTDiIiolosgj0cREREpE8KhYDwMs/gkBlJ0KRhzXmkeSkmHERERLVUTEoOcguLlcpaulhDZlTz/rzXvIiIiIhIK7ceZaiUtXGzNUAkFdNZwrFv3z74+fnpqjoiIiKqwD/xqhNGW7vXgYRj48aNePnll/H666/jwoULAICTJ0+iffv2GD9+PHr27KmXIImIiEjVP/Hqejhq3oRRoBIJxyeffILZs2fj/v37+O233/DCCy8gODgY48aNw9ixY/Hw4UN8/fXX+oyViIiI/iUIAm49Uu7hMJJKatxvqJTS+tdif/jhB2zevBkTJ07En3/+iT59+uDcuXO4c+cOLC1r3mxYIiKiuuxhWh4y8pR/SqRZQyuYyYwMFFH5tO7hePDgAV544QUAQO/evSGTybBs2TImG0RERAagbjiltXvN7N0AKpFwFBQUwMzMTHxtYmICBwcHvQRFRERE5Ss7nALU3DtUgEoMqQDAkiVLYGFhAQAoLCzE8uXLYWurfHCff/657qIjIiIitW6pmzBaQ+9QASqRcDz33HOIjIwUX/fo0QP37t1TWkcikeguMiIiIlKrZMKoasLhW0PvUAEqkXCcPn1aj2EQERGRthKzCpCcXahU1qSBJaxMKzVwUa0qFVlmZiYuXLiAwsJCdOnSBQ0bNtRXXERERKSBut6NmvrAr1JaJxzXrl3D0KFD8eTJEwCAtbU1fvnlFwwaNEhvwREREZGqm+oSjho8nAJU4i6V9957D97e3vj7778RFhaGfv36YdasWfqMjYiIiNS4HpeuUta2rvRwhIWF4dixY+jQoQMA4Pvvv4eDgwMyMzNhY1OzsyoiIqK6QhAEXCuTcEgkQFuPmp1waN3DkZqaCg8PD/G1nZ0dLC0tkZKSopfAiIiISFVcah7ScpWfMNq0oRVszGQGikg7lZo0Gh4eLs7hAEqyrNu3byMrK0ss4y/GEhER6c/VuDSVsnYedtUfSCVVKuHo168fBEFQKgsICIBEIoEgCJBIJCguLtZpgERERPQ/1+NUJ4z6e9lVfyCVpHXCERMTo884iIiISAvX1PRw+NelHo6tW7di/vz54qPNiYiIqHrJixW4Fa/8GyomxlL4uFobKCLtaT1pdNmyZcjOztZnLERERFSOiMdZKCxSKJW1cbOBzEjrP+cGo3WEZeduEBERUfW69jBdpczf0776A6mCSqVE/HE2IiIiw7n2IF2lrJ1nzX7+RqlK3aXSokWLCpOO1NTUZwqIiIiI1LuutofDrtrjqIpKJRzLli2DrW3tyKSIiIjqksx8Oe4mKc+ltLeQwcuhdtzMUamE49VXX4WTk5O+YiEiIiINbsRloOx0ynaedrVmuoPWczhqywERERHVRWGxap6/UUuGUwDepUJERFQrXI5VnSPZsVHtuEMFqMSQikKhqHglIiIi0rlihYCrZe5QkUqA9l61J+Go+U8KISIiqucin2Qhu6BIqczHxQZWppWaimlQTDiIiIhquDA1wymdGtee3g2ACQcREVGNd1nNhNHaNH8DYMJBRERU412+r5pwdG7sYIBIqq5WJRyffPIJJBIJ5s6dK5bl5+dj5syZcHR0hJWVFUaPHo2EhATDBUlERKRDjzPy8Cg9T6nMzdYMbnbmBoqoampNwnHp0iVs3LgRfn5+SuXvvPMODh48iN27d+PMmTOIj4/HqFGjDBQlERGRbqnr3ehYy3o3gFqScGRnZ2PcuHHYvHkz7O3/N2aVkZGB7777Dp9//jleeOEFdOzYET/88APOnTuH8+fPGzBiIiIi3VD3wK9OtWz+BlBLEo6ZM2di2LBh6N+/v1J5WFgY5HK5UrmPjw+8vLwQGhpa3WESERHpXG1/4FepGn8D765du3DlyhVcunRJZdmTJ09gYmICOzs7pXJnZ2c8efJEY50FBQUoKCgQX2dmZgIA5HI55HK5yvqlZeqWkTJ50b9tBTnAh9NqJMe/7VQkB3halYvXn/bYVtqpTe2UlV+E8PhMpTJLUyM0dTSrlvi1aStt46jRCUdcXBzmzJmDkJAQmJmZ6azeFStWYNmyZSrlx44dg4WF5l/dCwkJ0VkMdV2IeQgTjvL8O9cr5DzPKW3x+tMe20o7taGd/kmTQCEYKZV5mslx7I+j1RpHeW2Vm5urVR01OuEICwtDYmIiOnToIJYVFxfj7Nmz+PLLL/HHH3+gsLAQ6enpSr0cCQkJcHFx0VhvYGAg5s2bJ77OzMyEp6cnBg4cCBsbG5X15XI5QkJCMGDAAMhkMt0cXB0lT5Uj5HwIBuQNgMyCbaWJPFeOEPMQDOg2ADIHtlN5eP1pj22lndrUTjeORgKIVSoL6NISQ5/zrpb9a9NWpaMEFanRCUe/fv1w8+ZNpbI333wTPj4+eO+99+Dp6QmZTIYTJ05g9OjRAIDIyEg8ePAA3bt311ivqakpTE1NVcplMlm5J19FywniGSWDDDIJ26oiMmOeU9ri9ac9tpV2akM7XYpNVynr0bxhtcddXltpG0uNTjisra3Rpk0bpTJLS0s4OjqK5ZMnT8a8efPg4OAAGxsbzJ49G927d0e3bt0METIREZFOZObLcetRhlKZpYkR2rrbGiiiZ1OjEw5trFmzBlKpFKNHj0ZBQQEGDRqEr776ytBhERERPZNLMalQlJkL16mxA2RGteIGUxW1LuE4ffq00mszMzNs2LABGzZsMExAREREehB6N0WlrFsTRwNEohu1M00iIiKq487HqCYc3Zsy4SAiIiIdyciV45+yz98wMUIbN9U7KWsLJhxEREQ1zMX7qRDKzN/o7O0A41o6fwNgwkFERFTjnL+nZjilFs/fAJhwEBER1Th/30lWKavNE0YBJhxEREQ1SmJmPiKeZCmVWZsao3Utnr8BMOEgIiKqUf6MVu3d6NHMsVbP3wCYcBAREdUof0YnqZQ916KhASLRLSYcRERENYRCIajt4XiuORMOIiIi0pHwx5lIySlUKmvsaAFPBwsDRaQ7TDiIiIhqCLW9G3VgOAVgwkFERFRjnI1Snb/Ruw4MpwBMOIiIiGqE3MIiXI5NVSozlkrQrYmDgSLSLSYcRERENcCFe6mQFys/z7xDI3tYm8kMFJFuMeEgIiKqAc6oGU55rnkDA0SiH0w4iIiIDEwQBJyKTFQprysTRgEmHERERAZ3NykHsSm5SmWOliZo7WZroIh0jwkHERGRgZ24naBS9ryPE4ykEgNEox9MOIiIiAzsxG3V4ZR+Pk4GiER/mHAQEREZUHpuocrtsCZGUvSuQ/M3ACYcREREBnU6MgkK5bth0bWJA6xMjQ0TkJ4w4SAiIjKg42rmb/Rv5WyASPSLCQcREZGByIsVap+/8UIdm78BMOEgIiIymEv3U5GVX6RU1tLZuk78OmxZTDiIiIgM5OitJypl/VrVvd4NgAkHERGRQSgUgtqEY2BrFwNEo39MOIiIiAzgalwaErMKlMrcbM3QzqPuPF30aUw4iIiIDODwTdXejUFtXCCR1J2niz6NCQcREVE1EwT1wylD27oaIJrqwYSDiIiomt18lIFH6XlKZQ2tTdHRy95AEekfEw4iIqJqdkRN78ag1s6Q1qEfayuLCQcREVE1EgQBR24+Vikf2qbuDqcATDiIiIiq1a1HmbifkqtUZm8hQxdvBwNFVD2YcBAREVWjA9ceqZQNau0CY6O6/Se5bh8dERFRDVKsEPDb9XiV8hf93QwQTfViwkFERFRNzt9LUXnYl4uNGbp5OxooourDhIOIiKia7L+qOpzyor9bnb47pRQTDiIiomqQLy9W+7CvEfVgOAVgwkFERFQtTkYkIqtA+afomzlZwdfVxkARVS8mHERERNVA3XDKSH+3OvvbKWUx4SAiItKzlOwCnIpMVCkf4e9ugGgMgwkHERGRnv169RHkxYJSWcdG9vB0sDBQRNWPCQcREZEeCYKAny/FqZSP7eRpgGgMhwkHERGRHl2NS0d0YrZSmaWJEYb51e3fTimLCQcREZEe/aKmdyPAzw2WpsYGiMZwmHAQERHpSU5BEQ6qeZT5mM71azgFYMJBRESkN7/ffIycwmKlsmZOVujgZWeYgAyICQcREZGe7Lz4QKVsbCfPevPsjacx4SAiItKDmw8zcPVBulKZzEiClzrUn2dvPI0JBxERkR5sOXdfpWxgaxc0sDKt/mBqACYcREREOpaSXYCDN1Qni07q0bj6g6khmHAQERHp2K5LcSgsUiiV+braoFMjewNFZHhMOIiIiHSoqFiB7edjVcon9mhULyeLlmLCQUREpEMh4QmIz8hXKrOzkNWrH2pThwkHERGRDv3w932VsrGdPWEmM6r+YGoQJhxEREQ6Ehabhov3U5XKpBLgja6NDBRRzcGEg4iISEe+OXNXpWyAr3O9+hl6TWp0wrFixQp07twZ1tbWcHJywsiRIxEZGam0Tn5+PmbOnAlHR0dYWVlh9OjRSEhIMFDERERUX91JzEJIuOrfn+l9mhogmpqnRiccZ86cwcyZM3H+/HmEhIRALpdj4MCByMnJEdd55513cPDgQezevRtnzpxBfHw8Ro0aZcCoiYioPvrmzD2Vsm5NHNDeq/7eCvu0Gv3buEePHlV6vWXLFjg5OSEsLAzPPfccMjIy8N1332HHjh144YUXAAA//PADWrVqhfPnz6Nbt26GCJuIiOqZxxl5OHDtkUo5ezf+p0YnHGVlZGQAABwcHAAAYWFhkMvl6N+/v7iOj48PvLy8EBoaqjHhKCgoQEFBgfg6MzMTACCXyyGXy1XWLy1Tt4yUyYv+bSvIAcHAwdRgcvzbTkVygKdVuXj9aY9tpR19tNPmM3chL1b+0PNxsUYPb7ta/X5o01baHp9EEIRa8WdBoVDgxRdfRHp6Ov766y8AwI4dO/Dmm28qJQ8A0KVLFzz//PNYuXKl2ro+/PBDLFu2TKV8x44dsLDgxB4iItJeZiHw8VUjFCqUH+o1oXkxOjaoFX9in0lubi5ef/11ZGRkwMbGRuN6taaHY+bMmbh165aYbDyLwMBAzJs3T3ydmZkJT09PDBw4UG1jyeVyhISEYMCAAZDJZM+8/7pMnipHyPkQDMgbAJkF20oTea4cIeYhGNBtAGQObKfy8PrTHttKO7pup+AjkShUKD9Z1MPODIHjesHYqEZPlayQNm1VOkpQkVqRcMyaNQuHDh3C2bNn4eHhIZa7uLigsLAQ6enpsLOzE8sTEhLg4uKisT5TU1OYmqr+Wp9MJiv35KtoOUE8o2SQQSZhW1VEZsxzSlu8/rTHttKOLtopITMfOy7GqZTPfKE5zM3qzq/CltdW2rZhjU69BEHArFmz8Ouvv+LkyZPw9vZWWt6xY0fIZDKcOHFCLIuMjMSDBw/QvXv36g6XiIjqma9O3UFBmR9p83Qwx8sdPTRsUX/V6B6OmTNnYseOHThw4ACsra3x5MkTAICtrS3Mzc1ha2uLyZMnY968eXBwcICNjQ1mz56N7t278w4VIiLSq/j0POxU07vxnxeaQ1bLh1L0oUYnHF9//TUAoG/fvkrlP/zwAyZNmgQAWLNmDaRSKUaPHo2CggIMGjQIX331VTVHSkRE9c2akCgUFiv3bng3sMRL7ev3j7RpUqMTDm1uoDEzM8OGDRuwYcOGaoiIiIgIuP04E3uuPFQpn9Ovea2fKKovbBUiIqJKCj58G2W/E7d0tsbwdm6GCagWYMJBRERUCWeikvBndLJKeeBQHxhJJWq2IIAJBxERkdaKFQJWHL6tUt6rWQP0adHQABHVHkw4iIiItLT9QiwinmQplUkkJb0bEgl7N8rDhIOIiEgLydkF+PSPSJXyUe090NrN1gAR1S5MOIiIiLSw8kgEsvKLlMosTYywYFBLA0VUuzDhICIiqkBYbCp2h6neBju3fwu42JoZIKLahwkHERFROQqLFHj/11sq5c2drDCpZ+PqD6iWYsJBRERUjq9P31WZKAoAy0a05iPMK4EtRUREpEHkkyx8eSpapfzFdm7o0bSBASKqvZhwEBERqVFUrMDCPdchL1Z+pKi9hQwfDPc1UFS1FxMOIiIiNTb9eQ/XH2aolC8d3hoNrEwNEFHtxoSDiIiojBsP0/H5sSiV8n4+Thjhz99LqQomHERERE/JLSzCnF3XUKRQHkqxNjXGf19qyyeKVhETDiIioqd8fCgcMck5KuVLX2zNZ248AyYcRERE/zp4PR47L8aplAf4uWJ0B3cDRFR3MOEgIiICEJ2Qhff23lApd7cz51CKDjDhICKiei+7oAjTfwpDbmGxUrlEAnw+ph1szWUGiqzuYMJBRET1miAIWLjnOu4mqc7bmNuvBbo2cTRAVHUPEw4iIqrX1h6PxuGbT1TKn2/ZELNfaGaAiOomJhxERFRvHbj2CF+cUH10uYe9OdaM9YdUynkbusKEg4iI6qWw2DQs2K06SdTEWIpv3ugIOwsTA0RVdzHhICKieudJLvB/O66hsFihsuyzV9qhjbutAaKq24wNHQAREVF1epSeh69uGyGjUK6y7N0BLTC8HR9drg/s4SAionojKasAk7aEIaNQdW7GqPbumMVJonrDhIOIiOqF5OwCjP/uAu6n5Kos697EEStG8+Fe+sQhFSIiqvMSM/Px+rcXcCcxW2VZOw9bbJ7YCabGRgaIrP5gwkFERHXak4x8vL75PO6p+UG2Zk5W+OHNLrAy5Z9DfWMLExFRnRWbkoPx313Eg1TVYRRPe3P8OLkLHCx5+2t1YMJBRER10tUHaXh762Wk5BSqLGtoJmD75M5wtTU3QGT1ExMOIiKqc4798wT/2XUV+XLV52w0aWCJSY0y4GprZoDI6i/epUJERHWGIAj49s97mPZTmNpko6WzNbZP7gRbjqJUO/ZwEBFRnZBbWIRFe2/it+vxapd3bmyPzRM6wVLGW18NgQkHERHVejHJOZj+YxgiE7LULg/wc8XqV9rBTGYEuVz1CaOkf0w4iIioVjtw7REW/3oLWQVFapdP69ME7w3y4S+/GhgTDiIiqpUycuVYcuCWxiEUEyMplo1ojde6eFVzZKQOEw4iIqp1/r6TjPm7r+NxRr7a5W62Zvj6jY5o52lXvYGRRkw4iIio1kjNKcTy38Ox78ojjev0bOaIda+2h6OVaTVGRhVhwkFERDWeIAjYE/YQwYdvIy1X/aRPI6kEc/o1x4y+TWFsxKc+1DRMOIiIqEYLi03Fx4du41pcusZ1vBtYYs1Yf/hzCKXGYsJBREQ10oOUXKw8GoHfbz4ud71xXb3w/rBWsDDhn7SajO8OERHVKA9ScrHh1B3svfIQRQpB43pNG1oi+KW26NrEsRqjo6piwkFERDXC/eQcfHnqDn69+gjF5SQaJsZSzH6+Gab2aQJTY6NqjJCeBRMOIiIyGEEQcCEmFVv+vo9j4U9QTp4BABjSxgWLhvigkaNl9QRIOsOEg4iIql2+vBi/XY/HD3/fx+3HmRWu387TDouHtULnxg7VEB3pAxMOIiKqFoIg4GpcOvaEPcTB6/HIylf/KPKntXS2xn/6NceQNi58NHktx4SDiIj0KiY5B4dvPsavVx/hTmK2Vtv4uFhjTr/mGNSaiUZdwYSDiIh0LjohC4dvPsGRW48R8UT9L7iq06WxAyb39saAVs5MNOoYJhxERPTMcguLcP5eCs5GJeNMVBJiknO03tbESIoX/d0wqUdjtHG31WOUZEhMOIiIqNIKiopx82EGLt5Pxbk7KbgYk4rCYkWl6mjSwBIvd/LAmE6eaMDfPanzmHAQEVGF0nIKcf1hOi7fT8PF+6m4HpeOgqLKJRgAYGVqjOHtXPFyRw908LKHRMJhk/qCCQcRESlJzSnErUcZuPkoQ/z3YVpeleuzMjVG/1ZOGNzGFX1aNIS5CR/WVR8x4SAiqocEQUBKTiGiE7IRnZil9G9KTuEz1+9oaYK+LZ0wtK0LejVvwCeCEhMOIqK6Kl9ejMcZ+XiQmosHqbmIS83Fg5RcxP77/9kFFT8HQ1tGUgk6etmjT8uGeK55Q7R2s+FdJqSECQcRUS2TLy9Gak4hUnMKkZRVgCeZ+XiSkY/49FzcuivFV/fOISGrAOm5cr3FIDOSwM/DDp0bO6CLtz06NXaAjZlMb/uj2o8JBxGRASgUArILi5CVX4SsfDmy8ouQmScXX2fmFyEjT46U7EKk5hQgNacQKf8mGbmFxeXULAWg3cO1KsPV1gxt3G3RzsMWnRs7oJ2nHcxkHCYh7dWZhGPDhg349NNP8eTJE7Rr1w7r169Hly5dDB0WEdUixQoBhUUKFBYpUFBcjMIiBeTF/ysrLC5Gwb//Ly8WUFBUjLzCYuTJS/7NLfP/+fJi5BYWiWV58mLkFBQjM1+O7IIiCBX8UJkhSCVAY0dLNHe2Qhs3W7TxsEUbN1s0tOZtq/Rs6kTC8fPPP2PevHn45ptv0LVrV6xduxaDBg1CZGQknJyctK4nLDYVVtaqY5pFRUW4mwlcjk2DsbH2TaavDxNBTxXrotaizCLcyQAuFKbCuMhYJ3Wqo6820AdB5V8BRfnFiCiQwDomGUYpxuLxCGU2EiCI55H4b+kqT20jlNnw6XWFp+oqrx6Ut66gvL7w778KAVAIJTE+/Vrx72vhqdeKfytTKJ7e7t91oVyXQlGyD4UgoKhYgfsxUlw4GA4BEhQrBBQphP/9W1z6WoFiAShWKFBULKiup1CgWKFcXpJMFKOwuCSJqOiXSusSc5kRvBws4N3AEi2crdDM2RrNnazg3cCSPRekFxKhNn1ya9C1a1d07twZX375JQBAoVDA09MTs2fPxqJFiyrcPjMzE7a2tvCc+wukphb6DpeISO9MjaVwsTWDs40ZPOzM4eVoAS8HCzRytICngwUaWpnW22dgyOVyHD58GEOHDoVMxnkn5dGmrUr/hmZkZMDGxkZjXbW+h6OwsBBhYWEIDAwUy6RSKfr374/Q0FC12xQUFKCgoEB8nZGRAQBQFOTqN1giomdkY2YEewuTkv8sZbC3kMHJygwNbUzgYG6M+7evI6BfLzham2tIKASgMAepqdo/eryukcvlyM3NRUpKChOOCmjTVllZJb+VU1H/Ra1POJKTk1FcXAxnZ2elcmdnZ0RERKjdZsWKFVi2bJlK+aOvJ+kjRCKiajXf0AFQvZSVlQVbW82/hVPrE46qCAwMxLx588TXCoUCqampGDx4MC5fviyWd+7cGZcuXUJmZiY8PT0RFxdXbndRVZXuR9fblLeOpmXqyisqe/r/9dlW+mqnitZ7lrbS9LounlPlLWdbabeM15/2y3hOab9c320lCAKysrLg5uZWbvy1PuFo0KABjIyMkJCQoFSekJAAFxcXtduYmprC1FR5xrWdnR2MjY2VGtTIyEjptY2NjV5OzrL70dU25a2jaZm68orK1C3XR1vpq50qWu9Z2qqi13XpnCpvOdtKu2W8/rRfxnNK++XV0Vbl9WyUkla4Rg1nYmKCjh074sSJE2KZQqHAiRMn0L1790rVNXPmzHJf60tV9qPNNuWto2mZuvKKymp7O1W03rO0VX06p8pbzrbSbhmvP+2X8ZzSfnlNaas6cZfKzz//jIkTJ2Ljxo3o0qUL1q5di19++QUREREqczuqQtsZuMS20hbbSXtsK+2xrbTDdtKeLtuq1g+pAMDYsWORlJSEDz74AE+ePIG/vz+OHj2qk2QDKBmCWbp0qcowDKliW2mH7aQ9tpX22FbaYTtpT5dtVSd6OIiIiKhmq/VzOIiIiKjmY8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIiUnL27FkMHz4cbm5ukEgk2L9//zPXyYRDx2JiYvD888/D19cXbdu2RU5O/f2BpIo0btwYfn5+8Pf3x/PPP2/ocGq83NxcNGrUCPPn85cy1ElPT0enTp3g7++PNm3aYPPmzYYOqcaKi4tD37594evrCz8/P+zevdvQIdVoL730Euzt7fHyyy8bOpRqk5OTg3bt2mHDhg06q5O3xepYnz59sHz5cvTu3RupqamwsbGBsXGdeNyJzjVu3Bi3bt2ClZWVoUOpFd5//33cuXMHnp6eWL16taHDqXGKi4tRUFAACwsL5OTkoE2bNrh8+TIcHR0NHVqN8/jxYyQkJMDf3x9PnjxBx44dERUVBUtLS0OHViOdPn0aWVlZ2Lp1K/bs2WPocKqdRCLBr7/+ipEjRz5TPezh0KF//vkHMpkMvXv3BgA4ODgw2SCdiI6ORkREBIYMGWLoUGosIyMjWFhYAAAKCgogCEKFP5ddX7m6usLf3x8A4OLiggYNGiA1NdWwQdVgffv2hbW1taHDqPXqVcKhzZjUhg0b0LhxY5iZmaFr1664ePGi1vVHR0fDysoKw4cPR4cOHRAcHKzD6KuXvtsKKMma+/Tpg86dO2P79u06irz6VUdbzZ8/HytWrNBRxIZRHe2Unp6Odu3awcPDAwsWLECDBg10FH31qo62KhUWFobi4mJ4eno+Y9SGUZ1tRc+mXiUcFY1J/fzzz5g3bx6WLl2KK1euoF27dhg0aBASExPFdUrHh8v+Fx8fj6KiIvz555/46quvEBoaipCQEISEhFTX4emUvtsKAP766y+EhYXht99+Q3BwMG7cuFEtx6Zr+m6rAwcOoEWLFmjRokV1HZJeVMc5ZWdnh+vXryMmJgY7duxQ+RXp2qI62goAUlNTMWHCBGzatEnvx6Qv1dVWpANCPQVA+PXXX5XKunTpIsycOVN8XVxcLLi5uQkrVqzQqs5z584JAwcOFF+vWrVKWLVqlU7iNSR9tFVZ8+fPF3744YdniLJm0EdbLVq0SPDw8BAaNWokODo6CjY2NsKyZct0GXa1q45z6v/+7/+E3bt3P0uYNYK+2io/P1/o3bu3sG3bNl2FanD6PK9OnToljB49Whdh1jrq2rUq6lUPR3kKCwsRFhaG/v37i2VSqRT9+/dHaGioVnV07twZiYmJSEtLg0KhwNmzZ9GqVSt9hWwwumirnJwcZGVlAQCys7Nx8uRJtG7dWi/xGpIu2mrFihWIi4vD/fv3sXr1akyZMgUffPCBvkI2CF20U0JCgnhOZWRk4OzZs2jZsqVe4jUkXbSVIAiYNGkSXnjhBYwfP15foRqcLtqKdIczGv+VnJyM4uJilV+YdXZ2RkREhFZ1GBsbIzg4GM899xwEQcDAgQMREBCgj3ANShdtlZCQgJdeeglAyd0FU6ZMQefOnXUeq6Hpoq3qA120U2xsLKZOnSpOFp09ezbatm2rj3ANShdt9ffff+Pnn3+Gn5+fOOfhxx9/rHPtpavrr3///rh+/TpycnLg4eGB3bt3o3v37roOt0bJzs7GnTt3xNcxMTG4du0aHBwc4OXlVaU6mXDo2JAhQ3gngRaaNGmC69evGzqMWmfSpEmGDqHG6tKlC65du2boMGqFXr16QaFQGDqMWuP48eOGDqHaXb58Wen5SPPmzQMATJw4EVu2bKlSnUw4/tWgQQMYGRmpTDJLSEiAi4uLgaKqmdhW2mNbaYftpD22lfbYVlXXt29fnd9Wzjkc/zIxMUHHjh1x4sQJsUyhUODEiRN1vuussthW2mNbaYftpD22lfbYVjVLverhqGhMat68eZg4cSI6deqELl26YO3atcjJycGbb75pwKgNg22lPbaVdthO2mNbaY9tVYs8830utcipU6cEACr/TZw4UVxn/fr1gpeXl2BiYiJ06dJFOH/+vOECNiC2lfbYVtphO2mPbaU9tlXtwd9SISIiIr3jHA4iIiLSOyYcREREpHdMOIiIiEjvmHAQERGR3jHhICIiIr1jwkFERER6x4SDiIiI9I4JBxEREekdEw4iIiLSOyYcRGQQkyZNgkQiUfnvzp07SstMTEzQrFkzfPTRRygqKgIAnD59Wmmbhg0bYujQobh586aBj4qINGHCQUQGM3jwYDx+/FjpP29vb6Vl0dHRePfdd/Hhhx/i008/Vdo+MjISjx8/xh9//IGCggIMGzYMhYWFhjgUIqoAEw4iMhhTU1O4uLgo/WdkZKS0rFGjRvi///s/9O/fH7/99pvS9k5OTnBxcUGHDh0wd+5cxMXFISIiwhCHQkQVYMJBRLWCubm5xt6LjIwM7Nq1CwBgYmJSnWERkZaMDR0AEdVfhw4dgpWVlfh6yJAh2L17t9I6giDgxIkT+OOPPzB79mylZR4eHgCAnJwcAMCLL74IHx8fPUdNRFXBhIOIDOb555/H119/Lb62tLQU/780GZHL5VAoFHj99dfx4YcfKm3/559/wsLCAufPn0dwcDC++eab6gqdiCqJCQcRGYylpSWaNWumdllpMmJiYgI3NzcYG6t+XHl7e8POzg4tW7ZEYmIixo4di7Nnz+o7bCKqAs7hIKIaqTQZ8fLyUptslDVz5kzcunULv/76azVER0SVxYSDiOoECwsLTJkyBUuXLoUgCIYOh4jKYMJBRHXGrFmzcPv2bZWJp0RkeBKBXwWIiIhIz9jDQURERHrHhIOIiIj0jgkHERER6R0TDiIiItI7JhxERESkd0w4iIiISO+YcBAREZHeMeEgIiIivWPCQURERHrHhIOIiIj0jgkHERER6R0TDiIiItI7JhxERESkd0w4iIiISO+YcBAREZHeMeEgIiIivWPCQURERHrHhIOIiIj0jgkHERER6R0TDiIiItI7Y0MHUFcsXrwYZmZmkEpLcriJEyfiwYMH+PXXX2FnZwdBEDB69Gj4+PiI6yoUCnh7e2PcuHHidpWRm5uLBQsWYNKkSejcubOuD0mt0NBQ/Prrr+jWrRtGjRollsfGxiIsLEyprCrWrFmDd95551nD1Ep6ejq++uorpKWl4dNPPy133ZSUFHz00UdwdnYGABgbG2PhwoVq11UoFPjkk0+QmpqKiRMnom3btjqNu/T8AQAPDw+MHz8eRkZGVapLXXuHhoaiTZs2sLa2Vio/e/YsLCws0KlTp6oFrkbp+WRnZwcAePnll9GiRYtyt4mKioKJiQkaN278TPtOSUnBgQMH8NZbb6mNpUmTJnj11VfVtvfFixc1xh0aGgqFQoGePXs+U3w1VVXP759++gmDBg1Cw4YN9Ryhfm3btg2RkZHo0KEDRo8ebehwaheBdOL9998X8vLylMrOnTsn7NmzRxAEQXj06JGwZMkSpXWLi4uFzz//XLh06VKV9nnlyhVh5cqVwtatW58t+Ep4+pjqivnz51e4TnJysrBixYpK1bt161bhxo0bVQ1Lo9LzR6FQCN99950QGhqq0/o///xz4dGjRzqtU5Onz6fbt28LH374YYXbHDx4UDh16tQz7/vnn38WIiIi1MbyNHXtXV7chYWFlT5XaiN9nd+1QV38HKwO7OGoJm5ubigsLIRCoRDLpFIpvL29kZaWJpadPn0a6enpGDlyZIV13r59GwMHDsSvv/4qli1btgzvvfcezMzMcPfuXZw8eRJTpkzB7du38csvv8DW1haFhYUYOXJkhd8ktXXkyBGcP38eHh4emDJlCoCSb49fffUVnJycEBcXh+HDh6Nr164a6wgPD8f+/fuVehuioqKwb98+FBQUoEWLFoiKisLUqVPh6uqKDRs2ICMjA1KpFAEBAWjTpg2KioqwZcsWJCYmwtXVFenp6XjnnXeQm5uL7du3IyUlBaamppg0aRLs7e11cuzXrl3DkSNHIJFI4OTkhDfeeAMmJiYa1z9x4gT++usvGBkZoXv37ujXrx8A4MCBAwgPDwcAjBo1Ci1btqxw3xKJBM2aNUNSUhIAICQkBOfPn4dUKsXLL7+Mli1bQqFQYMuWLYiPjwcAjB07Fs2bN1fb3ikpKdi4cSOSkpKwefNmyGQyzJgxA3Z2dti4cSNiYmIwePBg9O3bF0DJN71OnTrB19cXcrkcH3/8MT788EPk5+dXqb2bNm0qXgtJSUnYsmULioqKYGZmhjfeeAMNGzbEF198gcePH0MqleLcuXMICAiAn58f4uPjsWvXLhQUFMDV1VWrXp/w8HC8/PLLFcalrr0bNGigNm4AkMlkcHR0xIMHD+Dl5VVunZGRkdizZw8UCgW6deuGAQMGVPra2bZtGwRBQHx8PIyNjTFnzhyYmJiorRsoua5CQkIgkUiQkpKCHj164MaNGzA1NYVUKkVeXh6cnJwwbty4Sp/f6mRnZ2PdunVISkrCggUL4ObmBqCkp87S0hKenp64f/8++vXrh+7du+PgwYO4efMmJBIJ/P39MWTIEADAH3/8gQsXLsDDwwPh4eFYvXo1APXXTnp6OjZv3gy5XA6pVIr//Oc/sLCw0BijumsnNDQUV69eRX5+PtLT0zF9+nQx9rIyMjKwYcMGBAUFAQDOnDmDnJwcDB06tFJtVddxDocOrV69GsHBwVizZo3Ksrt378LS0lJp6EQul+PBgwdo3bq1WJadnY3MzEyt9hcVFQUfHx/Y2tri8ePHAABfX1/cvn0bAHDjxg34+fkBAH7++WfMnDkT06ZNE/9A6cqQIUMwbtw4lfLk5GS89tprmDVrFo4fP15uHb6+vuLF+rTGjRvjxRdfhEQiwYsvvoiIiAgAwLhx4xAUFIT//Oc/2L17NwAgLCwMJiYmCAoKgouLi1jH4cOH0b59eyxatAj9+/fH4cOHq3Scjx8/RnBwMIKDg7Fv3z4AJd3u7733HhYtWgQHBwdcunSp3DqOHj2KoKAgLF68GF26dAFQ8j7l5uYiMDAQs2bNEo+nIsXFxbh16xZcXV2RnJyM8+fPY9GiRZgyZQp27NgBAHj48CHS09OxePFiBAYGwt3dHYD69nZ0dERQUBC8vLwwZcoUBAUFiUMG06ZNQ69evZTW9/Pzw82bNwGUnIstWrSAVCqtcntHRESgffv2AAArKyvMnj0bgYGBGDhwIH7//XcAwJw5c9CrVy8MHDgQQUFB4vm9Y8cOTJw4EYGBgbC2tkZYWFi5+0pOToa1tbXKUOaFCxfE9/jy5csa21tT3KU8PT1x7969Co95x44dmDp1KhYtWoRz584hOTlZjE/bawcALC0tERgYCAcHB/GPr6a6AeDOnTt45ZVXsGTJEnTv3h0AMH78eCQnJ+Ptt99GYmIigMqf3+pYWVmJ59XTTE1NMXv2bMTExOCdd97BjRs3AAC9e/dGUFAQFi1ahFu3biEpKQkpKSm4ePEiAgMD0adPH+Tm5gLQfO1cvnwZrVq1Ej8jykuSNF07AJCZmYm5c+eif//++OuvvzTWYWtrCysrKzGxDwsL0+nQY13BHg4dmj9/vjjWW+rChQuIjIyEubk5JkyYIJavXr0aKSkp6NOnj1LWHBAQoNW+kpOTYWFhATMzM7Ro0QIRERFwdXVFu3btcP78ebRv3x7//PMPBg0ahJycHEgkEvFb2bOOfWvLyckJNjY2sLa2RkZGRpXqsLKygoWFhfhvaWJ1+vRp8YM1NTUVQMk8El9fXwBAq1atxOQkKioKEREROHbsGBQKBRwcHKoUi6urKxYtWqRUlpGRge+//x65ubnIzs5W+aNcloeHB3766Se0adMG7dq1E+MLDw9HcHAwgJK5OUVFRTA21nx5rl69GlKpFD4+PujQoQOuX7+OZs2aQSaTwcnJCVKpFLm5uXBwcEBaWhr27dsHHx8fsX10wdfXF7/99huAkg/+p4+nMu194cIF3LhxA4WFhZg/fz6AknkCu3btwpMnT1BUVARbW1uN2+fl5SEuLg4bN24EABQWFsLS0rLcfWZkZKhdp2vXrmrH5cu294ULF9TGXcrKygopKSnlxpCTkwOpVCrOaWjRogUePnwIT0/PSl87zZo1AwA4OzsjIyNDY92lnwHe3t7istJv/k9fa0+3U2XO78qwtLQU/7OwsEB+fj6AkmTo+PHjUCgUSElJQXp6OrKzs8Xzu2nTppDJZAA0Xzuenp7YuXMnjI2N0aZNG3h4eGiMIy4uTu21A5QkXFKpFM7OzoiKiir3eLp06YJLly6hT58+KC4uhpOTky6aqU5hwqFnmj7A5s+fj7y8PKxcuRKdOnUq94JQ5/bt20hKSsKyZcsgl8vh5uaG559/Hs2aNcPOnTuRkJAAGxsbWFhYICcnR1eHUymlXdoSiQSCIIjlf/zxB8LCwmBra4uZM2dWWI9EIhH/VSgUiIyMRGxsLBYuXCh2IVe0/cyZM3U2jPK03bt3Y+jQofDx8cGxY8dQWFhY7vqzZs3CnTt3cO7cOVy9ehVTp04FUJJoltdtXpa65FYdKysrvP/++wgPD8ehQ4eQkJCA559/Xuv9lMfExAROTk54+PAhoqKixOGJyrZ3165d8dJLL+H333/Hvn37MGXKFJw6dQoNGzbE5MmTERMTg/3795dbh52dndoeMk1kMhmKioq0Xl9de6uLu5RcLq/00MPTNF07Fa0PQKv1zc3NVcokEon4X6nKnt+V8fT+Sq9tuVyOvXv3IjAwEDY2Nvjyyy8rPB51107Lli0xZ84c3Lx5E19//TWmT58OT0/PSseo6X14uo1K+fv7Y/Xq1bCysmLvhgYcUjEge3t7DBw4ECdPnhTLTp8+XeGHK1DSjfvmm29i6dKl+PDDDxEbG4vi4mJxXsiBAwfE7mZLS0sIgoDk5GTk5eXh/v37SnVt2bIF165d0+GRlW/QoEEICgrSKtlQJz8/H9bW1jA2NsaNGzcgl8sBAI0aNRKHk0r/BYDmzZsjNDQUQMkfgocPHz7jESjHYm9vj6KiIpU2tLS0VPp2KggC0tPT0bJlSwwdOlTs4m7RogUuXryI4uJiAFB5f7Th6emJO3fuQC6XIzExEQqFAhYWFsjOzoYgCOjQoQN69OhR4bduADAzM9M6SW3Xrh0OHz4MNzc38VtnVdpbKpVi8ODBuHfvHpKTk8V2BaAyPGJubi5+Ay19bWZmJn4DzcrKqvA4nZyctGqLysZdKikpSWlYT52nr0u5XI7o6OhKf/HQd92VOb91oXTOhaWlJdLT08VhKS8vL/H8vnv3rnjNa7p2UlNTYWtri+eeew5NmzYVe0HV0XTtlMfCwkLl2M3MzODm5objx4+jQ4cOSsuq+zO2pmIPh4H17NkTH374ITIzM2FjY6PVHA6FQoHo6GiMHz8eQMktmm5uboiJiUGzZs3g5+eHTZs24ZVXXhG3GTt2LDZs2AA7Ozs4OTkpddenpaUhLy+vysewatUq5OTkIDs7G8HBwRg6dGilv03s2rUL9+7dQ15eHoKDg9G6dWu0atVK7bq+vr44c+YMPv74YzRt2hQ2NjYAgI4dO+LmzZsIDg6Gm5ubeIzDhg3D9u3bsXz5cgiCgP79++vsg33IkCH46quvYGlpqTK236NHD3z77bc4e/Yspk+fDjs7O/zwww9i1/GIESMAlMyFiI2NxYoVK1BcXAwfH59KD3s1aNAA3bp1wyeffAKpVIrXXnsNQEmX+NatWwGUfKufNGkSAPXtXRpPz549sXPnTlhaWuLtt99GdnY2tm7diszMTHGy5vTp0+Hg4IC2bdti+/bt4rkIVL29ZTIZ+vbti1OnTuG5557D5s2b8eeff6pMbm7Xrh02bdqE8PBwDBkyBK1bt8bEiROxc+dO5OXlwcjICOPGjYOjo6PGfZmZmcHGxgYZGRnlDtdo4+m4S6+52NhYrYZHX331VWzcuBEKhQLdu3dHgwYNdJIIaaq7sipzfmsaOgsNDcWpU6fEycgODg6YPXu22nUtLCzQuXNnfPzxx3BwcBCvA0dHR3Tp0gUrVqyAl5eXeMu2pmsnOjoax44dEye7ljeUqOnaKU+rVq1w/PhxLF++HAEBAfD39wcAtG/fHpmZmSrn1LN+xtYVEkGb/jeq9fLz82FmZoaioiKsWLECc+fOVXnOgjZCQ0MRHx9fI+8/FwQBhYWFMDU1RWhoKB4+fKiUdGmyYMGCCp/DQXXP5cuXkZaWJt69oSuPHz/GH3/8ISZ3pBuln2HJycnYtGlTpYbQqsvevXvh4uJSZ5/B8qzYw1FPXLx4EWfOnAEA9OrVq0rJBlDybe769euQSCTP/JAvXVMoFPjss88gCAIsLCzw5ptvlrt+6YO/SntIqH7p0KEDLl68qPN6s7KytJ78Tdr75Zdf8ODBA/HW1ZpmzZo1kEqlGD58uKFDqbHYw0FERER6x0mjREREpHdMOIiIiEjvmHAQERGR3jHhICIiIr1jwkFERER6x4SDiIiI9I4JBxEREekdEw4iIiLSOyYcREREpHdMOIiIiEjvmHAQERGR3jHhICIiIr3jr8XqwObNm5GUlITU1FSYm5vD3Nwcffr0gVQqxa+//go7OzsIgoDRo0fDx8cHixcvhpmZGRQKBby9vTFu3DhIpZXP/VJSUvDRRx+hcePGeOedd5SWrVmzRqWssn766ScMGjQIDRs2fKZ6tLVhwwbcuXMHCxYsgJubm8b1NLW3Pn8SOjQ0VO17WRVnz56FhYUFOnXqJJalpKTg/v376Nixo8r6ungvy6rKOXjs2DEMHDjwmfe9Y8cO9OnTB+7u7jq7dhQKBdavX49Zs2bByMjomWOsqm3btiE6OhqmpqawsrLCpEmTYGdnp5O6z5w5gzNnzsDExASLFi1S2mdkZCQ6dOiA0aNH62RfRHohkM5s3bpVuHHjhvj63Llzwp49ewRBEIRHjx4JS5YsEQRBEN5//30hLy9PKC4uFj7//HPh0qVLVdpfcnKysGLFimcPvAb5/PPPhUePHmm1btn21idN76WuREZGCps2bdJpneWpyjk4f/78Z95vZmamsHbtWpVyXVw7R48eFS5evPjMMT6Lp4/jwIEDwvbt23Vav6Zr/un2Iqqp2MNRTdzc3FBYWAiFQiGWSaVSeHt7Iy0tTSw7ffo00tPTMXLkyCrtJzw8HPv370daWho+/fRTsXzx4sXw8/NDeHg4fHx88Oqrr2qsIzs7G+vWrUNSUpJSb8PixYthaWkJT09P3L9/H/369UP37t1x8OBB3Lx5ExKJBP7+/hgyZAgA4I8//sCFCxfg4eGB8PBwrF69GgBw4MABhIeHAwBGjRqFli1bVulY1Tl06BCys7Px4MED5ObmYsKECfjrr7/Qvn17tG3bFocOHYKVlRX69u2L+Ph47Nq1CwUFBXB1dcX48eO1+nb89HuZnp6OLVu2IDc3F+7u7hg/fjyMjY0RGRmJ3bt3QyKRwMXFBZMnTwYAbNy4ETExMRg8eDD69u0LANi/fz+uXr2K7OxsBAcHo23bthg+fLja9zIlJQXff/89FixYAAA4cuQIzM3N0bdvX1y5cgXHjh1DUVERunfvjn79+lV4LGXPQXXv5ZUrV3D06FHk5eUhODgYtra2mDlzJoDKv5dXrlyBr69vhXFpam9NcQNA27ZtceDAAXTu3LlS9ZdKTU1V+14CwIIFC9CjRw/8888/cHV1Fd/P8jRt2hSnT58GAOTm5mL79u1ISUmBqakpJk2aBHt7e+Tl5WH79u1ISEiAkZERpk2bBnt7e2zYsAEZGRmQSqUICAhAmzZtqnRMRDUJE45qcvfuXVhaWip1W8vlcjx48AAvv/yyWJadnY3MzMwq78fX1xe+vr7iH6Sn+fj4YPTo0Vi6dCmys7NhZWWltg4rKysEBQVhzZo1SuWmpqaYPXs21qxZg3nz5uGnn35C9+7d0bt3bwwfPhyCIGD16tXo1KkTpFIpLl68iMDAQDx48ACXL18GANy4cQO5ubkIDAxEVlYWvvjiCyxevLjKx6vO/fv3MW/ePBgbG6OgoEDjejt27MCbb74JR0dH7N27F2FhYejSpUuF9T/9Xv7+++/o0aMHunXrhi1btuDy5cvo1q0bTpw4gVdeeQUtW7ZEVlaWuO20adNw6NAhpfpGjhwJX19fnDlzBlOmTBHL1b2Xjo6OKCoqQlZWFqytrXHz5k1MmTIFmZmZOHnyJN59910YGRnhs88+Q/v27eHg4FDusZQ9B9W9lx06dECHDh2wYMECBAUFidtW5b2MiYlBt27dKmzjp2l77bi4uCAuLq5SdT9N03sJlCQMnp6eeOmll5Cbm6tVfREREejQoQMA4PDhw2jfvj06deqEmzdv4vDhwxg3bhx+//13eHh44O2330ZOTo647bhx42BnZ4fc3FysXLmSCQfVCUw49OzChQuIjIyEubk5JkyYIJavXr0aKSkp6NOnj9J8hYCAAL3F0qxZMxgZGcHBwQFZWVkaEw5NLC0txf8sLCyQn58PALhz5w6OHz8OhUKBlJQUpKenIzs7G82aNYNMJkPTpk0hk8kAAFFRUQgPD0dwcDCAkg/yoqIi8ZukLvj7+8PExAQAYG5urnadvLw8xMXFYePGjQCAwsJCWFpalluvuvcyNjYWI0aMAAC0adMG9+7dQ7du3eDt7Y1Dhw7hyZMn8Pf319GRlWjbti1u3boFX19fCIIAe3t7XL9+HYmJiWJPSF5eHpKTk8tNONSdg+reS01zeKryXmZmZlbYzqUqe+1IpVIIgoDi4uIqzePQ9F4CgJGRkTi/xsLCotx6fvnlF2zfvh1OTk5iT2VUVBQiIiJw7NgxKBQK8X2JiorCjBkzAECpXU6fPi32HKWmplb6WIhqIiYceta1a1e1E7nmz5+PvLw8rFy5Ep06dYKHh4feYyn9hiiRSCAIAoCSb2H79u0DAMyYMaPcCW4SiUTpP4VCAblcjr179yIwMBA2Njb48ssvxbo1CQgIQNeuXXVzUGqYmZmpxF3q6W55Ozs7pW/sFdH0XqozZMgQtG3bFteuXcOqVauwbNkynSVV7dq1w5EjR1BcXAw/Pz8AJcfo5+eHN954Q+t6yp6Dzs7Oen8vZTIZioqKtFq3KteOIAh6mTRqamqqdB6VZ8yYMfDx8cFXX32FP//8E3379oVEIsHMmTNhb2+vtK66OiMjIxEbG4uFCxfC2NgYc+bMqXCf2sZGZEi8LdaA7O3tMXDgQJw8eVIsO336NPbv319tMfj4+CAoKAhBQUFVmk0vl8shlUphaWmJ9PR03Lt3DwDg5eWFO3fuQC6X4+7du5DL5QCAFi1a4OLFiyguLgZQMvyhbzY2NkhPTwcAPHz4EEBJz4eZmRmioqIAAFlZWUhJSal03Y0aNcLt27cBlMyf8fLyAgAkJyfDw8MDQ4YMgUQiQWFhYbn1mJmZKXWpl8fT0xOJiYm4cuUK2rVrJ8YRGRkpDt88evRIbPPyPH0OanovS0kkEqVkoSrvpbOzM5KTk7U6Tm3jLpWbmwtra2uVdbdt26YyjKWOpveyKmQyGUaNGoVTp05BoVCgefPmCA0NBVByzZSeh82bN8f58+cBlPRK5ebmIj8/H9bW1jA2NsaNGzeU3kcLCwtkZWUpJc6l5RkZGSpxVPfnCVF52MNhYD179sSHH36IzMxM2NjYPPMcjl27duHevXviBL/WrVuL3cTaCg0NxalTp5CUlITNmzfDwcEBs2fPVruuhYUFOnfujI8//hgODg5o3LgxgJK5Bl26dMGKFSvg5eUl/iHw8/NDbGwsVqxYgeLiYvj4+Ijb6Eu3bt3w3Xff4e7du0rzACZOnIidO3ciLy8PRkZGGDduHBwdHStV97Bhw7BlyxaEhITA3d1dvNX15MmTiIyMhCAI6NmzJywsLPDo0SNs3boVmZmZkEqlOHfuHKZPnw4HBwe4u7tDKpVi1apV8PPzw+DBg8t9L1u2bIlbt26JQwq2trYYNWoU1q1bB0EQYGlpKXbVV6T0HCwqKlL7Xpbq1asXPvnkEzg7O2PKlClVei/btGmDy5cvV3lip7q4S6+d6OhotRNS09LS0L59+wrr0/ReVpWnpycaNmyIGzduYNiwYdi+fTuWL18OQRDQv39/eHh4iOX//e9/IZPJMGXKFHE+z8cff4ymTZvCxsZGrNPc3Bzt27fHxx9/DH9/f/F8aNWqFY4fP47ly5cjICBAHMZ71s8TIl2SCBX1mVKNlZKSgs2bNyvdk1+T5Ofnw8zMDMnJydi0aZNWwxdr1qzB2LFjy30OB9VegiBgzZo1mD17tjivR1e+//57DBs2DM7OzmJZcXExPv30U7z33nscdiAyMPZw1GISiQSZmZl6eTCULvzyyy948OABpFKp0t0EmmzYsAHJyckGfXAT6ZdEIsGLL76ItLQ0ODk56axehUIBX19fpWQDKJnsWVMTcqL6hj0cREREpHecNEpERER6x4SDiIiI9I4JBxEREekdEw4iIiLSOyYcREREpHdMOIiIiEjvmHAQERGR3jHhICIiIr1jwkFERER6x4SDiIiI9I4JBxEREekdEw4iIiLSOyYcREREpHdMOHQgOzsbwcHBmD9/PpYsWYJt27ZVuE1UVBTu37+v/+CeEhoaiqysLJXys2fP4vLly9Uaiz799NNPSEpKqtK2MTExCA4OxuzZs5Gfn1/luhcvXqx2ewA4duxYlWJ7WmxsLPbt21fheikpKfjkk0+eeX9Vdfr0aRw6dMhg+9e1zZs3IyoqSqd1lr0uDf2e6UtKSgq+//57pbIbN25gxowZSE9PF8sWLFgg/v/evXsRGhoKAIiPj8eqVavw0Ucf4dNPPxXXCQ0Nxd9//611HNpeO4awbds2vP/++9i7d6+hQ9ELJhw6YGVlhaCgIPj5+WHMmDGYMGFChdsYIuE4f/682oTjueeeQ6dOnao1Fn1644030LBhwypt6+3tjaCgINja2uq87lIhISHPtD0ANGrUCKNGjXrmesjwNF2Xdc2JEyfQs2dPpbLbt2+jcePGiIiIqHD7bdu24bXXXsMHH3yAcePGieWdOnXCn3/+qXUcNfnamTBhAgICAgwdht4YGzqAuiw0NBRXr15Ffn4+0tPTMX36dLi5ueGLL77A48ePIZVKce7cOQQEBMDPzw/x8fHYtWsXCgoK4OrqivHjx8PIyAh//PEHLly4AA8PD4SHh2P16tUAgEOHDiE7OxsPHjxAbm4uJkyYgMzMTBw5cgQSiQROTk544403kJWVhY0bNyIpKQmbN2+GTCbDjBkzYGdnh40bNyImJgaDBw9G3759AQBFRUX48ccf8ejRI1hYWGDSpElwcHDAtm3bIAgC4uPjYWxsjDlz5sDExETj8a9fvx5ZWVkQBAGpqan47LPPKl23pjZRJzs7G+vWrUNSUhIWLFgANzc3AKh03JWpOzU1Fd9++y0EQYC1tTW8vLzED4zffvsN4eHh8PHxwauvvoorV67g6NGjyMvLQ3BwMGxtbTFz5kyN+9y2bRvkcjkSExNhY2ODt99+G6ampjhy5AjOnz8PDw8PTJkyBUDJt8evvvoKTk5OiIuLw/Dhw9G1a1el+v766y9ER0dj0qRJkEgklTr+DRs2ICMjA1KpFAEBAWjTpo3G8zslJQXfffcdBEGAnZ0d3N3dK103oP78dnFxwfbt25GSkgJTU1NMmjQJ9vb2OHjwIG7evAmJRAJ/f38MGTJE4/5CQ0MRHx+P0aNHIyoqCmfOnMGUKVM0nifh4eHYvXs37OzsIJfLxXquXLmCY8eOoaioCN27d0e/fv00xt2kSROVOFJSUtRelwAgl8uxceNGlffywIEDCA8PBwCMGjUKLVu21HiceXl52L59OxISEmBkZIRp06ZBoVBg8+bNWLRoEYCS3oTS3gJ1cf/444947733YGZmhrt37+LkyZOYMmVKpa7LUuHh4Xj55ZeVyqKiojBixAiEhYWhW7duGreNi4uDlZUVPD09AUC8/gBAJpPB0dERDx48gJeXV7kxVPXaeZqm63Lx4sVYvHgxzMzMsGbNGowdOxZubm6IiopCSEgIJBIJUlJS0KNHD1hYWCAsLAx5eXmQy+V466234OLionGfVWnvmog9HHqWmZmJuXPnon///vjrr78AAHPmzEGvXr0wcOBAsWcEAHbs2IGJEyciMDAQ1tbWCAsLQ0pKCi5evIjAwED06dMHubm5SvXfv38fc+fOxQcffABXV1c0adIE7733HhYtWgQHBwdcunQJjo6OCAoKgpeXF6ZMmYKgoCDY2dkBAKZNm4ZevXop1Xn58mUYGRlh8eLF6NatG37//XdxmaWlJQIDA+Hg4CB+8Gkye/ZsBAUFoXPnznjuueeqVLe6NtGktKdJ3YdOZeKuTN2///47nnvuObz33nsoKipSWubj44MlS5bg1q1byM7ORocOHRAUFARzc3MEBQWVm2yUMjMzQ2BgIJydncWu5SFDhih9wyuVnJyM1157DbNmzcLx48eVll27dg3Xr1/HhAkTKp1sAMC4ceMQFBSE//znP9i9e7dYru78/v3339G3b18sXLhQ47CSNnUDquf34cOH0b59eyxatAj9+/fH4cOHAQC9e/dGUFAQFi1ahFu3blV5SE3defLzzz9j1qxZmDJlCp48eSIe98mTJ/Huu+8iKCgIV65cQWpqqsa41SnvulT3Xt64cQO5ubkIDAzErFmzVNqqrN9//x0eHh54//33MXv2bK2S7LJx+/r64vbt2+L+y/usKk9ycjKsra0hlf7vT05qaipMTU3RqlUr3L17F4IglLu9o6OjxuWenp64d+9ehcdXlWtHHXXXZXnu3LmDV155BUuWLEH37t0BAGlpaZg3bx4CAgJw8ODBcrevbHvXVOzh0LMmTZpAKpXC2dm53LHfvLw8xMXFYePGjQCAwsJCWFpaQiaToVmzZpDJZGjatClkMpnSdv7+/uIHibm5OZKTk/H9998jNzcX2dnZKsmENh48eABfX18AQJs2bXDq1ClxWbNmzQAAzs7OyMjIqLCuuLg43Lx5E3Pnzq103ZrapCoqG7e2YmNjMWLECAAlCUZhYaHSPo2MjODg4ICsrCxYWVlVun4fHx/x36tXr5a7rpOTE2xsbGBtba10jCkpKfjhhx8wc+bMKn8rOn36tPgH+Ok/rOrO7wcPHuCll16CRCJBq1atlNqkMnUDqud3VFQUIiIicOzYMSgUCjg4OAAo+UA/fvw4FAoFUlJSkJ6eXqWhr7LnSU5ODoyMjMQ/do0bNwZQMtcnMTFR7B3Iy8tDcnKyGE/ZuCtL3XsZFRWF8PBwBAcHAwByc3NRVFQEY2P1H+NRUVFij0npdVNRAlg27nbt2uH8+fNo3749/vnnHwwaNKhK12VGRobKOhEREWjevDlkMhkcHBzw6NEjeHh4qCTE2iTIVlZWSElJqXA9TTRdO5pU5roESoZqS89HCwsLAP/7fPDx8cGePXs0bqvLz0FDY8KhZ6Uf8BKJpNwMHgDs7OwQFBSkVFbRyWxmZqb0evfu3Rg6dCh8fHxw7NixCj/sK+vpP1gVHU9hYSG2b9+Ot956S6s/dOrqVtcmVVGZuAHtPuQqUvptTpv3Xhc0nWvGxsaYPHky9u7di4ULF1Y66YiMjERsbCwWLlwoDjVUtE9d1A2ont8SiQQzZ86Evb29WCaXy7F3714EBgbCxsYGX375ZbmxPP3eKhQKpWXanicSiQR+fn5444031C4vG3dlaWrXgICAcrv7y8ZYXllxcbHK8rJxN2vWDDt37kRCQgJsbGxgYWGBvLy8Sl+XMplMpQcwIiICd+7cwY0bN5CTk4OIiAh4eHgo9YIAEBO+8hIKuVxe6WHSsvsAnu1aLe+8qijpfHqf6t43XX0OGhqHVAzE3NxcaXjE3NwcZmZm4rfErKwspKSkwMvLC3fu3IFcLsfdu3eVxpDVyc/Ph729PYqKinDt2jWlZWZmZsjJyakwNi8vL7Eb9Z9//qlwXFSTvXv3olevXnBycqpS3ZrapDpYWFho9U2nUaNG4vFoM/ENKPlAKfvhq0lkZKT4b1XfB1tbW/j4+KBp06ZKE1ZTUlLEb8Dlyc/Ph7W1NYyNjXHjxo0Kz8FGjRohIiICgiCIbaOrups3by52Ycvlcjx8+BByuRxSqRSWlpZIT0+vsGvdxsZGfG8fPnxY7rqWlpZQKBRITU1FXl4eYmNjxWOMjIwUJ3s+evSowtg10fa6bNGiBS5evCgmChVNOm/evDnOnz8PoORbcm5uLqysrMR5VY8ePapwn1KpFN7e3jhw4IA4nFKV69LJyUlpHUEQEB0djaVLl2Lp0qWYOnWqeP1YWVkhLS0NAPD48WM4OzvD09MTmZmZiIuLA1Ayp+FpSUlJ5c6B0DV112XpeSWXy5GQkFBhHXfu3EFxcTEiIiKUru2ynz0VtfeWLVtUPutrKvZw6EDphMLU1FRER0fj6tWrFd6p0q5dO2zatAnh4eEYMmQIWrdujYkTJ2Lnzp3Iy8uDkZERxo0bBy8vL3Tp0gUrVqyAl5cXrK2ty613yJAh+Oqrr2BpaakydtyzZ0/s3LkTlpaWePvtt5GdnY2tW7ciMzNTnMA6ffp0dOrUCbdv38by5cvFiZ1V8ddff8Hd3R1nz56Fqakp3n333UrXra5NNI3lhoaG4tSpU+IkPAcHB8yePbtKsQ8YMABfffUVGjRogNmzZ2use9iwYfj2229x5swZ2NjYaOzeflqvXr3wySefwNnZWZy4pklubi5WrFgBW1tbDB8+HACwatUq5OTkiLdjDx06VJxMV54XX3wRK1asQPv27eHs7Iy0tDStkhhfX1+cOXMGH3/8MZo2bQobG5ty1x86dCi+++47nDx5UqknQhd1Dxs2DNu3b8fy5cshCAL69+8PDw8PdO7cGR9//DEcHBzEYQ9Nmjdvjj/++APffvut2L1dnldeeQXr16+HnZ0dnJ2dAZQkcaNGjcK6desgCAIsLS21St7UKXtdauLn54fY2FisWLECxcXF8PHxKfdYS9vqv//9L2QyGaZMmQILCwu0b98e69atE4ePKuLn54dNmzbhlVdeEcsqc10CJUlV6R9kW1tbPHz4EA4ODjA1NQVQMlT14MEDyOVyvPLKK/j6669RXFyM1q1bi+fo+PHjsX37dhQUFMDc3BwLFy4U64+NjdXq7o6qXjtlqbsuX3jhBXz//fdo3LixOBenPHZ2duJE+smTJ4vlrVq1wvHjx7F8+XIEBATA39+/3PZOS0tDXl5epY/BECRCdfT10jPJz8+HmZkZkpOTsWnTpjrRtVZXFBYWwtjYGFKpFNu2bYOfnx/8/f11Uve2bdvQvn17tG3bVif1lXX06FE4Ojqic+fOeqmf6GmXL19GWloaBgwYoNN6Hz9+jD/++KPKX4wqSxfX5dN3StUn7OGoBX755Rc8ePAAUqlU5bYyMqz4+Hj8+OOPkEqlcHd3F7uda4PBgwcbOgSqRzp06ICLFy/qvN6srKw6/eyKuoQ9HERERKR3nDRKREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHDqwefNmBAcHY/78+ViyZAmCg4Px999/IzQ0FHv37tW43Zo1a6oxSt2pbXFv27YN77//vtr3IjY2Fvv27dPJfk6fPo1Dhw5pXJ6SkoJPPvlEJ/sKDQ1FVlaWXuquDWryOZiSkoKwsDClsm3btuHmzZuVqufYsWNar3vjxg3MmDED6enpYtmCBQvE/9+7dy9CQ0ORkpKCOXPmIDg4GB999BFOnDgBAAgPD0dwcDBmz56N5cuX48yZM0hJScGMGTNw5coVACW/Wr148WIAwJ07dxAcHIzly5fj/PnzWsX4008/ISkpSetjUufs2bO4fPnyM9Wxbds2LFmyRKme8+fPY+nSpfjoo49w//79KtX7559/YvHixUrtDpS8N0uXLsXmzZufJew6gQmHDkyZMgVBQUHw8/PDmDFjEBQUhJ49e1a43TvvvFMN0elebYt7woQJGn++ulGjRhg1alQ1R/Tszp8/r5Rw1Dc1+RxMSUkR/0g/i5CQEK3XvX37Nho3boyIiIgK13V1dUVQUBAWLlyIs2fPIjk5Gb6+vggKCoKtrS3mz5+PPn36AAAaNmyI27dvAwCSk5MBAIIg4Mcff8S0adOwYMECHDlyBNnZ2RXu94033kDDhg21PiZ1nnvuOXTq1OmZ6gCAMWPGiPXI5XIcPnwY7733HiZOnFjlLyCNGzfGvHnzVMr9/Pwwbty4Z4q3rjA2dAB1XUJCAj7//HOkp6dj+vTpcHNzQ3h4OPbv34+0tDR8+umn4rqRkZHYvXs3JBIJXFxcMHnyZI31JiUlYcuWLSgqKoKZmZnSxbxmzRo0a9YMN2/ehFwux/vvv4/ExETs2rULBQUFcHV1xfjx42FkZIQNGzYgIyMDUqkUAQEBaNOmjcZ96iLu0NBQXL16Ffn5+Uptkpqaii1btiA3Nxfu7u4YP348jI1LTs8FCxagR48e+Oeff+Dq6gpnZ2fcu3cPWVlZ8PLywsOHD7Fw4UKxDnVtos6RI0dw/vx5eHh4YMqUKWL5+vXrkZWVBUEQkJqais8++wwAcODAAYSHhwMARo0ahZYtWyIlJQXfffcdBEGAnZ0d3N3dNe4PAAoKCvD1118jMTERw4cPR4cOHbBz5060bNkSHTp0gEKhwLJly/D+++/DxMREZfuUlBRs3LgRSUlJ2Lx5M2QyGWbMmAGg5INz48aNiIuLw/Dhw9G1a1eNcZenbHtPnjxZ7XlS3nv57bffQhAEWFtbw8vLCwEBAYiPj1d7DqqjUCiwZcsWxMfHAwDGjh2L5s2bazwHDxw4gH/++QcAkJiYiPnz58PDwwNXrlzBsWPHUFRUhO7du6Nfv34ajzs9PR2bN2+GXC6HVCrFf/7zH1hYWCAyMhJ79uyBQqFAt27dMGDAAI117N+/H1evXkV2djaCg4PRtm1bDB8+HABw5coVHDp0CMbGxpgzZw5MTExw8OBB3Lx5ExKJBP7+/hgyZAiuXLmCo0ePIi8vD8HBwbC1tcXMmTPLfc+ioqIwYsQIhIWFoVu3buWuW8rMzAyenp5ISUlBgwYN1K5jbm6OgoICPH78GM7OzoiPj0dycjIsLS3h6OgIAGjWrBnu3bsHPz8/tXVkZ2dj3bp1SEpKwoIFC+Dm5gZAc3trsnHjRsTExGDw4MHo27cvAM2fJ5URExMDLy8vWFhYoFGjRkhLS0NhYaHa6688np6elVq/PmLCoWeZmZlYuHAh/vrrL/z1118YM2YMfH194evrq9L1duLECbzyyito2bJlhd9eraysMHv2bJiZmeGff/7B77//jkmTJonLc3JyEBQUhLy8PEilUuzYsQNvvvkmHB0dsXfvXoSFhaFLly4YN24c7OzskJubi5UrV5abcOgibk1t8vvvv6NHjx7o1q0btmzZgsuXL4sfnLm5ufD09MRLL72E3NxcnDx5Er169UJ0dDSaN28OU1NTPH78GI6OjuW2SVlDhgxB06ZNcebMGaXy2bNnAyjp0s7LywNQ0i2am5uLwMBAZGVl4YsvvsDixYvx+++/o2/fvujcuTPWrVtX4bEnJSVh1qxZkMlkWLVqFdq1a4cuXbrg5MmT6NChA6Kjo+Ht7a3xw87R0RFBQUFYs2YNxo4dK364pqSkIDk5GXPmzEFubi6+++47dO3aVWPc5Snb3gA0niea3svnnnsO3bp1U2oTTeegOg8fPkR6ejoWL16M4uJiFBQUANB8Do4YMQIjRozAvXv3cODAAbi5uSEzMxMnT57Eu+++CyMjI3z22Wdo3749HBwc1O7z8uXLaNWqFQICApCbmyu+Bzt27MCsWbNgZ2eH4OBgtG/fXuMf6JEjR8LX1xdnzpxRSmIBwNLSEoGBgfjuu+8QHh4Of39/9O7dG8OHD4cgCFi9ejU6deqEDh06oEOHDliwYAGCgoLKfa8AIDU1FaampmjVqhV++eUXCIIAiURS4XZZWVmIjY3Fyy+/XO56np6e+P3339G7d29cvXoVmZmZsLKyEpdbW1sjMzNT4/ZWVlbiOfs0Te2tybRp09QOWao7BysjKysL1tbWOHbsGBo1aiQej6b3mKqOCYeeNWnSBFKpFM7OzoiKiip3XW9vbxw6dAhPnjyBv79/uesqFArs2rULT548QVFREWxtbZWWd+7cGUDJN5S8vDzExcVh48aNAIDCwkJYWloCKJl3UPrtNzU1tSqHWKm4AfVtEhsbixEjRgAA2rRpg3v37okJh5GRETp27AgA4jcgS0tL8T8LCwvk5+dX2CaVERcXh5s3b2Lu3LkASr5Blo5zAyV/lIuKivDgwQO89NJLkEgkaNWqFQoLC8ut18nJSfxmaG1tjYyMDDRt2hQ7duxAfn4+wsLCxPeuspycnGBjYyPWW17cpb1H6qhrb03nSUXvpY+PDwoLC8s9B9VxcHBAWloa9u3bBx8fH/j6+lZ4/Pn5+fj5558xdepUSKVSxMTEIDExUewJycvLQ3JyssaEw9PTEzt37oSxsTHatGkDDw8P5OTkQCqVij1lLVq0wMOHD6v0x6hZs2YAAGdnZ/H9uXPnDo4fPw6FQoGUlBSkp6dXetghIiICzZs3h0wmg4ODAx49egQPDw+VpKP09ePHjxEcHCz2VtnZ2ZVbf5s2bXD8+HG88cYblYqrIurauyoq8xlbnoEDB1Z5W9IOEw49K+0ylkgkEASh3HWHDBmCtm3b4tq1a1i1ahWWLVum8Q/DqVOn0LBhQ0yePBkxMTHYv3+/0nJzc3Ol13Z2dirfliIjIxEbG4uFCxeK3bxVUZm4gcq1CQCYmppq/PAs/VehUJTbJtp84ytVWFiI7du346233lLq8g8ICBCHKXTN398fV69exd27dzF27Ngq1aGpXSsbd9n2Lu88qcx7qe4c1MTKygrvv/8+wsPDcejQISQkJOD5558vd5uff/4Z/fv3FxM6iUQCPz8/rf9QtmzZEnPmzMHNmzfx9ddfY/r06RqTk6p4+lwSBAFyuRx79+5FYGAgbGxs8OWXX2p1PZQVERGBO3fu4MaNG8jJyUFERAQ8PDwglSpP0Svdv6urKxYtWqR1/a6urnjvvffE1zY2NkpzNrKzs9GkSZNKx62uvasyLFHZz5OybGxslHpms7KyYGNjU+l6qGKcNFqDJCcnw8PDA0OGDIFEIin323J+fj7s7e0BQGVGfFnm5uYwMzMTs/+srCykpKQgPz8f1tbWMDY2xo0bNyCXy/UetyaNGjUSJ6eFh4fDy8ur0nWU1yYWFhbit8qK7N27F7169YKTk5NY1qJFC1y8eBHFxcUAIM5kb9SoESIiIiAIghh/eRITE5GamoqsrCxkZ2eL3y67dOmCAwcOoHnz5hrnNTzNzMwMOTk5Fa6nKe7KqOx58vR7WTqJUdM5qEl2djYEQUCHDh3Qo0ePctcFSt5vhUKh1DvUqFEjREZGin9MHj16VG7sqampsLW1xXPPPYemTZsiNTUVlpaWEAQBycnJkMvliI6OrvCbuLbvTencBUtLS6Snp+PevXtKyyUSCYqKisqtQxAEREdHY+nSpVi6dCmmTp0qtrmVlRXS0tIAQJyDUVVPJ14NGjRATk4OUlNTUVBQgDt37lQp4VDX3obQuHFjxMXFITc3Fw8ePIC9vb3S8M7ixYufqeekPMuWLVO6s6iuYw+HAezatQv37t0TJ4W1bt0aI0aMwMmTJxEZGQlBENCzZ89yJ1D17t0bmzdvxp9//okWLVpUuM+JEydi586dyMvLg5GREcaNGyeONX/88cdo2rRphVm9LuLWZNiwYdiyZQtCQkLg7u5epZno5bVJq1atcPz4cSxfvhwBAQHw9/fHqlWrkJOTI07wGzp0KPz9/fHXX3/B3d0dZ8+ehampKd599134+fkhNjYWK1asQHFxMXx8fNC4cWMMHToU3333HU6ePCkmO+Vp2LAhdu7cieTkZIwaNUr8Furs7AwrKytxKKMiPXv2xM6dO2FpaYm3335b43qa4q6Myp4nw4YNw7fffoszZ87AxsZG7O1Sdw6W9kaUlZGRga1btwIAZDKZOBdH0zl49uxZZGRkiENHkydPhrOzM0aNGoV169ZBEARYWlqKE2zViY6OxrFjxyCRSODk5CQO47z66qvYuHEjFAoFunfvXuFwiru7O6RSKVatWgU/Pz8MHjxY7XoWFhbo3LkzPv74Yzg4OKi8L7169cInn3wCZ2dnlfkgpR4+fAgHBweYmpoCKPnj+eDBA8jlcrzyyiv4+uuvUVxcjNatW8PLy0tj4lY6GTcjIwOrV69G7969Nc7nkkgkGD9+PL755hsoFAoMHjxYaU5HWaGhoTh16pQ40dnBwQGzZ8/W2N7qPHr0CFu3bkVmZiakUinOnTuH6dOna1y/MmQyGYYMGYKVK1fCyMgIEyZMEJeVzh/SZiLqmTNn8McffyA3NxdBQUF4+eWX0aFDh3K3SUhIEL8M1AcSoSp9UESkU1lZWfjss8+wdOnSSg3/1ESFhYUwNjaGVCrFtm3b4Ofnp9XcHqLqsG3bNrRv3x5t27atcN179+7h/PnzeP31159pn1FRUWonEtc37OEgMrBr165h79694uTT2i4+Ph4//vgjpFIp3N3dNd4uSWQIZmZm2LdvHwoKCirsSW3SpEmVhoueduPGDezfvx9NmzZ9pnrqAvZwEBERkd5x0igRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI75hwEBERkd4x4SAiIiK9Y8JBREREeseEg4iIiPSOCQcRERHpHRMOIiIi0jsmHERERKR3TDiIiIhI7/4f5MmU3+a3oesAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# utility plot functions (from the previous notebook)\n", + "\n", + "\n", + "def fmt_pow10(value: float) -> str:\n", + " \"\"\"Format the power of 10.\"\"\"\n", + " return \"1\" if value == 1 else f\"$10^{{{int(np.log10(value))}}}$\"\n", + "\n", + "\n", + "def plot_pimo_with_auc_zone(\n", + " ax: Axes,\n", + " tpr: ndarray,\n", + " fpr: ndarray,\n", + " lower_bound: float,\n", + " upper_bound: float,\n", + " fpr_in_auc: ndarray,\n", + " tpr_in_auc: ndarray,\n", + ") -> None:\n", + " \"\"\"Helper function to plot the PIMO curve with the AUC zone.\"\"\"\n", + " # plot\n", + " ax.plot(fpr, tpr, linewidth=3.5)\n", + " ax.axvspan(lower_bound, upper_bound, color=\"magenta\", alpha=0.3, zorder=-1)\n", + " ax.fill_between(fpr_in_auc, tpr_in_auc, alpha=1, color=\"tab:purple\", zorder=1)\n", + "\n", + " # config plots\n", + " ax.set_ylabel(\"TPR [%]\")\n", + " ax.yaxis.set_major_locator(FixedLocator(np.linspace(0, 1, 6)))\n", + " ax.yaxis.set_major_formatter(PercentFormatter(1, 0, symbol=\"\"))\n", + " ax.set_ylim(0, 1 + 3e-2)\n", + " ax.set_xlabel(\"FPR\")\n", + " ax.set_xscale(\"log\")\n", + " ax.xaxis.set_major_locator(FixedLocator(np.logspace(-6, 0, 7)))\n", + " ax.xaxis.set_major_formatter(lambda x, _: fmt_pow10(x))\n", + " ax.set_xlim(1e-6 / (eps := (1 + 3e-1)), 1 * eps)\n", + " ax.grid()\n", + "\n", + "\n", + "# simulate a random model's curve\n", + "lower_bound, upper_bound = 1e-5, 1e-4\n", + "threshs_auc_mask = (fpr > lower_bound) & (fpr < upper_bound)\n", + "fpr_in_auc = fpr[threshs_auc_mask]\n", + "tpr_in_auc = tpr[threshs_auc_mask]\n", + "\n", + "fig, ax = plt.subplots(figsize=(6, 4.5))\n", + "plot_pimo_with_auc_zone(ax, tpr, fpr, lower_bound, upper_bound, fpr_in_auc, tpr_in_auc)\n", + "\n", + "fig.text(\n", + " 0.15,\n", + " -0.01,\n", + " \"\"\"\n", + "FPR: Avg. [in-image] False Positive Rate (FPR) on normal images only.\n", + "\n", + "TPR: [in-image] True Positive Rate (TPR), or Recall.\n", + "\n", + "Integration zone in light pink, and area under the curve (AUC) in purple.\n", + "\n", + "This area is normalized by the range size so that AUPIMO is in [0, 1].\n", + "\"\"\",\n", + " ha=\"left\",\n", + " va=\"top\",\n", + " fontsize=\"x-small\",\n", + " color=\"dimgray\",\n", + " font=\"monospace\",\n", + ")\n", + "\n", + "fig.suptitle(\"Random model's PIMO curve\")\n", + "fig # noqa: B018, RUF100" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# AUPIMO Score\n", + "\n", + "Recall that AUPIMO is computed from this integral:\n", + "\n", + "$$\n", + " \\frac{1}{\\log(U/L)}\n", + " \\int_{\\log(L)}^{\\log(U)} \n", + " \\operatorname{TPR}^{i}\\left( \\operatorname{FRP^{-1}}( z ) \\right)\n", + " \\, \n", + " \\mathrm{d}\\log(z) \n", + "$$\n", + "\n", + "where the integration bounds -- $L$[ower] and $U$[pper] -- are the FPR bounds.\n", + "\n", + "By assuming $\\operatorname{TPR}^{i} = \\operatorname{FPR}$, the AUPIMO score only depends on the FPR bounds:\n", + "\n", + "$$\n", + " \\text{AUPIMO of a random model} = \\frac{U - L}{\\log(U/L)}\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "random_model_aupimo(1e-4, 1e-5)=0.004%\n" + ] + } + ], + "source": [ + "def random_model_aupimo(lower_bound: float, upper_bound: float) -> float:\n", + " \"\"\"AUPIMO score obtained by a random model (no class discrimination).\"\"\"\n", + " return (upper_bound - lower_bound) / np.log(upper_bound / lower_bound)\n", + "\n", + "\n", + "print(f\"{random_model_aupimo(1e-4, 1e-5)=:.3%}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice how a random model's AUPIMO score of $0.004%$ is numerically neglegible in the scale up to 100% -- while its AUROC is 50%.\n", + "\n", + "It's easier to interpret the meaning of AUPIMO scores: \n", + "- $0$%: random or worse, \n", + "- $100$%: perfect." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "anomalib-dev", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.14" + }, + "orig_nbformat": 4 + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 307939b758bf7a4f197021980bdb6dbf6517e099 Mon Sep 17 00:00:00 2001 From: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com> Date: Mon, 7 Oct 2024 13:44:31 +0200 Subject: [PATCH 2/3] add cite us Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com> --- .../701d_aupimo_advanced_iii.ipynb | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb b/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb index 01eb4dfc20..6d446d171e 100644 --- a/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb +++ b/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb @@ -314,6 +314,37 @@ "- $0$%: random or worse, \n", "- $100$%: perfect." ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Cite Us\n", + "\n", + "AUPIMO was developed during Google Summer of Code 2023 (GSoC 2023) with the `anomalib` team from OpenVINO Toolkit.\n", + "\n", + "Our work was accepted to the British Machine Vision Conference 2024 (BMVC 2024).\n", + "\n", + "```bibtex\n", + "@misc{bertoldo2024aupimo,\n", + " title={{AUPIMO: Redefining Visual Anomaly Detection Benchmarks with High Speed and Low Tolerance}}, \n", + " author={Joao P. C. Bertoldo and Dick Ameln and Ashwin Vaidya and Samet Akçay},\n", + " year={2024},\n", + " eprint={2401.01984},\n", + " archivePrefix={arXiv},\n", + " primaryClass={cs.CV},\n", + " url={https://arxiv.org/abs/2401.01984}, \n", + "}\n", + "```\n", + "\n", + "Paper on arXiv: [arxiv.org/abs/2401.01984](https://arxiv.org/abs/2401.01984) (accepted to BMVC 2024)\n", + "\n", + "Medium post: [medium.com/p/c653ac30e802](https://medium.com/p/c653ac30e802)\n", + "\n", + "Official repository: [github.com/jpcbertoldo/aupimo](https://github.com/jpcbertoldo/aupimo) (numpy-only API and numba-accelerated versions available)\n", + "\n", + "GSoC 2023 page: [summerofcode.withgoogle.com/archive/2023/projects/SPMopugd](https://summerofcode.withgoogle.com/archive/2023/projects/SPMopugd)" + ] } ], "metadata": { From 31e0dd244dacd2fb4d606f271fbbaa08d3ad9474 Mon Sep 17 00:00:00 2001 From: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com> Date: Wed, 9 Oct 2024 14:14:10 +0200 Subject: [PATCH 3/3] update notebooks readme Signed-off-by: jpcbertoldo <24547377+jpcbertoldo@users.noreply.github.com> --- notebooks/README.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/notebooks/README.md b/notebooks/README.md index 8d8724a228..15935b93cf 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -54,8 +54,9 @@ To install Python, Git and other required tools, [OpenVINO Notebooks](https://gi ## 7. Metrics -| Notebook | GitHub | Colab | -| ----------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| AUPIMO basics | [701a_aupimo](/notebooks/700_metrics/701a_aupimo.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701a_aupimo.ipynb) | -| AUPIMO representative samples and visualization | [701b_aupimo_advanced_i](/notebooks/700_metrics/701b_aupimo_advanced_i.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701b_aupimo_advanced_i.ipynb) | -| PIMO curve and integration bounds | [701c_aupimo_advanced_ii](/notebooks/700_metrics/701c_aupimo_advanced_ii.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701c_aupimo_advanced_ii.ipynb) | +| Notebook | GitHub | Colab | +| ----------------------------------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| AUPIMO basics | [701a_aupimo](/notebooks/700_metrics/701a_aupimo.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701a_aupimo.ipynb) | +| AUPIMO representative samples and visualization | [701b_aupimo_advanced_i](/notebooks/700_metrics/701b_aupimo_advanced_i.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701b_aupimo_advanced_i.ipynb) | +| PIMO curve and integration bounds | [701c_aupimo_advanced_ii](/notebooks/700_metrics/701c_aupimo_advanced_ii.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701c_aupimo_advanced_ii.ipynb) | +| (AU)PIMO of a random model | [701d_aupimo_advanced_iii](/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701d_aupimo_advanced_iii.ipynb) |