{
    "nbformat": 4,
    "nbformat_minor": 5,
    "metadata": {
        "kernelspec": {
            "display_name": "Python",
            "language": "python",
            "name": "python"
        },
        "language_info": {
            "name": "python"
        }
    },
    "cells": [
        {
            "id": "7e0394ae",
            "cell_type": "markdown",
            "source": "<div class='alert alert-warning'>\n\nSciPy's interactive examples with Jupyterlite are experimental and may not always work as expected. Execution of cells containing imports may result in large downloads (up to 60MB of content for the first import from SciPy). Load times when importing from SciPy may take roughly 10-20 seconds. If you notice any problems, feel free to open an [issue](https://github.com/scipy/scipy/issues/new/choose).\n\n</div>",
            "metadata": {}
        },
        {
            "id": "c5c9c5b7",
            "cell_type": "code",
            "metadata": {},
            "execution_count": null,
            "source": "import numpy as np\nfrom scipy.stats import differential_entropy, norm",
            "outputs": []
        },
        {
            "id": "ba871cee",
            "cell_type": "markdown",
            "source": "Entropy of a standard normal distribution:\n",
            "metadata": {}
        },
        {
            "id": "4303f0b3",
            "cell_type": "code",
            "metadata": {},
            "execution_count": null,
            "source": "rng = np.random.default_rng()\nvalues = rng.standard_normal(100)\ndifferential_entropy(values)",
            "outputs": [
                {
                    "output_type": "execute_result",
                    "metadata": {},
                    "data": {
                        "text/plain": "1.3407817436640392"
                    },
                    "execution_count": null
                }
            ]
        },
        {
            "id": "0a275f98",
            "cell_type": "markdown",
            "source": "Compare with the true entropy:\n",
            "metadata": {}
        },
        {
            "id": "a404d1d2",
            "cell_type": "code",
            "metadata": {},
            "execution_count": null,
            "source": "float(norm.entropy())",
            "outputs": [
                {
                    "output_type": "execute_result",
                    "metadata": {},
                    "data": {
                        "text/plain": "1.4189385332046727"
                    },
                    "execution_count": null
                }
            ]
        },
        {
            "id": "130462f2",
            "cell_type": "markdown",
            "source": "For several sample sizes between 5 and 1000, compare the accuracy of\nthe ``'vasicek'``, ``'van es'``, and ``'ebrahimi'`` methods. Specifically,\ncompare the root mean squared error (over 1000 trials) between the estimate\nand the true differential entropy of the distribution.\n",
            "metadata": {}
        },
        {
            "id": "349523cd",
            "cell_type": "code",
            "metadata": {},
            "execution_count": null,
            "source": "from scipy import stats\nimport matplotlib.pyplot as plt\n\n\ndef rmse(res, expected):\n    '''Root mean squared error'''\n    return np.sqrt(np.mean((res - expected)**2))\n\n\na, b = np.log10(5), np.log10(1000)\nns = np.round(np.logspace(a, b, 10)).astype(int)\nreps = 1000  # number of repetitions for each sample size\nexpected = stats.expon.entropy()\n\nmethod_errors = {'vasicek': [], 'van es': [], 'ebrahimi': []}\nfor method in method_errors:\n    for n in ns:\n       rvs = stats.expon.rvs(size=(reps, n), random_state=rng)\n       res = stats.differential_entropy(rvs, method=method, axis=-1)\n       error = rmse(res, expected)\n       method_errors[method].append(error)\n\nfor method, errors in method_errors.items():\n    plt.loglog(ns, errors, label=method)\n\nplt.legend()\nplt.xlabel('sample size')\nplt.ylabel('RMSE (1000 trials)')\nplt.title('Entropy Estimator Error (Exponential Distribution)')",
            "outputs": []
        }
    ]
}