{
    "nbformat": 4,
    "nbformat_minor": 5,
    "metadata": {
        "kernelspec": {
            "display_name": "Python",
            "language": "python",
            "name": "python"
        },
        "language_info": {
            "name": "python"
        }
    },
    "cells": [
        {
            "id": "3cd083ab",
            "cell_type": "markdown",
            "source": "<div class='alert alert-warning'>\n\nSciPy's interactive examples with Jupyterlite are experimental and may not always work as expected. Execution of cells containing imports may result in large downloads (up to 60MB of content for the first import from SciPy). Load times when importing from SciPy may take roughly 10-20 seconds. If you notice any problems, feel free to open an [issue](https://github.com/scipy/scipy/issues/new/choose).\n\n</div>",
            "metadata": {}
        },
        {
            "id": "8f9cfc0d",
            "cell_type": "markdown",
            "source": "We follow the example from [1].\n\n    Thrombolysis with recombinant tissue-type plasminogen activator (rt-PA)\n    and anisoylated plasminogen streptokinase activator (APSAC) in\n    myocardial infarction has been proved to reduce mortality. [4]\n    investigated the effects of a new front-loaded administration of rt-PA\n    versus those obtained with a standard regimen of APSAC, in a randomized\n    multicentre trial in 421 patients with acute myocardial infarction.\n\nThere were four families of hypotheses tested in the study, the last of\nwhich was \"cardiac and other events after the start of thrombolitic\ntreatment\". FDR control may be desired in this family of hypotheses\nbecause it would not be appropriate to conclude that the front-loaded\ntreatment is better if it is merely equivalent to the previous treatment.\n\nThe p-values corresponding with the 15 hypotheses in this family were\n",
            "metadata": {}
        },
        {
            "id": "654268ad",
            "cell_type": "code",
            "metadata": {},
            "execution_count": null,
            "source": "ps = [0.0001, 0.0004, 0.0019, 0.0095, 0.0201, 0.0278, 0.0298, 0.0344,\n      0.0459, 0.3240, 0.4262, 0.5719, 0.6528, 0.7590, 1.000]",
            "outputs": []
        },
        {
            "id": "c2ae059c",
            "cell_type": "markdown",
            "source": "If the chosen significance level is 0.05, we may be tempted to reject the\nnull hypotheses for the tests corresponding with the first nine p-values,\nas the first nine p-values fall below the chosen significance level.\nHowever, this would ignore the problem of \"multiplicity\": if we fail to\ncorrect for the fact that multiple comparisons are being performed, we\nare more likely to incorrectly reject true null hypotheses.\n\nOne approach to the multiplicity problem is to control the family-wise\nerror rate (FWER), that is, the rate at which the null hypothesis is\nrejected when it is actually true. A common procedure of this kind is the\nBonferroni correction [1].  We begin by multiplying the p-values by the\nnumber of hypotheses tested.\n",
            "metadata": {}
        },
        {
            "id": "d541a098",
            "cell_type": "code",
            "metadata": {},
            "execution_count": null,
            "source": "import numpy as np\nnp.array(ps) * len(ps)",
            "outputs": [
                {
                    "output_type": "execute_result",
                    "metadata": {},
                    "data": {
                        "text/plain": "array([1.5000e-03, 6.0000e-03, 2.8500e-02, 1.4250e-01, 3.0150e-01,\n       4.1700e-01, 4.4700e-01, 5.1600e-01, 6.8850e-01, 4.8600e+00,\n       6.3930e+00, 8.5785e+00, 9.7920e+00, 1.1385e+01, 1.5000e+01])"
                    },
                    "execution_count": null
                }
            ]
        },
        {
            "id": "b8391797",
            "cell_type": "markdown",
            "source": "To control the FWER at 5%, we reject only the hypotheses corresponding\nwith adjusted p-values less than 0.05. In this case, only the hypotheses\ncorresponding with the first three p-values can be rejected. According to\n[1], these three hypotheses concerned \"allergic reaction\" and \"two\ndifferent aspects of bleeding.\"\n\nAn alternative approach is to control the false discovery rate: the\nexpected fraction of rejected null hypotheses that are actually true. The\nadvantage of this approach is that it typically affords greater power: an\nincreased rate of rejecting the null hypothesis when it is indeed false. To\ncontrol the false discovery rate at 5%, we apply the Benjamini-Hochberg\np-value adjustment.\n",
            "metadata": {}
        },
        {
            "id": "2335e33e",
            "cell_type": "code",
            "metadata": {},
            "execution_count": null,
            "source": "from scipy import stats\nstats.false_discovery_control(ps)",
            "outputs": [
                {
                    "output_type": "execute_result",
                    "metadata": {},
                    "data": {
                        "text/plain": "array([0.0015    , 0.003     , 0.0095    , 0.035625  , 0.0603    ,\n       0.06385714, 0.06385714, 0.0645    , 0.0765    , 0.486     ,\n       0.58118182, 0.714875  , 0.75323077, 0.81321429, 1.        ])"
                    },
                    "execution_count": null
                }
            ]
        },
        {
            "id": "b0d8f185",
            "cell_type": "markdown",
            "source": "Now, the first *four* adjusted p-values fall below 0.05, so we would reject\nthe null hypotheses corresponding with these *four* p-values. Rejection\nof the fourth null hypothesis was particularly important to the original\nstudy as it led to the conclusion that the new treatment had a\n\"substantially lower in-hospital mortality rate.\"",
            "metadata": {}
        }
    ]
}