i.i.d order statistics and extreme value theory - advice on whereasymptotic normal ends and nonnorma

Discussion in 'Math Research' started by jdm, Dec 30, 2011.

  1. jdm

    jdm Guest

    I've been studying some cryptographic research in which the asymptotic
    normal distribution of the empirical sample quartile of order q is
    used to construct statistical models of the amount of data required
    for a successful cryptanalysis.

    The main issue I have is that, while I'm pretty sure that such models
    have continued to be used for order statistics X_i (with i near to n)
    where the asymptotic normal distribution is inaccurate and where
    something based on extreme-value theory for the mth extremes would
    have been better, I don't have any idea as to how to compute an
    estimate for the value of i (or indeed q) above which the asymptotic
    normal might be considered suspect.

    As an example, I'm currently dealing with the situation X_1 <=
    X_2 ...<= X_n, where n = 2^{41}-1 = 2,199,023,255,551. In particular,
    I'm trying to work out whether the asymptotic normal is likely to be
    adequate when drawing conclusions about the top 2^{17} = 131,072
    values or not - and while this seems a high m for m-th-extreme, it's
    not so high in relation to n, and this would mean I was dealing with
    the top 0.000006388% of values.

    Can anyone give me some advice here?

    Many thanks,

    James McLaughlin.
     
    jdm, Dec 30, 2011
    #1
    1. Advertisements

  2. jdm

    Herman Rubin Guest

    If one is sampling from the uniform distribution, one only
    needs to avoid the extremes. This is the case for other
    reasonable distributions as well; the rate of convergence
    to the normal distribution, as found by calculus, in general
    needs only that there are large numbers on both sides, and
    not on anything else.
     
    Herman Rubin, Dec 31, 2011
    #2
    1. Advertisements

  3. jdm

    jdm Guest

    If one is sampling from the uniform distribution, one only
    Does the chi-squared distribution count as a "reasonable distribution"
    for these purposes?

    Moreover, if "avoid the extremes" means avoiding only the asymptotic
    normal distribution for X_1 and X_N, but not for instance X_2 and
    X_{N-1}, this would appear to contradict H.A. David stating in "Order
    Statistics" that the extremes and mth extremes X_m, X_{N-m+1} are non-
    Normally distributed. (albeit with little information on how, given N,
    to identify the values of m for which this is the case, except the
    statement

    "If r/n -> \lambda as n-> infinity, fundamentally different results
    (regarding the distribution of order statistic X_r) are obtained
    according as
    (a) 0 < \lambda < 1, or
    (b) \lambda = 0 or 1

    with r or N-r fixed ... The latter case includes the extremes X_1,
    X_N, and corresponds to the mth extremes X_m, X_{N-m+1} with m fixed.
    These have nonnormal limiting distributions.")

    (taken from the opening paragraphs of Section 9.1)
     
    jdm, Dec 31, 2011
    #3
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.