Change point detection in count data

Discussion in 'Scientific Statistics Math' started by SlideRuleGuy, Dec 5, 2011.

  1. SlideRuleGuy

    SlideRuleGuy Guest

    I have a situation I have not yet dealt with, which involves count
    data, where a total count is reported each day. But the counts are
    small (the range 0-5 brackets about 75% of the points), and vary by
    day of the week. In the simplest case the counts are Poisson, where
    the mean differs by day of the week. (These are a particular type of
    customer arrival.)

    I would like to perform change point detection, where a change in the
    arrival rate occurs at some point in time and subsequently affects all
    days of the week. (In other words, a change won’t _just_ affect
    Mondays for example.) I am considered using the EWMA method described
    in the thesis at http://smartech.gatech.edu/handle/1853/34828, but
    with this change point detection method, the pre-change time series is
    assumed to have a _single_ mean, not 7 different means.

    I can see two ways to approach this: 1) Remove the
    “seasonality” (i.e., the day of the week variation), and then apply
    change point detection using a single threshold. This appears
    problematic, because the counts are too small to treat as continuous
    data for removal of seasonality. Or 2) obtain 7 different change
    point detection thresholds from the data for each respective day of
    the week. This is a concern, as some days of the week inherently
    provide less information than other days, due to especially small
    counts.

    I can of course run some simulations to try these, but was wondering
    if there is a standard approach I have not run across. Thanks!
     
    SlideRuleGuy, Dec 5, 2011
    #1
    1. Advertisements

  2. SlideRuleGuy

    David Duffy Guest

    This reminded me of disease surveillance, where there is a sizable literature
    eg

    http://biostatistics.oxfordjournals.org/content/7/3/422.long
     
    David Duffy, Dec 5, 2011
    #2
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.