Which test?

Discussion in 'SPSS' started by santc, Nov 10, 2011.

  1. santc

    santc Guest

    I have done a study which first measure the psychological
    characteristics of 130 individuals at time 1 and again measured the
    same characteristics in Time 2 after an intervention. My objectives
    are 1) see whether any significant difference in scores between time 1
    and 2. And 2) I want to see is there any significant difference in
    scores between time 1 and 2 by categories like male/female,
    occupation, education etc.

    For my first objective I think I can use t-test. Can anybody suggest
    which test should I use for my second objective. Can I use separate t-
    tests for each category??
    santc, Nov 10, 2011
    1. Advertisements

  2. santc

    Rich Ulrich Guest

    Either there is an overall effect of treatment, or there isn't.

    If there isn't, you don't have much to brag about, at all,
    and you are stuck with trying to scavenge something
    "suggestive" for further research. Looking at the outside
    categories separately, by simple t-tests or ANOVA, will
    let you readily describe and discuss any extreme means.
    If that helps. If you want to do testing that is more
    precise, you might proceed the same is if an effect had
    been demonstrated.

    If there *is* an effect, then you certainly can't place any
    emphaisis on separate tests for subgroups -- with smaller
    N, the separate tests will have less power. (It is possible
    that *neither* gender (say) shows the effect, if it requires
    the full N to have sufficient power.) It is never a proper
    statistical inference to claim that two groups are different
    because, "This one has a significant effect and that one
    does not."

    If you want to claim that the change in group1 is different
    from the change in group2, you can test the change-score
    directly. Or you can check the outcome at period 2
    directly, ignoring period 1. Or you can check the "regressed
    change" to period 2. This potentially gets tricky to decide
    between -- there is much literature on testing of "change",
    not all of it competent. The problems mainly arise when
    the groups differ at Pre. I can imagine that this easily
    could be the case for your data, since you say that it is
    "psychological data" and your extra variables include
    sex, education, and occupation.

    So you have three choices for tests, and none of them
    protect you against invalid inference when the Pre levels
    differ. Another problem that can complicate matters is
    bad scaling, especially when you may have "basement"
    or "ceiling" effects on scales.

    How do you proceed? That depends on how serious the
    problems are. You are trying to make a convincing argument,
    and you have to consider the criticisms that are based on
    the different problems. You can test all three ways,
    and if results differ, figure out why. If results don't differ,
    that gives a better start for a convincing argument.

    Hope this helps.
    Rich Ulrich, Nov 10, 2011
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.