t-tests of adjusted means after an anova with within- and between-subjects factors?

Discussion in 'SPSS' started by martin knollmann, Jan 26, 2005.

  1. hi there,
    does anyone know how to compare group means that have been adjusted for
    a covariate? i would like to run t-tests to interpret an interaction in
    my 2x2-design (1 within-factor with two levels, 1 between-factor with
    two levels, 1 covariate, 1 dv).
    My first approach was to adjust the individual raw values in the dv
    using a regression (covariate-dv)and then to calculate the means and to
    compare them, but it didn't work. now i have no idea what to
    do....somebody please help!!!
    martin
     
    martin knollmann, Jan 26, 2005
    #1
    1. Advertisements

  2. martin knollmann

    Bruce Weaver Guest

    You may have to use MANOVA rather than GLM. I don't have an example for
    a split-plot design like yours, but here's one for a two-way design with
    both factors between-subjects.

    MANOVA
    y BY a(1 2) b(1 3)
    /NOPRINT PARAM(ESTIM)
    /METHOD=UNIQUE
    /ERROR within+residual
    /DESIGN= a, b W a(1), b W a(2).

    This will give simple main effects of B at each level of A. I expect
    you can do something similar for your design. MANOVA is not available
    via the pull-down dialogs, so you'll have to look it up in the help
    files and write your own code.
     
    Bruce Weaver, Jan 26, 2005
    #2
    1. Advertisements

  3. thanks for the fast answer! i will try using manova, although it will
    take some time because i'm not that familiar with the spss-syntax yet;
    unfortunately i've always worked with the pull-down dialogs :) another
    problem is that i'd like to do all possible comparisons with the
    adjusted means (group1: condition1 vs. condition2, group2: condition1 vs
    condition2, condition1: group1 vs. group2, condition2: group1 vs. group2).
    isnt' there a way to build new variables by reconstructing the
    adjustment of the dv made by the covariate and then run normal t-tests
    (for dependent & independent samples) with these variables? a colleague
    told me that saving the residuals of the regressions: covariate (iv) -->
    dv and then comparing the means of the residuals would do the job. yet
    another colleague told me to use the betas of the above mentioned
    regressions to build "adjusted variables" with a calculation like this:
    ADJUSTED INDIVIDUAL SCORE DV = INDIVIDUAL SCORE DV - (BETA * INDIVIDUAL
    SCORE COVARIATE). by now i'm totally confused since both approaches
    didn't work, but i would really prefer to use some kind of "adjusted
    variables" and compare their means, because i never worked with the spss
    syntax so far. do you know any easier ways for an spss-beginner like me?
    thanks again,
    martin
    ps: maybe it is helpful when i describe my study in more detail: 2
    groups (defined by extreme motivational aptitudes), each group was
    exposed to 2 conditions (2 different instructional methods), each group
    reported their emotions (dv) in each condition, self-concept as a
    covariate to control for differences in emotions due to different
    ability-related beliefs
     
    martin knollmann, Jan 26, 2005
    #3
  4. martin knollmann

    jim clark Guest

    Hi

    Without considering the covariate, the following would illustrate
    MANOVA simple effects analyses for mixed (split-plot) designs.

    manova y1 y2 BY a(1 2) /WSF = B(2)
    /wsd = b /design = mwithin a(1) mwithin a(2).

    manova y1 y2 BY a(1 2) /WSF = B(2)
    /wsd = mwithin b(1) mwithin b(2) /design = a.

    I'm not certain with these examples, but I have noticed
    occasionally designs where you cannot in fact include all of the
    called-for effects (e.g., an effect for each of the 3 dfs in the
    above examples).

    Best wishes
    Jim

    ============================================================================
    James M. Clark (204) 786-9757
    Department of Psychology (204) 774-4134 Fax
    University of Winnipeg 4L05D
    Winnipeg, Manitoba R3B 2E9
    CANADA http://www.uwinnipeg.ca/~clark
    ============================================================================
     
    jim clark, Jan 26, 2005
    #4
  5. martin knollmann

    Bruce Weaver Guest

    ---- snip -----

    Just a quick question: Do you mean you that based on scores from a
    motivational aptitude scale, you sorted people into two groups (i.e.,
    group 1 scored higher than some cutoff, and group 2 lower than the same
    or a different cutoff)? If so, you might consider using the actual
    scores as another covariate rather than as a basis for sorting folks
    into extreme groups. Here is a summary of some posts on that issue from
    a year or two ago:

    http://core.ecu.edu/psyc/wuenschk/StatHelp/Dichot-Not.doc
     
    Bruce Weaver, Jan 26, 2005
    #5
  6. i heard about the problems of median splits and the like; in fact, i
    used upper and lower tertiles to form my groups. however, this has been
    done for quite important theoretical reasons (the interaction i wanted
    to test - an aptitude-treatment-interaction - is only supposed for
    extreme aptitudes rather than for the whole range of the aptitude) so i
    decided intentionally to accept less statistical power and variance. of
    course i could form the groups by using TWO cutoffs (motivation AND
    self-concept) and integrate no covariate at all into the model, but that
    would lead to a really serious sample-size problem :) . i had no
    problems performing the necessary computations so far - i just can't
    handle the post-hoc interpretation of my interaction, because i don't
    know how to compare the 4 adjusted means. i think that you can compare
    adjusted means with t-tests, because i found an article (in german,
    unfortunately) with a design similar to mine (a training study; they
    used the pretest-scores as a covariate) but i have no idea how to do
    this. what do you think about the idea of my colleagues (building new
    variables by reconstruction of the covariance adjustment -->
    dv-variables with "adjusted individual scores" --> compare the means of
    these variables with t-tests)? is something like this possible at all?
    thanks,
    martin
     
    Martin Knollmann, Jan 26, 2005
    #6
  7. i heard about the problems of median splits and the like; in fact, i
    used upper and lower tertiles to form my groups. however, this has been
    done for quite important theoretical reasons (the interaction i wanted
    to test - an aptitude-treatment-interaction - is only supposed for
    extreme aptitudes rather than for the whole range of the aptitude) so i
    decided intentionally to accept less statistical power and variance. of
    course i could form the groups by using TWO cutoffs (motivation AND
    self-concept) and integrate no covariate at all into the model, but that
    would lead to a really serious sample-size problem . i had no problems
    performing the necessary computations so far - i just can't handle the
    post-hoc interpretation of my interaction, because i don't know how to
    compare the 4 adjusted means. i think that you can compare adjusted
    means with t-tests, because i found an article (in german,
    unfortunately) with a design similar to mine (a training study; they
    used the pretest-scores as a covariate) but i have no idea how to do
    this. what do you think about the idea of my colleagues (building new
    variables by reconstruction of the covariance adjustment -->
    dv-variables with "adjusted individual scores" --> compare the means of
    these variables with t-tests)? is something like this possible at all?
    thanks,
    martin
     
    Martin Knollmann, Jan 26, 2005
    #7
  8. i heard about the problems of median splits and the like; in fact, i
    used upper and lower tertiles to form my groups. however, this has been
    done for quite important theoretical reasons (the interaction i wanted
    to test - an aptitude-treatment-interaction - is only supposed for
    extreme aptitudes rather than for the whole range of the aptitude) so i
    decided intentionally to accept less statistical power and variance. of
    course i could form the groups by using TWO cutoffs (motivation AND
    self-concept) and integrate no covariate at all into the model, but that
    would lead to a really serious sample-size problem . i had no problems
    performing the necessary computations so far - i just can't handle the
    post-hoc interpretation of my interaction, because i don't know how to
    compare the 4 adjusted means. i think that you can compare adjusted
    means with t-tests, because i found an article (in german,
    unfortunately) with a design similar to mine (a training study; they
    used the pretest-scores as a covariate) but i have no idea how to do
    this. what do you think about the idea of my colleagues (building new
    variables by reconstruction of the covariance adjustment -->
    dv-variables with "adjusted individual scores" --> compare the means of
    these variables with t-tests)? is something like this possible at all?
    thanks,
    martin
     
    Martin Knollmann, Jan 26, 2005
    #8
  9. martin knollmann

    Art Kendall Guest

    Did I misread this thread? if there is a 2*2 design, there is only one
    "difference of differences", i.e., 1 df. the t is the square root of F.
    sketch out the results.
    use the DV on the Y-axis (vertical).
    Put the within subject group factor on the X-axis (horizontal).
    Plot the 4 adjusted means.
    draw a line segment between each pair of repeats (one line for each
    level of the between group factor).

    If the F for interaction is significant the lines depart from parallel
    to a degree that is not compatible with random variation.

    Seriously consider leaving the between subjects factor continuous and
    use a "regression approach". remember to center the between IV and
    "covariate" before creating the interaction terms. bends are created by
    using power terms square for 1 bend cube for 2 bends.



    see:
    Cohen, Jacob, et al (2003) Applied multiple regression/correlation
    analysis for the behavioral sciences, third edition. Lawrence Erlbaum
    Associates, Mahwah, NJ.
    ISBN 0-8058-2223-2
    LoC HA31.3 .A67.2003

    Art

    Social Research Consultants
    University Park, MD USA
    (301) 864-5570
     
    Art Kendall, Jan 28, 2005
    #9
  10. hello art,
    first let me thank you for your interest in my problems with spss. i
    think i somehow solved my problem now - i assessed my interaction by
    disentangling my split-plot-design, i.e. computing 1 single-factor
    within-subjects anova for each group and one between-subjects anova
    (with the covariate) for each treatment condition. i found this approach
    in: Keselman, H.J. & Keselman, J.C. (1993). Analysis of repeated
    measures. In L.K. Edwards (Ed.), Applied analysis of variance in
    behavioral science. New York: Dekker. of course, i adjusted alpha for
    the number of comparisons.
    a remark to your suggestion concerning using a regression approach: in
    fact i first considered to do that, and consulted the book of cohen you
    recommended. however, i really want to compare extreme groups only,
    because my ATI is only supposed for extreme scores on the aptitude. as i
    understand it, the biggest problems with median splits, quartiles, and
    tertiles (which i used) concern a) that you can not be sure that the
    treshold or cut-of you used is the "real" drawing line between the
    groups and b) that you lose a lot of statistical power, i.e. a R of .44
    tuns into .30 when you dichotomize the sample. well, in my case,
    concerning a), i had no other choice due to theretical considerations,
    and b) only makes the test of my assumptions more conservative. but i
    have to admit that i'm not that much into statistics (you should have
    noticed that by now :) ), and maybe this kind of arguing is complete
    nonsense. at the moment i`m reading Waller & Meehl, 1998, Multivariate
    taxometric procedures: Distinguishing types from continua. Thousand
    Oaks, CA.: Sage Publications, i heard from a colleague they discuss when
    and how to dichotomize continous variables. again, thank you very much
    for your helpful remarks; just in case you want to look at my design in
    more detail, i attached the spss-procedure and the main results
    (including a graphic display of the interaction, just as you
    recommended). best regards,
    martin
     
    martin knollmann, Jan 28, 2005
    #10
  11. "why i transformed a continous variable", part II: as i already said, my
    study is concerned with aptitude-treatment-interaction-research (ATI),
    which has been "invented" by cronbach & snow ("aptitudes and
    instructional methods: a handbook of research on interactions", 1977).
    here's what they had to say about extreme group designs and what led me
    to use one: "Measuring an aptitude, dropping cases from the middle of
    the distribution, and randomly dividing cases in each of the tails to
    form treatment groups produces a comparatively powerful design." (p.59).
    And: "A design treating upper and lower thirds of the population, or
    upper and lower quarters, is appreciably more powerful than a study with
    the same N distributed over the full aptitude range (...). With more
    extreme selection of cases, the efficiency of the design is even
    greater" (p. 60).
     
    martin knollmann, Jan 28, 2005
    #11
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.